2024-09-21T06:15:44.182 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T06:15:44.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:15:44.284 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} email: pdonnell@redhat.com first_in_suite: false job_id: '7913546' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Scrub error on inode sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIxe0vejnC5D3xtzH//zZpcvB2UDHhuLt+a/fVk3+UmV9uW5DJoFMnKeLVsDanhZOz1cQIM4fIFb11Uk8GJFmg= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBICDeAFhIv1zoJsCyQ5tkbfrYA7EbMrXELzItru/cYMBLvHI0OV2RvsWfDpMao9aI6vomW5s6Lw2BZs81H6VLzw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T06:15:44.284 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T06:15:44.285 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T06:15:44.285 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T06:15:44.286 INFO:teuthology.task.internal:Checking packages... 2024-09-21T06:15:44.304 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T06:15:44.304 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T06:15:44.304 INFO:teuthology.packaging:ref: None 2024-09-21T06:15:44.304 INFO:teuthology.packaging:tag: None 2024-09-21T06:15:44.304 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:15:44.305 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:15:44.305 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:15:44.447 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T06:15:44.448 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T06:15:44.454 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T06:15:44.454 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T06:15:44.463 INFO:teuthology.task.internal:Saving configuration 2024-09-21T06:15:44.479 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T06:15:44.487 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T06:15:44.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:10:31.662456', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIxe0vejnC5D3xtzH//zZpcvB2UDHhuLt+a/fVk3+UmV9uW5DJoFMnKeLVsDanhZOz1cQIM4fIFb11Uk8GJFmg='} 2024-09-21T06:15:44.537 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:10:31.664417', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBICDeAFhIv1zoJsCyQ5tkbfrYA7EbMrXELzItru/cYMBLvHI0OV2RvsWfDpMao9aI6vomW5s6Lw2BZs81H6VLzw='} 2024-09-21T06:15:44.537 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T06:15:44.548 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T06:15:44.548 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T06:15:44.548 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T06:15:44.611 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea90b88160>, signals=[15]) 2024-09-21T06:15:44.611 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T06:15:44.623 INFO:teuthology.task.internal:Opening connections... 2024-09-21T06:15:44.623 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:15:44.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:15:44.702 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:15:44.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:15:44.776 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T06:15:44.784 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-09-21T06:15:44.803 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-09-21T06:15:44.803 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:15:44.860 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:15:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:15:44.862 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-09-21T06:15:44.883 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-21T06:15:44.901 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-21T06:15:44.901 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-21T06:15:44.959 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-09-21T06:15:44.959 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-09-21T06:15:44.959 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:15:44.960 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:15:44.961 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-21T06:15:44.983 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T06:15:44.994 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T06:15:45.110 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T06:15:45.110 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:15:45.113 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:15:45.130 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T06:15:45.141 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T06:15:45.141 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:15:45.170 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:15:45.277 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T06:15:45.332 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T06:15:45.351 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T06:15:45.351 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-21T06:15:45.351 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-21T06:15:45.351 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:15:45.352 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:15:45.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:45.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T06:15:45.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:45.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T06:15:45.532 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:15:45.532 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T06:15:45.532 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-21T06:15:45.534 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:15:45.534 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T06:15:45.534 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-21T06:15:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:15:45.550 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:15:45.550 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T06:15:45.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum upgrade -y linux-firmware 2024-09-21T06:15:45.552 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:15:45.552 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:15:45.553 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T06:15:45.553 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum upgrade -y linux-firmware 2024-09-21T06:15:47.895 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T06:15:49.414 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:02 2024-09-21T06:15:52.487 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-09-21T06:15:53.515 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-21T06:15:57.847 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-21T06:15:59.289 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 19 kB 00:00 2024-09-21T06:16:00.043 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T06:16:00.099 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-21T06:16:01.626 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-21T06:16:02.512 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T06:16:05.657 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-21T06:16:07.716 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 1 Package 2024-09-21T06:16:07.728 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:16:07.729 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 388 M 2024-09-21T06:16:07.729 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-21T06:16:07.868 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-21T06:16:09.853 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-21T06:16:09.864 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-21T06:16:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-21T06:16:09.864 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-21T06:16:09.864 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-21T06:16:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 1 Package 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 388 M 2024-09-21T06:16:09.865 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-21T06:16:15.086 INFO:teuthology.orchestra.run.smithi150.stdout:linux-firmware-20240905-146.el9.noarch.rpm 76 MB/s | 388 MB 00:05 2024-09-21T06:16:15.086 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:16:15.086 INFO:teuthology.orchestra.run.smithi150.stdout:Total 74 MB/s | 388 MB 00:05 2024-09-21T06:16:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-21T06:16:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-21T06:16:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-21T06:16:21.624 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-21T06:16:21.624 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-21T06:16:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-21T06:16:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:16:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:16:32.562 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:16:32.562 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:16:33.326 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:16:33.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:16:33.326 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-21T06:16:33.326 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T06:16:33.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:16:33.327 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-21T06:16:33.650 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T06:16:33.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:16:33.802 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T06:16:33.802 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T06:16:33.805 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi150.front.sepia.ceph.com to complete... 2024-09-21T06:16:40.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21 06:16:40 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T06:16:40.212 INFO:teuthology.orchestra.run.smithi150.stderr:FINISHED --2024-09-21 06:16:40-- 2024-09-21T06:16:40.212 INFO:teuthology.orchestra.run.smithi150.stderr:Total wall clock time: 6.4s 2024-09-21T06:16:40.212 INFO:teuthology.orchestra.run.smithi150.stderr:Downloaded: 1 files, 343M in 6.1s (56.7 MB/s) 2024-09-21T06:16:40.214 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T06:16:40.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T06:16:42.271 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying... ######################################## 2024-09-21T06:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing... ######################################## 2024-09-21T06:16:42.374 INFO:teuthology.orchestra.run.smithi150.stdout:Updating / installing... 2024-09-21T06:16:50.362 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T06:17:01.265 INFO:teuthology.orchestra.run.smithi037.stdout:linux-firmware-20240905-146.el9.noarch.rpm 7.3 MB/s | 388 MB 00:53 2024-09-21T06:17:01.265 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:17:01.265 INFO:teuthology.orchestra.run.smithi037.stdout:Total 7.3 MB/s | 388 MB 00:53 2024-09-21T06:17:04.544 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-21T06:17:04.627 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-21T06:17:04.627 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-21T06:17:08.042 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-21T06:17:08.042 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-21T06:17:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-21T06:17:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:17:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:17:18.423 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:17:18.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:17:19.262 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:17:19.262 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:17:19.262 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-09-21T06:17:19.263 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T06:17:19.263 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:17:19.263 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-21T06:17:19.596 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:17:19.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:17:19.744 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T06:17:19.745 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T06:17:19.747 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi037.front.sepia.ceph.com to complete... 2024-09-21T06:17:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21 06:17:26 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T06:17:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:FINISHED --2024-09-21 06:17:26-- 2024-09-21T06:17:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:Total wall clock time: 6.3s 2024-09-21T06:17:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:Downloaded: 1 files, 343M in 6.0s (56.7 MB/s) 2024-09-21T06:17:26.123 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:17:26.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T06:17:28.283 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying... ######################################## 2024-09-21T06:17:28.342 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing... ######################################## 2024-09-21T06:17:28.406 INFO:teuthology.orchestra.run.smithi037.stdout:Updating / installing... 2024-09-21T06:17:30.853 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T06:17:30.853 DEBUG:teuthology.orchestra.run.smithi150:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T06:17:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T06:17:30.936 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T06:17:30.937 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T06:17:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T06:17:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T06:17:30.940 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T06:17:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T06:17:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T06:17:30.943 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T06:17:30.944 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T06:17:30.945 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T06:17:30.946 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T06:17:30.947 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T06:17:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T06:17:30.949 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T06:17:30.950 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T06:17:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T06:17:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T06:17:30.953 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T06:17:30.954 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T06:17:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T06:17:30.956 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T06:17:30.957 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T06:17:30.958 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T06:17:30.959 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T06:17:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T06:17:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T06:17:30.963 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T06:17:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T06:17:30.965 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T06:17:30.966 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T06:17:30.967 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T06:17:30.968 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T06:17:30.969 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T06:17:30.970 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T06:17:30.971 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T06:17:30.972 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T06:17:30.973 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T06:17:30.974 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T06:17:30.975 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T06:17:30.976 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T06:17:30.977 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T06:17:30.978 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T06:17:30.979 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T06:17:30.980 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T06:17:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T06:17:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T06:17:30.983 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T06:17:30.984 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T06:17:30.985 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T06:17:30.986 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T06:17:30.987 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T06:17:30.988 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T06:17:30.989 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T06:17:30.990 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T06:17:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T06:17:30.992 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T06:17:30.993 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T06:17:30.994 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T06:17:30.995 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T06:17:30.996 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T06:17:30.997 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T06:17:30.998 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T06:17:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T06:17:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T06:17:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T06:17:31.002 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T06:17:31.003 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T06:17:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T06:17:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T06:17:31.006 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T06:17:31.007 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T06:17:31.008 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T06:17:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T06:17:31.010 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T06:17:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T06:17:31.012 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T06:17:31.013 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T06:17:31.014 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T06:17:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T06:17:31.016 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T06:17:31.017 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T06:17:31.018 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T06:17:31.019 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T06:17:31.020 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T06:17:31.021 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T06:17:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T06:17:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T06:17:31.024 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T06:17:31.025 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T06:17:31.026 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T06:17:31.027 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T06:17:31.028 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T06:17:31.029 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T06:17:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T06:17:31.031 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T06:17:31.032 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T06:17:31.033 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T06:17:31.034 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T06:17:31.035 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T06:17:31.036 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T06:17:31.037 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T06:17:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T06:17:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T06:17:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T06:17:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T06:17:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T06:17:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T06:17:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T06:17:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T06:17:31.046 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T06:17:31.047 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T06:17:31.048 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T06:17:31.049 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T06:17:31.050 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T06:17:31.051 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T06:17:31.052 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T06:17:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T06:17:31.054 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T06:17:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T06:17:31.056 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T06:17:31.057 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T06:17:31.058 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T06:17:31.059 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T06:17:31.060 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T06:17:31.061 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T06:17:31.062 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T06:17:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T06:17:31.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T06:17:31.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T06:17:31.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T06:17:31.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T06:17:31.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T06:17:31.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T06:17:31.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T06:17:31.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T06:17:31.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:17:31.076 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:31.076 DEBUG:teuthology.orchestra.run.smithi150:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T06:17:31.098 INFO:teuthology.orchestra.run.smithi150.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T06:17:31.098 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:for file in vmlinuz System.map config; do 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: fi 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-09-21T06:17:31.099 INFO:teuthology.orchestra.run.smithi150.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:31.100 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-09-21T06:17:31.101 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-09-21T06:17:31.173 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 82.el9 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:17:31.240 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8274670 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:17:31.241 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:17:31.242 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:17:31.242 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:17:31.242 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-09-21T06:17:31.242 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:17:31.242 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:17:31.243 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:17:31.243 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:31.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:17:31.896 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-21T06:17:33.304 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:17:33.337 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-21T06:17:33.339 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T06:17:33.355 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.xvMIFHJHSs 2024-09-21T06:17:33.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xvMIFHJHSs 2024-09-21T06:17:33.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.xvMIFHJHSs 2024-09-21T06:17:33.558 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.xvMIFHJHSs is 6KB 2024-09-21T06:17:33.579 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.xvMIFHJHSs 2024-09-21T06:17:33.594 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:17:33.656 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:17:33.656 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:17:33.656 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:17:33.656 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T06:17:33.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:17:33.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-21T06:17:36.726 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T06:18:03.817 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T06:18:03.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:18:03.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:16.836 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T06:18:16.836 DEBUG:teuthology.orchestra.run.smithi037:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T06:18:16.931 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T06:18:16.932 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T06:18:16.933 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T06:18:16.934 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T06:18:16.935 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T06:18:16.936 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T06:18:16.937 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T06:18:16.938 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T06:18:16.939 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T06:18:16.940 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T06:18:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T06:18:16.942 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T06:18:16.943 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T06:18:16.944 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T06:18:16.945 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T06:18:16.946 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T06:18:16.947 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T06:18:16.948 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T06:18:16.949 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T06:18:16.950 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T06:18:16.951 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T06:18:16.952 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T06:18:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T06:18:16.954 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T06:18:16.955 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T06:18:16.956 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T06:18:16.957 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T06:18:16.958 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T06:18:16.959 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T06:18:16.960 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T06:18:16.961 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T06:18:16.962 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T06:18:16.963 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T06:18:16.964 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T06:18:16.965 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T06:18:16.966 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T06:18:16.967 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T06:18:16.968 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T06:18:16.969 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T06:18:16.970 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T06:18:16.971 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T06:18:16.972 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T06:18:16.973 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T06:18:16.974 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T06:18:16.975 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T06:18:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T06:18:16.977 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T06:18:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T06:18:16.979 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T06:18:16.980 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T06:18:16.981 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T06:18:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T06:18:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T06:18:16.984 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T06:18:16.985 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T06:18:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T06:18:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T06:18:16.988 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T06:18:16.989 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T06:18:16.990 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T06:18:16.992 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T06:18:16.993 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T06:18:16.994 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T06:18:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T06:18:16.996 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T06:18:16.997 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T06:18:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T06:18:16.999 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T06:18:17.000 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T06:18:17.001 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T06:18:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T06:18:17.003 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T06:18:17.004 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T06:18:17.005 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T06:18:17.006 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T06:18:17.007 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T06:18:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T06:18:17.009 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T06:18:17.010 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T06:18:17.011 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T06:18:17.012 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T06:18:17.013 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T06:18:17.014 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T06:18:17.015 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T06:18:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T06:18:17.017 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T06:18:17.018 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T06:18:17.019 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T06:18:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T06:18:17.021 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T06:18:17.022 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T06:18:17.023 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T06:18:17.024 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T06:18:17.025 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T06:18:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T06:18:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T06:18:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T06:18:17.030 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T06:18:17.031 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T06:18:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T06:18:17.033 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T06:18:17.034 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T06:18:17.035 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T06:18:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T06:18:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T06:18:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T06:18:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T06:18:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T06:18:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T06:18:17.042 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T06:18:17.043 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T06:18:17.044 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T06:18:17.045 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T06:18:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T06:18:17.047 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T06:18:17.048 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T06:18:17.049 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T06:18:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T06:18:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T06:18:17.052 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T06:18:17.053 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T06:18:17.054 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T06:18:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T06:18:17.056 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T06:18:17.057 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T06:18:17.058 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T06:18:17.060 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T06:18:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T06:18:17.062 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T06:18:17.062 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T06:18:17.062 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T06:18:17.062 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:18:17.065 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:17.065 DEBUG:teuthology.orchestra.run.smithi037:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:for file in vmlinuz System.map config; do 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:18:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: fi 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-09-21T06:18:17.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-09-21T06:18:17.172 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-09-21T06:18:17.172 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 82.el9 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8274670 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:18:17.173 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:18:17.174 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:18:17.175 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:18:17.175 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:17.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:18:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-09-21T06:18:19.073 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:18:19.102 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-09-21T06:18:19.104 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-21T06:18:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.h5m2I5PDjC 2024-09-21T06:18:19.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.h5m2I5PDjC 2024-09-21T06:18:19.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.h5m2I5PDjC 2024-09-21T06:18:19.319 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.h5m2I5PDjC is 6KB 2024-09-21T06:18:19.339 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.h5m2I5PDjC 2024-09-21T06:18:19.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:18:19.420 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:18:19.420 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:18:19.420 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:18:19.420 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T06:18:19.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:18:19.556 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-09-21T06:18:22.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-21T06:18:31.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:18:31.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:34.383 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-21T06:18:46.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:18:46.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:49.562 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T06:18:49.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:18:49.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:19:07.887 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-21T06:19:16.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:19:16.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:19:35.279 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-21T06:19:46.407 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T06:19:47.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:19:47.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:19:50.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-21T06:20:01.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:20:01.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:20:01.794 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-21T06:20:02.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:20:02.304 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T06:20:02.305 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-21T06:20:02.338 INFO:teuthology.orchestra.run.smithi150.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:20:02.338 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:20:02.339 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:20:02.339 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:20:03.340 DEBUG:teuthology.parallel:result is None 2024-09-21T06:20:05.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:20:05.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:20:05.824 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-21T06:20:06.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:20:06.309 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T06:20:06.310 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-21T06:20:06.343 INFO:teuthology.orchestra.run.smithi037.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:20:06.344 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:20:06.344 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:20:06.344 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:20:07.346 DEBUG:teuthology.parallel:result is None 2024-09-21T06:20:07.347 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T06:20:07.354 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T06:20:07.354 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:20:07.358 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:20:07.397 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T06:20:07.403 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T06:20:07.410 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T06:20:07.410 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:20:07.434 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:20:07.484 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T06:20:07.491 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T06:20:07.491 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:20:07.520 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:20:07.612 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:20:07.619 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:20:07.646 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:20:07.649 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:20:07.652 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T06:20:07.659 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T06:20:07.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:20:07.662 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:20:07.718 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T06:20:07.726 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T06:20:07.727 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:20:07.730 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:20:07.765 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:20:07.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:20:07.943 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:20:08.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:20:08.077 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:20:08.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:20:08.131 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:20:08.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:20:08.271 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:20:08.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:20:08.404 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:20:08.404 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:20:08.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-09-21T06:20:08.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-21T06:20:08.538 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:20:08.572 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:20:09.026 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T06:20:09.033 INFO:teuthology.task.internal:Starting timer... 2024-09-21T06:20:09.034 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T06:20:09.043 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T06:20:09.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-09-21T06:20:09.126 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-09-21T06:20:09.129 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-09-21T06:20:09.210 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-09-21T06:20:09.213 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T06:20:09.213 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-09-21T06:20:09.254 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-09-21T06:20:09.254 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-09-21T06:20:09.295 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-09-21T06:20:09.295 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-09-21T06:20:09.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T06:20:09.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:20:09.351 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T06:20:09.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:20:09.408 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T06:20:09.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:20:09.467 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:20:09.536 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T06:20:09.545 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T06:20:09.554 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T06:20:09.555 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T06:24:46.113 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-09-21T06:24:46.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:24:46.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:24:46.204 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-21T06:24:46.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-21T06:24:46.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:24:46.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:24:46.400 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-21T06:24:46.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T06:24:46.517 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T06:24:46.534 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T06:24:46.534 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:24:46.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:24:46.542 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:24:46.542 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:24:46.600 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:24:46.621 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:24:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:24:46.681 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-09-21T06:24:46.701 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-09-21T06:24:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-09-21T06:24:46.725 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-09-21T06:24:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:24:46.754 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:24:46.765 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:24:46.784 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:24:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:24:46.877 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:24:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-21T06:24:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.886 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:24:46.894 INFO:teuthology.run_tasks:Running task install... 2024-09-21T06:24:46.902 DEBUG:teuthology.task.install:project ceph 2024-09-21T06:24:46.902 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T06:24:46.902 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T06:24:46.902 INFO:teuthology.task.install:Using flavor: default 2024-09-21T06:24:46.908 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T06:24:46.908 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T06:24:46.908 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:24:46.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:46.910 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:24:46.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:47.056 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:24:47.056 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:24:47.072 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:24:47.072 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:24:47.163 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:24:47.163 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:24:47.163 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:24:47.176 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:24:47.176 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:24:47.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:24:47.219 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:24:47.220 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:24:47.232 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:24:47.232 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:24:47.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:24:47.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:24:47.395 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:24:47.408 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:24:47.450 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-09-21T06:24:47.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-09-21T06:24:47.464 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-09-21T06:24:47.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-09-21T06:24:47.718 INFO:teuthology.orchestra.run.smithi150.stdout:45 files removed 2024-09-21T06:24:47.740 INFO:teuthology.orchestra.run.smithi037.stdout:45 files removed 2024-09-21T06:24:47.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:24:47.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:24:48.435 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-21T06:24:48.487 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 274 kB/s | 84 kB 00:00 2024-09-21T06:24:48.796 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 92 kB/s | 24 kB 00:00 2024-09-21T06:24:48.827 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 69 kB/s | 24 kB 00:00 2024-09-21T06:24:49.080 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T06:24:49.127 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 7.3 kB/s | 1.9 kB 00:00 2024-09-21T06:24:49.672 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-21T06:24:52.186 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.3 MB 00:03 2024-09-21T06:24:53.381 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 20 MB 00:02 2024-09-21T06:24:59.216 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-21T06:24:59.628 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-21T06:25:00.808 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 19 kB 00:00 2024-09-21T06:25:01.720 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T06:25:04.312 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T06:25:05.805 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 106 kB/s | 19 kB 00:00 2024-09-21T06:25:06.692 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T06:25:08.309 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T06:25:10.748 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:25:10.748 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:25:10.760 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:25:10.761 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:25:10.762 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:25:10.763 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:25:10.764 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:25:10.767 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:25:10.768 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:25:10.837 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-21T06:25:10.854 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-21T06:25:10.854 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:25:10.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:25:10.856 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:25:10.857 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:25:10.858 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:25:10.859 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:25:10.860 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:25:10.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:25:10.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:25:10.863 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:25:10.864 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:25:10.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:25:10.866 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:25:10.867 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:25:10.868 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-21T06:25:10.869 INFO:teuthology.orchestra.run.smithi037.stdout:Install 149 Packages 2024-09-21T06:25:10.870 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 8 Packages 2024-09-21T06:25:10.870 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:25:10.870 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 228 M 2024-09-21T06:25:10.870 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-21T06:25:11.661 INFO:teuthology.orchestra.run.smithi037.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-09-21T06:25:12.936 INFO:teuthology.orchestra.run.smithi037.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 685 kB/s | 873 kB 00:01 2024-09-21T06:25:12.998 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T06:25:13.186 INFO:teuthology.orchestra.run.smithi037.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 593 kB/s | 148 kB 00:00 2024-09-21T06:25:13.390 INFO:teuthology.orchestra.run.smithi037.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.8 MB/s | 5.5 MB 00:01 2024-09-21T06:25:13.603 INFO:teuthology.orchestra.run.smithi037.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.1 MB/s | 1.5 MB 00:00 2024-09-21T06:25:14.304 INFO:teuthology.orchestra.run.smithi037.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.1 MB/s | 5.0 MB 00:00 2024-09-21T06:25:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:25:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:25:15.365 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:25:15.366 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:25:15.367 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:25:15.368 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:25:15.369 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:25:15.372 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:25:15.373 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:25:15.437 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:25:15.463 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:25:15.464 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:25:15.465 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-21T06:25:15.466 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:25:15.467 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:25:15.468 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:25:15.469 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:25:15.470 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:25:15.471 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:25:15.472 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:25:15.473 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:25:15.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:25:15.475 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-09-21T06:25:15.476 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:Install 149 Packages 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 8 Packages 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 228 M 2024-09-21T06:25:15.477 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-21T06:25:15.637 INFO:teuthology.orchestra.run.smithi037.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 955 kB/s | 2.3 MB 00:02 2024-09-21T06:25:16.062 INFO:teuthology.orchestra.run.smithi037.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 10 MB/s | 18 MB 00:01 2024-09-21T06:25:16.146 INFO:teuthology.orchestra.run.smithi037.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 304 kB/s | 25 kB 00:00 2024-09-21T06:25:16.466 INFO:teuthology.orchestra.run.smithi150.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-09-21T06:25:17.325 INFO:teuthology.orchestra.run.smithi150.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.0 MB/s | 873 kB 00:00 2024-09-21T06:25:18.042 INFO:teuthology.orchestra.run.smithi150.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 207 kB/s | 148 kB 00:00 2024-09-21T06:25:18.796 INFO:teuthology.orchestra.run.smithi037.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T06:25:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.4 MB/s | 226 kB 00:00 2024-09-21T06:25:18.947 INFO:teuthology.orchestra.run.smithi037.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 465 kB/s | 27 kB 00:00 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi150.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 5.5 MB 00:02 2024-09-21T06:25:18.997 INFO:teuthology.orchestra.run.smithi037.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 637 kB/s | 32 kB 00:00 2024-09-21T06:25:19.109 INFO:teuthology.orchestra.run.smithi150.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-09-21T06:25:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.0 MB/s | 766 kB 00:00 2024-09-21T06:25:19.197 INFO:teuthology.orchestra.run.smithi037.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T06:25:19.234 INFO:teuthology.orchestra.run.smithi150.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 5.9 MB/s | 1.5 MB 00:00 2024-09-21T06:25:19.264 INFO:teuthology.orchestra.run.smithi037.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T06:25:19.348 INFO:teuthology.orchestra.run.smithi037.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 6.0 MB/s | 507 kB 00:00 2024-09-21T06:25:19.915 INFO:teuthology.orchestra.run.smithi037.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.3 MB/s | 5.3 MB 00:00 2024-09-21T06:25:20.007 INFO:teuthology.orchestra.run.smithi037.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 495 kB/s | 45 kB 00:00 2024-09-21T06:25:20.082 INFO:teuthology.orchestra.run.smithi037.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T06:25:20.174 INFO:teuthology.orchestra.run.smithi037.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.7 MB/s | 161 kB 00:00 2024-09-21T06:25:20.226 INFO:teuthology.orchestra.run.smithi150.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 4.4 MB/s | 5.0 MB 00:01 2024-09-21T06:25:20.258 INFO:teuthology.orchestra.run.smithi037.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T06:25:20.341 INFO:teuthology.orchestra.run.smithi037.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.6 MB/s | 307 kB 00:00 2024-09-21T06:25:20.425 INFO:teuthology.orchestra.run.smithi037.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.2 MB/s | 99 kB 00:00 2024-09-21T06:25:20.508 INFO:teuthology.orchestra.run.smithi037.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.0 MB/s | 86 kB 00:00 2024-09-21T06:25:20.825 INFO:teuthology.orchestra.run.smithi037.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 9.8 MB/s | 3.1 MB 00:00 2024-09-21T06:25:20.892 INFO:teuthology.orchestra.run.smithi037.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T06:25:20.959 INFO:teuthology.orchestra.run.smithi037.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 516 kB/s | 34 kB 00:00 2024-09-21T06:25:21.034 INFO:teuthology.orchestra.run.smithi037.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.0 MB/s | 154 kB 00:00 2024-09-21T06:25:21.168 INFO:teuthology.orchestra.run.smithi150.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 11 MB/s | 10 MB 00:00 2024-09-21T06:25:21.235 INFO:teuthology.orchestra.run.smithi150.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 378 kB/s | 25 kB 00:00 2024-09-21T06:25:21.276 INFO:teuthology.orchestra.run.smithi037.stdout:(30/157): ceph-radosgw-19.3.0-5120.g18432218.el 1.8 MB/s | 10 MB 00:05 2024-09-21T06:25:21.885 INFO:teuthology.orchestra.run.smithi037.stdout:(31/157): ceph-common-19.3.0-5120.g18432218.el9 1.9 MB/s | 20 MB 00:10 2024-09-21T06:25:22.002 INFO:teuthology.orchestra.run.smithi037.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.4 MB/s | 292 kB 00:00 2024-09-21T06:25:22.043 INFO:teuthology.orchestra.run.smithi150.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 6.3 MB/s | 18 MB 00:02 2024-09-21T06:25:22.069 INFO:teuthology.orchestra.run.smithi037.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 744 kB/s | 49 kB 00:00 2024-09-21T06:25:22.119 INFO:teuthology.orchestra.run.smithi150.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.0 MB/s | 226 kB 00:00 2024-09-21T06:25:22.152 INFO:teuthology.orchestra.run.smithi037.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 198 kB/s | 16 kB 00:00 2024-09-21T06:25:22.219 INFO:teuthology.orchestra.run.smithi150.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 271 kB/s | 27 kB 00:00 2024-09-21T06:25:22.286 INFO:teuthology.orchestra.run.smithi150.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 476 kB/s | 32 kB 00:00 2024-09-21T06:25:22.344 INFO:teuthology.orchestra.run.smithi037.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 11 MB/s | 14 MB 00:01 2024-09-21T06:25:22.369 INFO:teuthology.orchestra.run.smithi037.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.3 MB/s | 285 kB 00:00 2024-09-21T06:25:22.469 INFO:teuthology.orchestra.run.smithi150.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.1 MB/s | 766 kB 00:00 2024-09-21T06:25:22.470 INFO:teuthology.orchestra.run.smithi037.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 184 kB/s | 18 kB 00:00 2024-09-21T06:25:22.520 INFO:teuthology.orchestra.run.smithi037.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.4 MB/s | 784 kB 00:00 2024-09-21T06:25:22.578 INFO:teuthology.orchestra.run.smithi150.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.4 MB/s | 158 kB 00:00 2024-09-21T06:25:22.629 INFO:teuthology.orchestra.run.smithi037.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-21T06:25:22.678 INFO:teuthology.orchestra.run.smithi150.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.2 MB/s | 127 kB 00:00 2024-09-21T06:25:22.754 INFO:teuthology.orchestra.run.smithi037.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-21T06:25:22.812 INFO:teuthology.orchestra.run.smithi037.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-09-21T06:25:22.913 INFO:teuthology.orchestra.run.smithi037.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-21T06:25:22.963 INFO:teuthology.orchestra.run.smithi037.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T06:25:23.022 INFO:teuthology.orchestra.run.smithi037.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T06:25:23.047 INFO:teuthology.orchestra.run.smithi037.stdout:(45/157): libquadmath-11.5.0-2.el9.x86_64.rpm 457 kB/s | 191 kB 00:00 2024-09-21T06:25:23.072 INFO:teuthology.orchestra.run.smithi037.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:25:23.106 INFO:teuthology.orchestra.run.smithi037.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T06:25:23.179 INFO:teuthology.orchestra.run.smithi150.stdout:(17/157): ceph-test-19.3.0-5120.g18432218.el9.x 21 MB/s | 40 MB 00:01 2024-09-21T06:25:23.212 INFO:teuthology.orchestra.run.smithi150.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 950 kB/s | 507 kB 00:00 2024-09-21T06:25:23.239 INFO:teuthology.orchestra.run.smithi037.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-21T06:25:23.290 INFO:teuthology.orchestra.run.smithi037.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-21T06:25:23.296 INFO:teuthology.orchestra.run.smithi150.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 546 kB/s | 45 kB 00:00 2024-09-21T06:25:23.315 INFO:teuthology.orchestra.run.smithi037.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-21T06:25:23.340 INFO:teuthology.orchestra.run.smithi037.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-21T06:25:23.413 INFO:teuthology.orchestra.run.smithi150.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 23 MB/s | 5.3 MB 00:00 2024-09-21T06:25:23.415 INFO:teuthology.orchestra.run.smithi037.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-09-21T06:25:23.450 INFO:teuthology.orchestra.run.smithi037.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-21T06:25:23.522 INFO:teuthology.orchestra.run.smithi150.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 659 kB/s | 148 kB 00:00 2024-09-21T06:25:23.541 INFO:teuthology.orchestra.run.smithi037.stdout:(54/157): ceph-mgr-diskprediction-local-19.3.0- 3.3 MB/s | 7.4 MB 00:02 2024-09-21T06:25:23.547 INFO:teuthology.orchestra.run.smithi150.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.2 MB/s | 161 kB 00:00 2024-09-21T06:25:23.567 INFO:teuthology.orchestra.run.smithi037.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-21T06:25:23.597 INFO:teuthology.orchestra.run.smithi150.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 4.2 MB/s | 321 kB 00:00 2024-09-21T06:25:23.608 INFO:teuthology.orchestra.run.smithi037.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-21T06:25:23.634 INFO:teuthology.orchestra.run.smithi037.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T06:25:23.645 INFO:teuthology.orchestra.run.smithi150.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 3.1 MB/s | 307 kB 00:00 2024-09-21T06:25:23.673 INFO:teuthology.orchestra.run.smithi150.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.3 MB/s | 99 kB 00:00 2024-09-21T06:25:23.676 INFO:teuthology.orchestra.run.smithi037.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T06:25:23.698 INFO:teuthology.orchestra.run.smithi150.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.6 MB/s | 86 kB 00:00 2024-09-21T06:25:23.701 INFO:teuthology.orchestra.run.smithi037.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-21T06:25:23.748 INFO:teuthology.orchestra.run.smithi150.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T06:25:23.784 INFO:teuthology.orchestra.run.smithi037.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-21T06:25:23.799 INFO:teuthology.orchestra.run.smithi150.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 686 kB/s | 34 kB 00:00 2024-09-21T06:25:23.835 INFO:teuthology.orchestra.run.smithi037.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.3 MB/s | 581 kB 00:00 2024-09-21T06:25:23.849 INFO:teuthology.orchestra.run.smithi150.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T06:25:23.860 INFO:teuthology.orchestra.run.smithi037.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-21T06:25:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-09-21T06:25:23.969 INFO:teuthology.orchestra.run.smithi037.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T06:25:23.994 INFO:teuthology.orchestra.run.smithi037.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-21T06:25:24.019 INFO:teuthology.orchestra.run.smithi037.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T06:25:24.099 INFO:teuthology.orchestra.run.smithi150.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 7.3 MB/s | 3.1 MB 00:00 2024-09-21T06:25:24.261 INFO:teuthology.orchestra.run.smithi037.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-09-21T06:25:24.333 INFO:teuthology.orchestra.run.smithi150.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 29 MB/s | 14 MB 00:00 2024-09-21T06:25:24.391 INFO:teuthology.orchestra.run.smithi150.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.0 MB/s | 292 kB 00:00 2024-09-21T06:25:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 992 kB/s | 49 kB 00:00 2024-09-21T06:25:24.492 INFO:teuthology.orchestra.run.smithi150.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T06:25:24.542 INFO:teuthology.orchestra.run.smithi150.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T06:25:24.617 INFO:teuthology.orchestra.run.smithi150.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T06:25:24.667 INFO:teuthology.orchestra.run.smithi150.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 369 kB/s | 18 kB 00:00 2024-09-21T06:25:24.720 INFO:teuthology.orchestra.run.smithi037.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-09-21T06:25:24.776 INFO:teuthology.orchestra.run.smithi150.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 738 kB/s | 80 kB 00:00 2024-09-21T06:25:24.918 INFO:teuthology.orchestra.run.smithi150.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 9.0 MB/s | 7.4 MB 00:00 2024-09-21T06:25:24.995 INFO:teuthology.orchestra.run.smithi037.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.2 MB/s | 910 kB 00:00 2024-09-21T06:25:25.010 INFO:teuthology.orchestra.run.smithi150.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-21T06:25:25.070 INFO:teuthology.orchestra.run.smithi037.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-21T06:25:25.094 INFO:teuthology.orchestra.run.smithi150.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T06:25:25.119 INFO:teuthology.orchestra.run.smithi150.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 953 kB/s | 191 kB 00:00 2024-09-21T06:25:25.162 INFO:teuthology.orchestra.run.smithi037.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T06:25:25.169 INFO:teuthology.orchestra.run.smithi150.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T06:25:25.253 INFO:teuthology.orchestra.run.smithi150.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-21T06:25:25.303 INFO:teuthology.orchestra.run.smithi150.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:25:25.387 INFO:teuthology.orchestra.run.smithi150.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T06:25:25.437 INFO:teuthology.orchestra.run.smithi150.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-09-21T06:25:25.496 INFO:teuthology.orchestra.run.smithi037.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.3 MB/s | 4.9 MB 00:01 2024-09-21T06:25:25.530 INFO:teuthology.orchestra.run.smithi037.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-21T06:25:25.562 INFO:teuthology.orchestra.run.smithi150.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-21T06:25:25.563 INFO:teuthology.orchestra.run.smithi037.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:25:25.596 INFO:teuthology.orchestra.run.smithi150.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-21T06:25:25.605 INFO:teuthology.orchestra.run.smithi037.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T06:25:25.621 INFO:teuthology.orchestra.run.smithi150.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-21T06:25:25.697 INFO:teuthology.orchestra.run.smithi037.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T06:25:25.780 INFO:teuthology.orchestra.run.smithi037.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-21T06:25:25.814 INFO:teuthology.orchestra.run.smithi037.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 773 kB/s | 26 kB 00:00 2024-09-21T06:25:25.855 INFO:teuthology.orchestra.run.smithi150.stdout:(51/157): ceph-common-19.3.0-5120.g18432218.el9 2.1 MB/s | 20 MB 00:09 2024-09-21T06:25:25.872 INFO:teuthology.orchestra.run.smithi150.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 79 kB/s | 22 kB 00:00 2024-09-21T06:25:25.884 INFO:teuthology.orchestra.run.smithi037.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 276 kB/s | 19 kB 00:00 2024-09-21T06:25:25.889 INFO:teuthology.orchestra.run.smithi150.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 983 kB/s | 33 kB 00:00 2024-09-21T06:25:25.922 INFO:teuthology.orchestra.run.smithi150.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-09-21T06:25:25.939 INFO:teuthology.orchestra.run.smithi150.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-21T06:25:25.956 INFO:teuthology.orchestra.run.smithi037.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-21T06:25:25.957 INFO:teuthology.orchestra.run.smithi150.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-09-21T06:25:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:25:25.998 INFO:teuthology.orchestra.run.smithi037.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 835 kB/s | 35 kB 00:00 2024-09-21T06:25:26.015 INFO:teuthology.orchestra.run.smithi150.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T06:25:26.032 INFO:teuthology.orchestra.run.smithi150.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T06:25:26.074 INFO:teuthology.orchestra.run.smithi150.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T06:25:26.141 INFO:teuthology.orchestra.run.smithi150.stdout:(61/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-21T06:25:26.158 INFO:teuthology.orchestra.run.smithi150.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-21T06:25:26.183 INFO:teuthology.orchestra.run.smithi150.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T06:25:26.217 INFO:teuthology.orchestra.run.smithi150.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-21T06:25:26.242 INFO:teuthology.orchestra.run.smithi150.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T06:25:26.267 INFO:teuthology.orchestra.run.smithi150.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T06:25:26.292 INFO:teuthology.orchestra.run.smithi150.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-21T06:25:26.384 INFO:teuthology.orchestra.run.smithi150.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T06:25:26.451 INFO:teuthology.orchestra.run.smithi150.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 24 MB/s | 4.9 MB 00:00 2024-09-21T06:25:26.485 INFO:teuthology.orchestra.run.smithi150.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-21T06:25:26.511 INFO:teuthology.orchestra.run.smithi150.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T06:25:26.577 INFO:teuthology.orchestra.run.smithi150.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-21T06:25:26.611 INFO:teuthology.orchestra.run.smithi150.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-21T06:25:26.636 INFO:teuthology.orchestra.run.smithi150.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T06:25:26.687 INFO:teuthology.orchestra.run.smithi150.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T06:25:26.770 INFO:teuthology.orchestra.run.smithi150.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-21T06:25:26.795 INFO:teuthology.orchestra.run.smithi150.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T06:25:26.829 INFO:teuthology.orchestra.run.smithi150.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T06:25:26.846 INFO:teuthology.orchestra.run.smithi150.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T06:25:26.863 INFO:teuthology.orchestra.run.smithi150.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 579 kB/s | 19 kB 00:00 2024-09-21T06:25:26.910 INFO:teuthology.orchestra.run.smithi150.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-09-21T06:25:26.930 INFO:teuthology.orchestra.run.smithi150.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-09-21T06:25:26.973 INFO:teuthology.orchestra.run.smithi037.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-09-21T06:25:27.017 INFO:teuthology.orchestra.run.smithi150.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T06:25:27.047 INFO:teuthology.orchestra.run.smithi150.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T06:25:27.072 INFO:teuthology.orchestra.run.smithi150.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.0 MB/s | 222 kB 00:00 2024-09-21T06:25:27.073 INFO:teuthology.orchestra.run.smithi037.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-21T06:25:27.097 INFO:teuthology.orchestra.run.smithi150.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:25:27.115 INFO:teuthology.orchestra.run.smithi150.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T06:25:27.148 INFO:teuthology.orchestra.run.smithi150.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-21T06:25:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-21T06:25:27.173 INFO:teuthology.orchestra.run.smithi150.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T06:25:27.207 INFO:teuthology.orchestra.run.smithi150.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-21T06:25:27.216 INFO:teuthology.orchestra.run.smithi037.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:25:27.224 INFO:teuthology.orchestra.run.smithi150.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:25:27.241 INFO:teuthology.orchestra.run.smithi150.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T06:25:27.258 INFO:teuthology.orchestra.run.smithi037.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T06:25:27.274 INFO:teuthology.orchestra.run.smithi150.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-21T06:25:27.366 INFO:teuthology.orchestra.run.smithi150.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-21T06:25:27.383 INFO:teuthology.orchestra.run.smithi150.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-21T06:25:27.391 INFO:teuthology.orchestra.run.smithi037.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-21T06:25:27.417 INFO:teuthology.orchestra.run.smithi150.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:25:27.434 INFO:teuthology.orchestra.run.smithi150.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-21T06:25:27.450 INFO:teuthology.orchestra.run.smithi150.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T06:25:27.476 INFO:teuthology.orchestra.run.smithi150.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-21T06:25:27.493 INFO:teuthology.orchestra.run.smithi150.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T06:25:27.518 INFO:teuthology.orchestra.run.smithi150.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.3 MB/s | 96 kB 00:00 2024-09-21T06:25:27.535 INFO:teuthology.orchestra.run.smithi150.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T06:25:27.552 INFO:teuthology.orchestra.run.smithi150.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-21T06:25:27.636 INFO:teuthology.orchestra.run.smithi150.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-21T06:25:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:(88/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.7 MB/s | 6.0 MB 00:02 2024-09-21T06:25:27.725 INFO:teuthology.orchestra.run.smithi037.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 476 kB/s | 159 kB 00:00 2024-09-21T06:25:27.828 INFO:teuthology.orchestra.run.smithi150.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T06:25:27.834 INFO:teuthology.orchestra.run.smithi037.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-09-21T06:25:27.859 INFO:teuthology.orchestra.run.smithi037.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-09-21T06:25:27.884 INFO:teuthology.orchestra.run.smithi037.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:25:27.887 INFO:teuthology.orchestra.run.smithi150.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-21T06:25:27.910 INFO:teuthology.orchestra.run.smithi037.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-21T06:25:27.920 INFO:teuthology.orchestra.run.smithi150.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-09-21T06:25:27.935 INFO:teuthology.orchestra.run.smithi037.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-21T06:25:27.960 INFO:teuthology.orchestra.run.smithi037.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T06:25:27.962 INFO:teuthology.orchestra.run.smithi150.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.3 MB/s | 308 kB 00:00 2024-09-21T06:25:27.985 INFO:teuthology.orchestra.run.smithi037.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:25:28.002 INFO:teuthology.orchestra.run.smithi037.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 862 kB/s | 36 kB 00:00 2024-09-21T06:25:28.012 INFO:teuthology.orchestra.run.smithi150.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 539 kB/s | 26 kB 00:00 2024-09-21T06:25:28.019 INFO:teuthology.orchestra.run.smithi037.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T06:25:28.048 INFO:teuthology.orchestra.run.smithi150.stdout:(110/157): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-09-21T06:25:28.053 INFO:teuthology.orchestra.run.smithi037.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:25:28.071 INFO:teuthology.orchestra.run.smithi150.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-09-21T06:25:28.078 INFO:teuthology.orchestra.run.smithi037.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-09-21T06:25:28.096 INFO:teuthology.orchestra.run.smithi150.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-21T06:25:28.120 INFO:teuthology.orchestra.run.smithi037.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-21T06:25:28.122 INFO:teuthology.orchestra.run.smithi150.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T06:25:28.162 INFO:teuthology.orchestra.run.smithi037.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-09-21T06:25:28.172 INFO:teuthology.orchestra.run.smithi150.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T06:25:28.196 INFO:teuthology.orchestra.run.smithi037.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-09-21T06:25:28.206 INFO:teuthology.orchestra.run.smithi150.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T06:25:28.231 INFO:teuthology.orchestra.run.smithi150.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-09-21T06:25:28.256 INFO:teuthology.orchestra.run.smithi150.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-21T06:25:28.263 INFO:teuthology.orchestra.run.smithi037.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-21T06:25:28.281 INFO:teuthology.orchestra.run.smithi150.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T06:25:28.307 INFO:teuthology.orchestra.run.smithi150.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T06:25:28.321 INFO:teuthology.orchestra.run.smithi037.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T06:25:28.340 INFO:teuthology.orchestra.run.smithi150.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T06:25:28.355 INFO:teuthology.orchestra.run.smithi037.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-09-21T06:25:28.365 INFO:teuthology.orchestra.run.smithi150.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T06:25:28.391 INFO:teuthology.orchestra.run.smithi150.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T06:25:28.397 INFO:teuthology.orchestra.run.smithi037.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-21T06:25:28.424 INFO:teuthology.orchestra.run.smithi150.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T06:25:28.430 INFO:teuthology.orchestra.run.smithi037.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-21T06:25:28.441 INFO:teuthology.orchestra.run.smithi150.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 643 kB/s | 11 kB 00:00 2024-09-21T06:25:28.456 INFO:teuthology.orchestra.run.smithi037.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T06:25:28.458 INFO:teuthology.orchestra.run.smithi150.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T06:25:28.534 INFO:teuthology.orchestra.run.smithi150.stdout:(126/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-21T06:25:28.550 INFO:teuthology.orchestra.run.smithi150.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-09-21T06:25:28.567 INFO:teuthology.orchestra.run.smithi150.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-09-21T06:25:28.585 INFO:teuthology.orchestra.run.smithi150.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 583 kB/s | 20 kB 00:00 2024-09-21T06:25:28.601 INFO:teuthology.orchestra.run.smithi150.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T06:25:28.619 INFO:teuthology.orchestra.run.smithi037.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 298 kB/s | 49 kB 00:00 2024-09-21T06:25:28.626 INFO:teuthology.orchestra.run.smithi150.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:25:28.644 INFO:teuthology.orchestra.run.smithi150.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T06:25:28.681 INFO:teuthology.orchestra.run.smithi037.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-21T06:25:28.686 INFO:teuthology.orchestra.run.smithi150.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-21T06:25:28.711 INFO:teuthology.orchestra.run.smithi150.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T06:25:28.715 INFO:teuthology.orchestra.run.smithi037.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T06:25:28.736 INFO:teuthology.orchestra.run.smithi150.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T06:25:28.770 INFO:teuthology.orchestra.run.smithi150.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 5.0 MB/s | 1.0 MB 00:00 2024-09-21T06:25:28.782 INFO:teuthology.orchestra.run.smithi037.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T06:25:28.795 INFO:teuthology.orchestra.run.smithi150.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-09-21T06:25:28.820 INFO:teuthology.orchestra.run.smithi150.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-21T06:25:28.824 INFO:teuthology.orchestra.run.smithi037.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T06:25:28.841 INFO:teuthology.orchestra.run.smithi037.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T06:25:28.846 INFO:teuthology.orchestra.run.smithi150.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:25:28.858 INFO:teuthology.orchestra.run.smithi037.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T06:25:28.875 INFO:teuthology.orchestra.run.smithi037.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 860 kB/s | 14 kB 00:00 2024-09-21T06:25:28.896 INFO:teuthology.orchestra.run.smithi150.stdout:(140/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-21T06:25:28.908 INFO:teuthology.orchestra.run.smithi037.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-21T06:25:28.922 INFO:teuthology.orchestra.run.smithi150.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-09-21T06:25:28.942 INFO:teuthology.orchestra.run.smithi037.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T06:25:28.947 INFO:teuthology.orchestra.run.smithi150.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T06:25:28.972 INFO:teuthology.orchestra.run.smithi150.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T06:25:28.984 INFO:teuthology.orchestra.run.smithi037.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-09-21T06:25:28.997 INFO:teuthology.orchestra.run.smithi150.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T06:25:29.017 INFO:teuthology.orchestra.run.smithi037.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-21T06:25:29.023 INFO:teuthology.orchestra.run.smithi150.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 796 kB/s | 20 kB 00:00 2024-09-21T06:25:29.048 INFO:teuthology.orchestra.run.smithi150.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T06:25:29.085 INFO:teuthology.orchestra.run.smithi037.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-21T06:25:29.106 INFO:teuthology.orchestra.run.smithi150.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T06:25:29.110 INFO:teuthology.orchestra.run.smithi037.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.8 MB/s | 144 kB 00:00 2024-09-21T06:25:29.132 INFO:teuthology.orchestra.run.smithi150.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-21T06:25:29.135 INFO:teuthology.orchestra.run.smithi037.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T06:25:29.160 INFO:teuthology.orchestra.run.smithi037.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-21T06:25:29.182 INFO:teuthology.orchestra.run.smithi150.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T06:25:29.186 INFO:teuthology.orchestra.run.smithi037.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-21T06:25:29.211 INFO:teuthology.orchestra.run.smithi037.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 790 kB/s | 19 kB 00:00 2024-09-21T06:25:29.237 INFO:teuthology.orchestra.run.smithi037.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 797 kB/s | 20 kB 00:00 2024-09-21T06:25:29.295 INFO:teuthology.orchestra.run.smithi037.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-21T06:25:29.320 INFO:teuthology.orchestra.run.smithi037.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:25:29.371 INFO:teuthology.orchestra.run.smithi037.stdout:(131/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-09-21T06:25:29.396 INFO:teuthology.orchestra.run.smithi037.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-21T06:25:29.421 INFO:teuthology.orchestra.run.smithi037.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:25:29.446 INFO:teuthology.orchestra.run.smithi037.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T06:25:29.457 INFO:teuthology.orchestra.run.smithi150.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 9.7 MB/s | 3.4 MB 00:00 2024-09-21T06:25:29.471 INFO:teuthology.orchestra.run.smithi037.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-21T06:25:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:25:29.522 INFO:teuthology.orchestra.run.smithi037.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T06:25:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T06:25:29.573 INFO:teuthology.orchestra.run.smithi037.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:25:29.598 INFO:teuthology.orchestra.run.smithi037.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T06:25:29.615 INFO:teuthology.orchestra.run.smithi037.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-21T06:25:29.640 INFO:teuthology.orchestra.run.smithi037.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-21T06:25:29.665 INFO:teuthology.orchestra.run.smithi037.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T06:25:29.674 INFO:teuthology.orchestra.run.smithi150.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.8 MB/s | 3.2 MB 00:00 2024-09-21T06:25:29.716 INFO:teuthology.orchestra.run.smithi037.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T06:25:29.716 INFO:teuthology.orchestra.run.smithi150.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-09-21T06:25:29.741 INFO:teuthology.orchestra.run.smithi037.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-21T06:25:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:25:29.800 INFO:teuthology.orchestra.run.smithi150.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 326 kB/s | 41 kB 00:00 2024-09-21T06:25:29.833 INFO:teuthology.orchestra.run.smithi037.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-09-21T06:25:29.834 INFO:teuthology.orchestra.run.smithi150.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-21T06:25:29.891 INFO:teuthology.orchestra.run.smithi037.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T06:25:29.909 INFO:teuthology.orchestra.run.smithi150.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2024-09-21T06:25:30.150 INFO:teuthology.orchestra.run.smithi037.stdout:(149/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T06:25:30.251 INFO:teuthology.orchestra.run.smithi150.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.7 MB/s | 2.4 MB 00:00 2024-09-21T06:25:30.284 INFO:teuthology.orchestra.run.smithi037.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 3.4 MB 00:00 2024-09-21T06:25:30.309 INFO:teuthology.orchestra.run.smithi150.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 1.4 MB 00:00 2024-09-21T06:25:30.315 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:25:30.315 INFO:teuthology.orchestra.run.smithi150.stdout:Total 15 MB/s | 228 MB 00:14 2024-09-21T06:25:30.434 INFO:teuthology.orchestra.run.smithi037.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 8.2 MB/s | 2.3 MB 00:00 2024-09-21T06:25:30.484 INFO:teuthology.orchestra.run.smithi037.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 819 kB/s | 41 kB 00:00 2024-09-21T06:25:30.543 INFO:teuthology.orchestra.run.smithi037.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T06:25:30.584 INFO:teuthology.orchestra.run.smithi037.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.0 MB/s | 1.8 MB 00:00 2024-09-21T06:25:30.677 INFO:teuthology.orchestra.run.smithi037.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-21T06:25:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-21T06:25:32.444 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-21T06:25:32.633 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-21T06:25:32.633 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-21T06:25:33.394 INFO:teuthology.orchestra.run.smithi037.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.9 MB/s | 16 MB 00:05 2024-09-21T06:25:33.399 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:25:33.399 INFO:teuthology.orchestra.run.smithi037.stdout:Total 10 MB/s | 228 MB 00:22 2024-09-21T06:25:35.038 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-21T06:25:35.039 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-21T06:25:35.529 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-21T06:25:35.733 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-21T06:25:35.733 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-21T06:25:38.293 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-21T06:25:38.294 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-21T06:25:41.568 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-21T06:25:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:25:41.905 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:25:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:25:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:25:42.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:25:42.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:25:42.625 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:25:42.724 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:25:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:25:42.891 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:25:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:25:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:25:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:25:43.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:25:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:25:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:25:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:25:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:25:43.638 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:25:43.665 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:25:43.706 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:25:43.917 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:25:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:25:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:44.598 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:45.031 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:25:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-21T06:25:45.337 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:25:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:25:45.471 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:25:45.473 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:25:45.524 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:25:45.526 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:25:45.572 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:25:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:25:45.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:25:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:25:45.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:25:45.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:25:45.708 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:25:45.749 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:25:45.795 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:25:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:25:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:25:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:25:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:25:46.065 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:25:46.109 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:25:46.137 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:25:46.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:25:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:25:46.281 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:25:46.291 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:25:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:25:46.538 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:25:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:25:47.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:25:47.342 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:25:47.387 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:25:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:25:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:25:47.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:25:47.727 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:25:47.736 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:25:47.890 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:25:47.902 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:25:47.983 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:25:48.015 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:48.066 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:25:48.107 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:25:48.140 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:48.143 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:25:48.209 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:25:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:25:48.296 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:25:48.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:25:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:25:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:25:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:25:48.578 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:25:48.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:25:48.612 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:25:48.657 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:25:48.723 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:25:48.751 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:25:48.767 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:25:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:25:48.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:25:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:25:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:25:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:25:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:25:48.950 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:25:48.978 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:25:49.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:25:49.137 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:25:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:25:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:25:49.204 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:25:49.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:25:49.246 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:25:49.284 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:25:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:25:49.324 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:25:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:25:49.363 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:25:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:25:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:25:49.429 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:25:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:25:49.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:25:49.512 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:25:49.524 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:25:49.563 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:25:49.582 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:25:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:25:49.678 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:25:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:25:49.782 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:25:49.857 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:25:49.902 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:25:49.927 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:25:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:25:50.605 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:25:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:25:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:25:50.790 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:25:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:25:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:25:50.906 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:25:50.923 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:25:50.956 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:25:51.071 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:25:51.087 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:25:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:25:51.156 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:25:51.191 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:25:51.276 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:25:51.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:25:51.333 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:25:51.437 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:25:51.474 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:25:51.511 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:25:51.569 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:25:51.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:25:51.604 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:25:51.626 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:25:51.672 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:25:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:25:51.722 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:25:51.904 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:25:51.963 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:25:52.006 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:25:52.144 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:25:52.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:25:52.223 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:25:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:25:52.279 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:25:52.391 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:25:52.506 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:25:52.555 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:25:52.596 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:25:52.660 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:25:52.667 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:25:52.708 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:25:52.767 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:25:52.809 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:25:52.844 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:25:52.877 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:25:52.921 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:25:52.953 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:25:53.063 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:25:53.129 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:25:53.165 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:25:53.291 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:25:53.353 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:25:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:25:54.285 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:25:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:25:54.346 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:25:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:25:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:25:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:25:54.498 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:25:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:25:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:25:54.708 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:25:54.747 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:25:54.757 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:25:54.790 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:25:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:25:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:25:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:25:55.004 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:25:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:25:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:25:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:25:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:25:55.667 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:25:55.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:25:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:25:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:25:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:25:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:25:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:25:59.107 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:25:59.107 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:25:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:25:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:25:59.179 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:25:59.209 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:25:59.300 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:25:59.327 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:26:01.911 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:26:01.948 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:26:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:26:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-09-21T06:26:12.453 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-09-21T06:26:12.454 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-09-21T06:26:12.454 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-09-21T06:26:12.454 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:12.658 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:26:12.694 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:26:12.695 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:12.695 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:26:12.695 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:14.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:26:14.453 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:14.734 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:26:14.785 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:26:14.785 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:14.786 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:26:14.786 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:26:14.786 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:26:14.786 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:15.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:26:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:26:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:26:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:26:15.353 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:26:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:15.503 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:26:15.570 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:26:15.634 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:26:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:26:15.941 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:26:16.029 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-09-21T06:26:16.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:26:16.329 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:26:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:26:16.367 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:16.367 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:26:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:16.529 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:26:16.607 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:26:16.647 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:26:16.700 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:26:17.231 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:26:17.266 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:26:17.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:26:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:26:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:26:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:26:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:19.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:26:19.127 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:26:19.127 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:19.127 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:26:19.127 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:26:19.128 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:26:19.128 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:19.269 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:26:19.331 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:26:19.403 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:26:19.638 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:26:19.722 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:26:19.821 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:26:19.930 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:26:20.114 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:26:20.163 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:26:20.203 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:26:20.228 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:26:20.759 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:26:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:26:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:26:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:26:20.966 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:26:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:26:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:26:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:26:22.269 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:26:22.422 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:26:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:26:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:26:22.682 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:26:22.682 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:22.683 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:26:22.683 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:26:22.683 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:26:22.683 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:22.739 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:26:22.766 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:26:22.873 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:26:22.934 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:26:23.991 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:26:24.033 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:26:24.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:24.133 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:26:24.183 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:26:24.183 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:24.184 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:26:24.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:24.242 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:26:24.293 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:24.651 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:26:24.702 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:26:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:26:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:26:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:26:24.703 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:26:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:26:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:26:26.114 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:26:26.273 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:26:26.485 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:26:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:26:26.569 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:26:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:26:26.765 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:26:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:26:27.949 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:26:28.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:28.078 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:26:28.128 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:26:28.128 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:28.128 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:26:28.128 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:28.187 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:26:28.239 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:28.612 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:26:28.664 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:26:28.664 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:26:28.664 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:26:28.664 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:26:28.664 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:26:28.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:29.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:26:29.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:26:29.166 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:26:29.203 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:26:29.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:26:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:26:29.535 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:26:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:26:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:26:29.674 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:26:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:30.496 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:26:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:26:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:26:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:26:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:26:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:26:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:26:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:26:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:26:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:26:32.410 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:26:32.434 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:26:32.450 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:32.450 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:26:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:26:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:33.089 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:33.089 INFO:teuthology.orchestra.run.smithi150.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T06:26:33.089 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:33.286 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:26:33.335 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:26:33.387 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:26:33.423 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:26:33.459 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:26:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:26:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:26:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:26:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:26:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:26:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:26:36.074 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:26:36.181 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:26:36.236 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:26:36.348 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:26:36.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:26:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:26:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:26:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:26:36.569 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:26:36.569 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:26:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:26:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:26:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:26:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:26:37.097 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:26:37.183 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:37.394 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:37.394 INFO:teuthology.orchestra.run.smithi037.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T06:26:37.394 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:39.087 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:26:39.087 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:26:39.087 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:26:39.087 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:26:39.087 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:26:39.088 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:26:39.089 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:26:39.101 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:26:39.101 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:26:39.101 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:26:39.101 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:26:39.101 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:26:39.102 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:26:39.103 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:26:39.104 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:26:39.105 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:26:39.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:26:39.107 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:26:39.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:26:39.109 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:26:39.110 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:26:39.111 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:26:41.289 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:41.290 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.291 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.292 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.293 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:26:41.294 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:26:41.295 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:26:41.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:26:41.297 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:26:41.298 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:26:41.299 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:26:41.300 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:26:41.301 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:41.302 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-21T06:26:41.888 DEBUG:teuthology.parallel:result is None 2024-09-21T06:26:43.512 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:26:43.512 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:26:43.512 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:26:43.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:26:43.514 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:26:43.515 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:26:43.515 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:26:43.515 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:26:43.515 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:26:43.527 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:26:43.527 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:26:43.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:26:43.529 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:26:43.530 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:26:43.531 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:26:43.532 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:26:43.533 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:26:43.534 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:26:43.535 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:26:43.536 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:26:43.537 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:26:43.538 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:26:43.539 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:26:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:26:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:26:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:26:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:26:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:26:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:26:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:26:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:26:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:26:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:26:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:26:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-21T06:26:46.372 DEBUG:teuthology.parallel:result is None 2024-09-21T06:26:46.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:26:46.521 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:26:46.567 INFO:teuthology.orchestra.run.smithi037.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:26:46.568 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:26:46.568 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:26:46.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:26:46.712 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:26:46.761 INFO:teuthology.orchestra.run.smithi150.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:26:46.761 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:26:46.761 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:26:46.763 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:26:46.763 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:46.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:26:46.827 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:46.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:26:46.888 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:26:46.889 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:46.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:26:46.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:26:47.059 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:47.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:26:47.116 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:26:47.212 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:26:47.212 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:47.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:26:47.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:26:47.370 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:47.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:26:47.426 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:26:47.523 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:26:47.523 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:47.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:26:47.582 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:26:47.684 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:47.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:26:47.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:26:47.834 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T06:26:47.906 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T06:26:47.906 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-09-21T06:26:47.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-09-21T06:26:47.966 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T06:26:47.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T06:26:47.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T06:26:48.024 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T06:26:48.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T06:26:48.028 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T06:26:48.093 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T06:26:48.094 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T06:26:48.094 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T06:26:48.094 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:48.094 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T06:26:48.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T06:26:48.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T06:26:48.366 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:48.366 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T06:26:48.402 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T06:26:48.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T06:26:48.630 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T06:26:48.631 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T06:26:48.631 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913546', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIxe0vejnC5D3xtzH//zZpcvB2UDHhuLt+a/fVk3+UmV9uW5DJoFMnKeLVsDanhZOz1cQIM4fIFb11Uk8GJFmg=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBICDeAFhIv1zoJsCyQ5tkbfrYA7EbMrXELzItru/cYMBLvHI0OV2RvsWfDpMao9aI6vomW5s6Lw2BZs81H6VLzw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T06:26:48.631 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T06:26:48.635 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T06:26:48.670 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T06:26:48.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T06:26:48.771 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:48.771 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:26:48.807 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:26:48.808 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-21 06:26:37.741657476 +0000 2024-09-21T06:26:48.883 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-21 06:24:23.349914424 +0000 2024-09-21T06:26:48.884 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-21 06:24:23.349914424 +0000 2024-09-21T06:26:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-21 06:24:23.349914424 +0000 2024-09-21T06:26:48.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:26:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-21T06:26:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-21T06:26:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000339649 s, 1.5 MB/s 2024-09-21T06:26:48.979 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:26:49.016 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-21 06:26:37.742657463 +0000 2024-09-21T06:26:49.092 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-21 06:24:23.816908347 +0000 2024-09-21T06:26:49.093 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-21 06:24:23.816908347 +0000 2024-09-21T06:26:49.093 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-21 06:24:23.816908347 +0000 2024-09-21T06:26:49.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:26:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-21T06:26:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-21T06:26:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000278831 s, 1.8 MB/s 2024-09-21T06:26:49.187 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:26:49.224 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-21 06:26:37.743657449 +0000 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-21 06:24:24.273902400 +0000 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-21 06:24:24.273902400 +0000 2024-09-21T06:26:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-21 06:24:24.273902400 +0000 2024-09-21T06:26:49.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:26:49.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-21T06:26:49.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-21T06:26:49.392 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000263853 s, 1.9 MB/s 2024-09-21T06:26:49.394 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:26:49.431 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-21 06:26:37.744657436 +0000 2024-09-21T06:26:49.506 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-21 06:24:24.724896531 +0000 2024-09-21T06:26:49.507 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-21 06:24:24.724896531 +0000 2024-09-21T06:26:49.507 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-21 06:24:24.724896531 +0000 2024-09-21T06:26:49.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:26:49.600 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-21T06:26:49.600 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-21T06:26:49.600 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000322292 s, 1.6 MB/s 2024-09-21T06:26:49.602 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:26:49.639 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:49.639 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:49.639 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:26:49.679 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:26:49.679 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 06:26:33.456698841 +0000 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 06:24:21.521930123 +0000 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 06:24:21.521930123 +0000 2024-09-21T06:26:49.758 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 06:24:21.521930123 +0000 2024-09-21T06:26:49.759 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:26:49.852 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T06:26:49.853 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T06:26:49.853 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000263739 s, 1.9 MB/s 2024-09-21T06:26:49.855 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:26:49.893 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 06:26:33.456698841 +0000 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 06:24:21.999920452 +0000 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 06:24:21.999920452 +0000 2024-09-21T06:26:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 06:24:21.999920452 +0000 2024-09-21T06:26:49.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:26:50.059 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T06:26:50.060 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T06:26:50.060 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000298653 s, 1.7 MB/s 2024-09-21T06:26:50.062 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:26:50.098 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 06:26:33.457698820 +0000 2024-09-21T06:26:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 06:24:22.502910275 +0000 2024-09-21T06:26:50.175 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 06:24:22.502910275 +0000 2024-09-21T06:26:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 06:24:22.502910275 +0000 2024-09-21T06:26:50.175 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:26:50.267 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T06:26:50.267 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T06:26:50.267 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000302926 s, 1.7 MB/s 2024-09-21T06:26:50.270 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:26:50.307 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 06:26:33.458698800 +0000 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 06:24:22.969900827 +0000 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 06:24:22.969900827 +0000 2024-09-21T06:26:50.382 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 06:24:22.969900827 +0000 2024-09-21T06:26:50.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:26:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T06:26:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T06:26:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000289811 s, 1.8 MB/s 2024-09-21T06:26:50.478 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:26:50.514 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:50.514 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T06:26:50.514 INFO:tasks.ceph:Generating config... 2024-09-21T06:26:50.516 INFO:tasks.ceph:[client] pid file = 2024-09-21T06:26:50.516 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T06:26:50.516 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T06:26:50.516 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T06:26:50.516 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T06:26:50.517 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T06:26:50.517 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T06:26:50.518 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T06:26:50.681 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T06:26:50.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T06:26:50.851 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T06:26:50.906 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.37'), ('mon.b', '172.21.15.150'), ('mon.c', '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]')] 2024-09-21T06:26:50.906 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.37,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T06:26:50.907 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:50.907 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T06:26:50.941 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.37 --add b 172.21.15.150 --addv c '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid 13e9e1b2-9d52-4cf2-a477-95a0639530e0 2024-09-21T06:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-09-21T06:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2024-09-21T06:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 13e9e1b2-9d52-4cf2-a477-95a0639530e0 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2024-09-21T06:26:51.124644+0000 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-09-21T06:26:51.124644+0000 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 17 (quincy) 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:2: [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] mon.c 2024-09-21T06:26:51.126 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T06:26:51.135 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T06:26:51.169 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 13e9e1b2-9d52-4cf2-a477-95a0639530e0... 2024-09-21T06:26:51.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T06:26:51.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T06:26:51.305 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-21T06:26:51.305 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-09-21T06:26:51.306 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-09-21T06:26:51.307 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-09-21T06:26:51.308 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on skipped message = False 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 13e9e1b2-9d52-4cf2-a477-95a0639530e0 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.309 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-09-21T06:26:51.310 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-09-21T06:26:51.311 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-09-21T06:26:51.312 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-09-21T06:26:51.313 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-09-21T06:26:51.314 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-09-21T06:26:51.315 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.316 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.317 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-09-21T06:26:51.318 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.37,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 13e9e1b2-9d52-4cf2-a477-95a0639530e0 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-09-21T06:26:51.319 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.320 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T06:26:51.321 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-21T06:26:51.322 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T06:26:51.323 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: mon down mkfs grace = 300 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-09-21T06:26:51.324 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-09-21T06:26:51.325 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-09-21T06:26:51.341 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T06:26:51.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T06:26:51.509 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T06:26:51.510 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:51.510 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T06:26:51.544 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:51.544 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T06:26:51.619 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:26:51.619 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:51.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T06:26:51.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T06:26:51.736 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:26:51.736 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:26:51.811 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:26:51.811 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:51.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T06:26:51.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T06:26:51.894 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:26:51.894 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:26:51.970 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T06:26:51.970 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T06:26:51.970 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T06:26:52.138 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T06:26:52.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T06:26:52.346 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T06:26:52.360 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T06:26:52.360 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T06:26:52.530 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T06:26:52.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T06:26:52.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T06:26:52.764 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T06:26:52.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T06:26:52.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T06:26:52.999 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T06:26:53.010 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T06:26:53.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T06:26:53.231 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T06:26:53.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T06:26:53.297 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T06:26:53.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T06:26:53.445 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T06:26:53.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T06:26:53.623 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T06:26:53.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T06:26:53.802 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T06:26:53.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T06:26:53.978 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T06:26:54.012 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T06:26:54.012 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T06:26:54.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T06:26:54.069 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:54.069 INFO:tasks.ceph:role: osd.0 2024-09-21T06:26:54.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:26:54.069 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T06:26:54.131 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:26:54.131 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:26:54.132 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:26:54.537 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-21T06:26:54.541 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-21T06:26:54.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T06:26:54.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T06:26:54.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T06:26:54.809 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:54.809 INFO:tasks.ceph:role: osd.1 2024-09-21T06:26:54.809 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:26:54.809 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:26:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:26:55.228 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-21T06:26:55.232 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-21T06:26:55.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T06:26:55.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T06:26:55.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T06:26:55.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:55.433 INFO:tasks.ceph:role: osd.2 2024-09-21T06:26:55.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:26:55.433 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:26:55.882 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-21T06:26:55.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-21T06:26:55.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T06:26:55.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T06:26:56.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T06:26:56.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T06:26:56.102 INFO:tasks.ceph:role: osd.3 2024-09-21T06:26:56.102 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:26:56.103 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T06:26:56.165 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:26:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:26:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:26:56.166 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:26:56.589 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-21T06:26:56.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-21T06:26:56.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T06:26:56.732 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T06:26:56.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:26:56.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:26:56.960+0000 7f2e5fe1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T06:26:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:26:56.965+0000 7f2e5fe1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T06:26:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:26:56.965+0000 7f2e5fe1b640 -1 bdev(0x5628c102f900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T06:26:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:26:56.965+0000 7f2e5fe1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T06:27:00.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T06:27:00.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:00.454+0000 7f0aecfb0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T06:27:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:00.455+0000 7f0aecfb0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T06:27:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:00.455+0000 7f0aecfb0640 -1 bdev(0x55a04dda1900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:00.455+0000 7f0aecfb0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T06:27:03.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T06:27:03.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:03.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:03.710+0000 7fb44e860640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T06:27:03.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:03.711+0000 7fb44e860640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T06:27:03.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:03.711+0000 7fb44e860640 -1 bdev(0x56446f8eb900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:03.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:03.711+0000 7fb44e860640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T06:27:06.742 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T06:27:06.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:06.962+0000 7f184ca12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T06:27:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:06.962+0000 7f184ca12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T06:27:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:06.962+0000 7f184ca12640 -1 bdev(0x557c0ee4f900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:06.963+0000 7f184ca12640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T06:27:10.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T06:27:10.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T06:27:10.152 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T06:27:10.152 INFO:tasks.ceph:role: osd.4 2024-09-21T06:27:10.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:27:10.152 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:27:10.217 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:27:10.218 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:27:10.218 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:27:10.625 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T06:27:10.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T06:27:10.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T06:27:10.761 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T06:27:10.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T06:27:10.882 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T06:27:10.882 INFO:tasks.ceph:role: osd.5 2024-09-21T06:27:10.882 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:27:10.882 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T06:27:10.946 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:27:10.946 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:27:10.946 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:27:10.946 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:27:10.947 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:27:11.300 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T06:27:11.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T06:27:11.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T06:27:11.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T06:27:11.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T06:27:11.512 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T06:27:11.512 INFO:tasks.ceph:role: osd.6 2024-09-21T06:27:11.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:27:11.512 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:27:11.576 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:27:11.577 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:27:11.577 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:27:11.577 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:27:11.577 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:27:11.958 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T06:27:11.962 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T06:27:11.962 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T06:27:12.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T06:27:12.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T06:27:12.187 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T06:27:12.187 INFO:tasks.ceph:role: osd.7 2024-09-21T06:27:12.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:27:12.187 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T06:27:12.250 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T06:27:12.250 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T06:27:12.251 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T06:27:12.664 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T06:27:12.668 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T06:27:12.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T06:27:12.805 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T06:27:12.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:13.033+0000 7f081c81e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:13.039+0000 7f081c81e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:13.039+0000 7f081c81e640 -1 bdev(0x559c25e57900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:13.040+0000 7f081c81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T06:27:16.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T06:27:16.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:16.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:16.273+0000 7f7ddb0f1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T06:27:16.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:16.274+0000 7f7ddb0f1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T06:27:16.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:16.274+0000 7f7ddb0f1640 -1 bdev(0x559bde807900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:16.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:16.274+0000 7f7ddb0f1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T06:27:18.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:27:18.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:27:19.526 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T06:27:19.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:19.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:19.733+0000 7fea192d8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T06:27:19.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:19.733+0000 7fea192d8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T06:27:19.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:19.734+0000 7fea192d8640 -1 bdev(0x563b6f1af900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:19.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:19.734+0000 7fea192d8640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T06:27:22.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T06:27:22.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:22.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:22.989+0000 7f49bd62e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T06:27:22.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:22.990+0000 7f49bd62e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T06:27:22.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:22.990+0000 7f49bd62e640 -1 bdev(0x562296c23900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T06:27:22.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:22.990+0000 7f49bd62e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T06:27:26.049 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T06:27:26.104 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T06:27:26.105 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T06:27:26.168 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T06:27:26.266 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T06:27:26.363 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T06:27:26.459 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T06:27:26.554 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T06:27:26.649 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:26.649 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T06:27:26.744 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:26.744 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T06:27:26.803 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:26.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T06:27:26.899 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:26.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T06:27:26.994 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:26.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T06:27:27.089 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:27.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T06:27:27.184 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:27.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T06:27:27.279 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:27.279 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T06:27:27.376 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:27.376 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T06:27:27.413 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:27.414 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T06:27:27.492 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:27.492 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T06:27:27.527 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:27.527 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T06:27:27.602 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T06:27:27.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T06:27:27.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T06:27:27.657 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2024-09-21T06:27:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ== 2024-09-21T06:27:27.657 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g== 2024-09-21T06:27:27.658 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:27:27.659 INFO:teuthology.orchestra.run.smithi037.stdout:[client.2] 2024-09-21T06:27:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q== 2024-09-21T06:27:27.660 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2024-09-21T06:27:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA== 2024-09-21T06:27:27.660 INFO:teuthology.orchestra.run.smithi037.stdout:[client.3] 2024-09-21T06:27:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA== 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ== 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg== 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g== 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw== 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-09-21T06:27:27.695 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w== 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-09-21T06:27:27.696 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[client.2] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA== 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:[client.3] 2024-09-21T06:27:27.697 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA== 2024-09-21T06:27:27.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T06:27:27.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T06:27:27.900 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:27.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:28.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:28.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:28.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.245 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.577 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.742 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:28.904 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T06:27:28.908 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T06:27:29.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:29.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:29.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:29.244 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T06:27:29.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.414 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.741 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.745 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:29.912 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T06:27:30.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.078 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.582 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T06:27:30.752 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T06:27:30.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T06:27:30.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T06:27:30.991 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T06:27:31.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T06:27:31.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T06:27:31.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T06:27:31.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T06:27:31.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T06:27:31.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T06:27:31.623 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:31.627 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T06:27:31.663 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T06:27:31.663 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T06:27:31.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T06:27:31.706 INFO:tasks.ceph.mon.a:Started 2024-09-21T06:27:31.706 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T06:27:31.707 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T06:27:31.710 INFO:tasks.ceph.mon.b:Started 2024-09-21T06:27:31.711 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T06:27:31.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T06:27:31.714 INFO:tasks.ceph.mon.c:Started 2024-09-21T06:27:31.714 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T06:27:31.715 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T06:27:31.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T06:27:31.718 INFO:tasks.ceph.mgr.y:Started 2024-09-21T06:27:31.719 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T06:27:31.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T06:27:31.723 INFO:tasks.ceph.mgr.x:Started 2024-09-21T06:27:31.723 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T06:27:31.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T06:27:31.728 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T06:27:31.728 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T06:27:31.773 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T06:27:31.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T06:27:31.820 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T06:27:31.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T06:27:31.866 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T06:27:31.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T06:27:31.912 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T06:27:31.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T06:27:31.959 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T06:27:31.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T06:27:32.006 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T06:27:32.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T06:27:32.056 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T06:27:32.056 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T06:27:32.107 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T06:27:32.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T06:27:32.163 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T06:27:32.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T06:27:32.226 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T06:27:32.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T06:27:32.283 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T06:27:32.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T06:27:32.336 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T06:27:32.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T06:27:32.388 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T06:27:32.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T06:27:32.436 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T06:27:32.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T06:27:32.490 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T06:27:32.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T06:27:32.545 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T06:27:32.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T06:27:32.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.570+0000 7f1f52a00640 1 -- 172.21.15.37:0/1604151207 <== mon.2 v2:172.21.15.150:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7f1f54046a00 con 0x7f1f5c15e740 2024-09-21T06:27:32.577 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T06:27:32.577 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T06:27:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 -- 172.21.15.37:0/1604151207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5c15e740 msgr2=0x7f1f5c1e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 --2- 172.21.15.37:0/1604151207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5c15e740 0x7f1f5c1e6de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009000 tx=0x7f1f54004a40 comp rx=0 tx=0).stop 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 -- 172.21.15.37:0/1604151207 shutdown_connections 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 --2- 172.21.15.37:0/1604151207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5c0d2410 0x7f1f5c2019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 --2- 172.21.15.37:0/1604151207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5c15e740 0x7f1f5c1e6de0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 --2- 172.21.15.37:0/1604151207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f5c0b77a0 0x7f1f5c1fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 -- 172.21.15.37:0/1604151207 >> 172.21.15.37:0/1604151207 conn(0x7f1f5c0d4290 msgr2=0x7f1f5c0b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 -- 172.21.15.37:0/1604151207 shutdown_connections 2024-09-21T06:27:32.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.574+0000 7f1f61800640 1 -- 172.21.15.37:0/1604151207 wait complete. 2024-09-21T06:27:32.633 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T06:27:32.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T06:27:32.685 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T06:27:32.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T06:27:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.671+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fa088000e60 con 0x7fa0b015ed20 2024-09-21T06:27:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.692+0000 7fa0a7400640 1 -- 172.21.15.37:0/1129253765 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7fa09c018e10 con 0x7fa0b015ed20 2024-09-21T06:27:32.701 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T06:27:32.701 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T06:27:32.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.694+0000 7ff97d600640 1 -- 172.21.15.37:0/3762468660 <== mon.1 v2:172.21.15.150:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff978017b40 con 0x7ff9800c5cf0 2024-09-21T06:27:32.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.693+0000 7fbc8e000640 1 -- 172.21.15.37:0/1201933295 <== mon.2 v2:172.21.15.150:3301/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fbc84010480 con 0x7fbc90009ba0 2024-09-21T06:27:32.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b015ed20 msgr2=0x7fa0b0142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 --2- 172.21.15.37:0/1129253765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b015ed20 0x7fa0b0142aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00e6a0 tx=0x7fa09c00eb60 comp rx=0 tx=0).stop 2024-09-21T06:27:32.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.701+0000 7f6985600640 1 -- 172.21.15.37:0/1733754102 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f697c03b660 con 0x7f6988073000 2024-09-21T06:27:32.727 DEBUG:tasks.ceph:set 22 configs 2024-09-21T06:27:32.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi037') 2024-09-21T06:27:32.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi037') 2024-09-21T06:27:32.728 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi037') 2024-09-21T06:27:32.728 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-21T06:27:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 shutdown_connections 2024-09-21T06:27:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 --2- 172.21.15.37:0/1129253765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b015ed20 0x7fa0b0142aa0 secure :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00e6a0 tx=0x7fa09c00eb60 comp rx=0 tx=0).stop 2024-09-21T06:27:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 --2- 172.21.15.37:0/1129253765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b01fc6d0 0x7fa0b01fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 --2- 172.21.15.37:0/1129253765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0b00709e0 0x7fa0b00b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.707+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 >> 172.21.15.37:0/1129253765 conn(0x7fa0b00cfec0 msgr2=0x7fa0b015ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.716+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 shutdown_connections 2024-09-21T06:27:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.717+0000 7fa0a4c00640 1 -- 172.21.15.37:0/1129253765 wait complete. 2024-09-21T06:27:32.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.755+0000 7f218b400640 1 -- 172.21.15.37:0/2519067079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2168003a00 msgr2=0x7f2168003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.755+0000 7f218b400640 1 --2- 172.21.15.37:0/2519067079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2168003a00 0x7f2168003e50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f217400c610 tx=0x7f2168005830 comp rx=0 tx=0).stop 2024-09-21T06:27:32.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.786+0000 7f218b400640 1 -- 172.21.15.37:0/2519067079 shutdown_connections 2024-09-21T06:27:32.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.786+0000 7f218b400640 1 --2- 172.21.15.37:0/2519067079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2168003a00 0x7f2168003e50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.786+0000 7f218b400640 1 -- 172.21.15.37:0/2519067079 >> 172.21.15.37:0/2519067079 conn(0x7f218406cde0 msgr2=0x7f218406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.793+0000 7f218b400640 1 -- 172.21.15.37:0/2519067079 shutdown_connections 2024-09-21T06:27:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.797+0000 7f218b400640 1 -- 172.21.15.37:0/2519067079 wait complete. 2024-09-21T06:27:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.809+0000 7f218b400640 1 Processor -- start 2024-09-21T06:27:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.809+0000 7f218b400640 1 -- start start 2024-09-21T06:27:32.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.816+0000 7f218b400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2168003a00 0x7f21841610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.817+0000 7f218b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2184143410 0x7f2184143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.817+0000 7f218b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.817+0000 7f218b400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21840b3870 con 0x7f2184143820 2024-09-21T06:27:32.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.817+0000 7f218b400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2184074f40 con 0x7f2184143410 2024-09-21T06:27:32.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.818+0000 7f218b400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f21840b39f0 con 0x7f2168003a00 2024-09-21T06:27:32.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.818+0000 7f2189400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.818+0000 7f2189400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34722/0 (socket says 172.21.15.37:34722) 2024-09-21T06:27:32.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.818+0000 7f2189400640 1 -- 172.21.15.37:0/649920903 learned_addr learned my addr 172.21.15.37:0/649920903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2188a00640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2168003a00 0x7f21841610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2189400640 1 -- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2168003a00 msgr2=0x7f21841610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2189400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2168003a00 0x7f21841610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2189400640 1 -- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2184143410 msgr2=0x7f2184143dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:27:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2189400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2184143410 0x7f2184143dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.819+0000 7f2189400640 1 -- 172.21.15.37:0/649920903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2184160020 con 0x7f2184143820 2024-09-21T06:27:32.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.822+0000 7f2189400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2184072a40 tx=0x7f217c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:32.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.823+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c019070 con 0x7f2184143820 2024-09-21T06:27:32.827 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-21T06:27:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.823+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2184160300 con 0x7f2184143820 2024-09-21T06:27:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.823+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21841608c0 con 0x7f2184143820 2024-09-21T06:27:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.825+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f217c00cde0 con 0x7f2184143820 2024-09-21T06:27:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.832+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c019070 con 0x7f2184143820 2024-09-21T06:27:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.832+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c005290 con 0x7f2184143820 2024-09-21T06:27:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.836+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f217c01c070 con 0x7f2184143820 2024-09-21T06:27:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.836+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f217c01e910 con 0x7f2184143820 2024-09-21T06:27:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.842+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff954005310 con 0x7ff9800c5cf0 2024-09-21T06:27:32.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.850+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f217c023020 con 0x7f2184143820 2024-09-21T06:27:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.857+0000 7fc474000640 1 -- 172.21.15.37:0/3917941573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c06ec50 msgr2=0x7fc46c06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.857+0000 7fc474000640 1 --2- 172.21.15.37:0/3917941573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c06ec50 0x7fc46c06f060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc46c074d90 tx=0x7fc46402b7a0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 -- 172.21.15.37:0/3917941573 shutdown_connections 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 --2- 172.21.15.37:0/3917941573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c06ec50 0x7fc46c06f060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 --2- 172.21.15.37:0/3917941573 >> v2:172.21.15.150:3300/0 conn(0x7fc46c06ddf0 0x7fc46c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 --2- 172.21.15.37:0/3917941573 >> v2:172.21.15.37:3300/0 conn(0x7fc46c15ac20 0x7fc46c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 -- 172.21.15.37:0/3917941573 >> 172.21.15.37:0/3917941573 conn(0x7fc46c0b1b90 msgr2=0x7fc46c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 -- 172.21.15.37:0/3917941573 shutdown_connections 2024-09-21T06:27:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.863+0000 7fc474000640 1 -- 172.21.15.37:0/3917941573 wait complete. 2024-09-21T06:27:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.872+0000 7f6985600640 1 -- 172.21.15.37:0/1733754102 <== mon.0 v2:172.21.15.37:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f697c03b8a0 con 0x7f6988073000 2024-09-21T06:27:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.872+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f217c014570 con 0x7f2184143820 2024-09-21T06:27:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.877+0000 7ff97d600640 1 -- 172.21.15.37:0/3762468660 <== mon.1 v2:172.21.15.150:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff978003970 con 0x7ff9800c5cf0 2024-09-21T06:27:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.877+0000 7ff97d600640 1 -- 172.21.15.37:0/3762468660 <== mon.1 v2:172.21.15.150:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7ff97801a070 con 0x7ff9800c5cf0 2024-09-21T06:27:32.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.878+0000 7fc474000640 1 Processor -- start 2024-09-21T06:27:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.878+0000 7fc474000640 1 -- start start 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.886+0000 7fbc8e000640 1 -- 172.21.15.37:0/1201933295 <== mon.2 v2:172.21.15.150:3301/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc84010480 con 0x7fbc90009ba0 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.886+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f69880cc160 con 0x7f6988073000 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.883+0000 7f79fda00640 1 -- 172.21.15.37:0/1616192519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 msgr2=0x7f79dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.889+0000 7f79f5600640 1 -- 172.21.15.37:0/1616192519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f000c260 con 0x7f79dc005270 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.889+0000 7f79fda00640 1 --2- 172.21.15.37:0/1616192519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79dc00b9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c610 tx=0x7f79dc005830 comp rx=0 tx=0).stop 2024-09-21T06:27:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.889+0000 7f79f7400640 1 -- 172.21.15.37:0/1616192519 reap_dead start 2024-09-21T06:27:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.891+0000 7fc474000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46c06ddf0 0x7fc46c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.892+0000 7fc474000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7fc470c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7fc470c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50644/0 (socket says 172.21.15.37:50644) 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9800c5cf0 msgr2=0x7ff98009a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7ff988400640 1 --2- 172.21.15.37:0/3762468660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9800c5cf0 0x7ff98009a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9780077e0 tx=0x7ff978007ca0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 shutdown_connections 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.893+0000 7ff988400640 1 --2- 172.21.15.37:0/3762468660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9800c5cf0 0x7ff98009a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7ff988400640 1 --2- 172.21.15.37:0/3762468660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff98015fc50 0x7ff9800c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7ff988400640 1 --2- 172.21.15.37:0/3762468660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9800cf7d0 0x7ff9800c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 >> 172.21.15.37:0/3762468660 conn(0x7ff9800b1b90 msgr2=0x7ff98015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7fc470c00640 1 -- 172.21.15.37:0/1113030347 learned_addr learned my addr 172.21.15.37:0/1113030347 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7fc474000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 0x7fc46c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.895+0000 7fc472000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 0x7fc46c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.895+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c070e80 con 0x7fc46c06ddf0 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.895+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c071000 con 0x7fc46c06ec50 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.895+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c06f5a0 con 0x7fc46c15ac20 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7f79fda00640 1 -- 172.21.15.37:0/1616192519 shutdown_connections 2024-09-21T06:27:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.894+0000 7f79fda00640 1 -- 172.21.15.37:0/1616192519 >> 172.21.15.37:0/1616192519 conn(0x7f79f806cde0 msgr2=0x7f79f806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc470c00640 1 -- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 msgr2=0x7fc46c0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc470c00640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 0x7fc46c0b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc470c00640 1 -- 172.21.15.37:0/1113030347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46c06ddf0 msgr2=0x7fc46c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc470c00640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46c06ddf0 0x7fc46c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc470c00640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c0c5de0 con 0x7fc46c06ec50 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7fc472000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 0x7fc46c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.898+0000 7f79fda00640 1 -- 172.21.15.37:0/1616192519 shutdown_connections 2024-09-21T06:27:32.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.899+0000 7f79fda00640 1 -- 172.21.15.37:0/1616192519 wait complete. 2024-09-21T06:27:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.899+0000 7f6985600640 1 -- 172.21.15.37:0/1733754102 <== mon.0 v2:172.21.15.37:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v7)=0 v7) ==== 117+0+0 (secure 0 0 0) 0x7f697c03ba50 con 0x7f6988073000 2024-09-21T06:27:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.903+0000 7fc470c00640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc45800b410 tx=0x7fc45800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.903+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458015020 con 0x7fc46c06ec50 2024-09-21T06:27:32.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.904+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc458011920 con 0x7fc46c06ec50 2024-09-21T06:27:32.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.904+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c0c60c0 con 0x7fc46c06ec50 2024-09-21T06:27:32.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.904+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c154360 con 0x7fc46c06ec50 2024-09-21T06:27:32.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.905+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 shutdown_connections 2024-09-21T06:27:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.905+0000 7ff988400640 1 -- 172.21.15.37:0/3762468660 wait complete. 2024-09-21T06:27:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.906+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458011d50 con 0x7fc46c06ec50 2024-09-21T06:27:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.909+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc458018020 con 0x7fc46c06ec50 2024-09-21T06:27:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.909+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc45801a640 con 0x7fc46c06ec50 2024-09-21T06:27:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.909+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc448005290 con 0x7fc46c06ec50 2024-09-21T06:27:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.910+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fbc6c0052f0 con 0x7fbc90009ba0 2024-09-21T06:27:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.915+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc45801f070 con 0x7fc46c06ec50 2024-09-21T06:27:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.917+0000 7f79fda00640 1 Processor -- start 2024-09-21T06:27:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.917+0000 7f79fda00640 1 -- start start 2024-09-21T06:27:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.920+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6988073000 msgr2=0x7f69880b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.920+0000 7f697aa00640 1 --2- 172.21.15.37:0/1733754102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6988073000 0x7f69880b2ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f697c009a00 tx=0x7f697c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.926+0000 7fbc8e000640 1 -- 172.21.15.37:0/1201933295 <== mon.2 v2:172.21.15.150:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v8)=0 v8) ==== 108+0+0 (secure 0 0 0) 0x7fbc840108b0 con 0x7fbc90009ba0 2024-09-21T06:27:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.931+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc90009ba0 msgr2=0x7fbc90047a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.931+0000 7fbc97200640 1 --2- 172.21.15.37:0/1201933295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc90009ba0 0x7fbc90047a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400c2e0 tx=0x7fbc8400c7a0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.932+0000 7f79fda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79f8170170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7f79fda00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f815bf90 0x7f79f815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7f79fda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 shutdown_connections 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7fbc97200640 1 --2- 172.21.15.37:0/1201933295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc900da630 0x7fbc900d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7fbc97200640 1 --2- 172.21.15.37:0/1201933295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc90009ba0 0x7fbc90047a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7fbc97200640 1 --2- 172.21.15.37:0/1201933295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc90005c60 0x7fbc9001e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.935+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 >> 172.21.15.37:0/1201933295 conn(0x7fbc90051420 msgr2=0x7fbc900de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.936+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 shutdown_connections 2024-09-21T06:27:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.936+0000 7fbc97200640 1 -- 172.21.15.37:0/1201933295 wait complete. 2024-09-21T06:27:32.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.936+0000 7f79f7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.936+0000 7f79f7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50648/0 (socket says 172.21.15.37:50648) 2024-09-21T06:27:32.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.940+0000 7f79f6000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f815bf90 0x7f79f815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.940+0000 7f79f7400640 1 -- 172.21.15.37:0/483888575 learned_addr learned my addr 172.21.15.37:0/483888575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:32.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.940+0000 7f79fda00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79f808bd70 con 0x7f79dc005270 2024-09-21T06:27:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.940+0000 7f79fda00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79f808bbf0 con 0x7f79f815c8e0 2024-09-21T06:27:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.940+0000 7f79fda00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f79f80b3790 con 0x7f79f815bf90 2024-09-21T06:27:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.941+0000 7f79f6a00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79f8170170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.941+0000 7f79f7400640 1 -- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f815bf90 msgr2=0x7f79f815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.941+0000 7f79f7400640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f815bf90 0x7f79f815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.941+0000 7f79f7400640 1 -- 172.21.15.37:0/483888575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 msgr2=0x7f79f8170170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 shutdown_connections 2024-09-21T06:27:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 --2- 172.21.15.37:0/1733754102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69880cfac0 0x7f698815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 --2- 172.21.15.37:0/1733754102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69880cf6b0 0x7f698809d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 --2- 172.21.15.37:0/1733754102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6988073000 0x7f69880b2ef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 >> 172.21.15.37:0/1733754102 conn(0x7f69880b1b90 msgr2=0x7f698815ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:32.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 shutdown_connections 2024-09-21T06:27:32.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f697aa00640 1 -- 172.21.15.37:0/1733754102 wait complete. 2024-09-21T06:27:32.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f79f7400640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79f8170170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:32.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.942+0000 7f79f7400640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f80ab8c0 con 0x7f79f815c8e0 2024-09-21T06:27:32.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.946+0000 7f79f6a00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79f8170170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:32.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.950+0000 7f79f7400640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c730 tx=0x7f79f0004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:32.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f002b990 con 0x7f79f815c8e0 2024-09-21T06:27:32.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79f002bb30 con 0x7f79f815c8e0 2024-09-21T06:27:32.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79fda00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f80abba0 con 0x7f79f815c8e0 2024-09-21T06:27:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79fda00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f815b790 con 0x7f79f815c8e0 2024-09-21T06:27:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f00060f0 con 0x7f79f815c8e0 2024-09-21T06:27:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f79f0006620 con 0x7f79f815c8e0 2024-09-21T06:27:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.953+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f79f003f770 con 0x7f79f815c8e0 2024-09-21T06:27:32.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.969+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79cc005290 con 0x7f79f815c8e0 2024-09-21T06:27:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:32.990+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f79f0034e20 con 0x7f79f815c8e0 2024-09-21T06:27:33.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.031 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.057 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.077+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f215c005580 con 0x7f2184143820 2024-09-21T06:27:33.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.099+0000 7f2180c00640 1 -- 172.21.15.37:0/649920903 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f217c01ec30 con 0x7f2184143820 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.100+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 msgr2=0x7f2184144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2184072a40 tx=0x7f217c009e70 comp rx=0 tx=0).stop 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 shutdown_connections 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2184143820 0x7f2184144300 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2184143410 0x7f2184143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 --2- 172.21.15.37:0/649920903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2168003a00 0x7f21841610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.101+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 >> 172.21.15.37:0/649920903 conn(0x7f218406cde0 msgr2=0x7f21840b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.102+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 shutdown_connections 2024-09-21T06:27:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.102+0000 7f218b400640 1 -- 172.21.15.37:0/649920903 wait complete. 2024-09-21T06:27:33.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.109+0000 7f7765a00640 1 -- 172.21.15.37:0/3043408710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7744003a50 msgr2=0x7f7744003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.109+0000 7f7765a00640 1 --2- 172.21.15.37:0/3043408710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7744003a50 0x7f7744003ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77600768e0 tx=0x7f7744005830 comp rx=0 tx=0).stop 2024-09-21T06:27:33.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.109+0000 7f775f400640 1 -- 172.21.15.37:0/3043408710 reap_dead start 2024-09-21T06:27:33.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.109+0000 7f7765a00640 1 -- 172.21.15.37:0/3043408710 shutdown_connections 2024-09-21T06:27:33.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.110+0000 7f7765a00640 1 -- 172.21.15.37:0/3043408710 >> 172.21.15.37:0/3043408710 conn(0x7f776006cde0 msgr2=0x7f776006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.119+0000 7f7765a00640 1 -- 172.21.15.37:0/3043408710 shutdown_connections 2024-09-21T06:27:33.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.120+0000 7f7765a00640 1 -- 172.21.15.37:0/3043408710 wait complete. 2024-09-21T06:27:33.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.136+0000 7f7765a00640 1 Processor -- start 2024-09-21T06:27:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.136+0000 7f7765a00640 1 -- start start 2024-09-21T06:27:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 0x7f7760151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77601fb290 0x7f77601fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77600af330 con 0x7f776014f7e0 2024-09-21T06:27:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77600b3d80 con 0x7f77600cf550 2024-09-21T06:27:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.139+0000 7f7765a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f77600af4b0 con 0x7f77601fb290 2024-09-21T06:27:33.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.148+0000 7f2fa3200640 1 -- 172.21.15.37:0/305655352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f840039e0 msgr2=0x7f2f84003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.148+0000 7f2fa3200640 1 --2- 172.21.15.37:0/305655352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f840039e0 0x7f2f84003e10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009b50 tx=0x7f2f84005830 comp rx=0 tx=0).stop 2024-09-21T06:27:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.149+0000 7f775e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.150+0000 7f775e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50658/0 (socket says 172.21.15.37:50658) 2024-09-21T06:27:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.150+0000 7f775e000640 1 -- 172.21.15.37:0/1613458386 learned_addr learned my addr 172.21.15.37:0/1613458386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.150+0000 7f775f400640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77601fb290 0x7f77601fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.154+0000 7f775ea00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 0x7f7760151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.154+0000 7f775e000640 1 -- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77601fb290 msgr2=0x7f77601fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.154+0000 7f775e000640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77601fb290 0x7f77601fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.154+0000 7f775e000640 1 -- 172.21.15.37:0/1613458386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 msgr2=0x7f7760151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.155+0000 7f775e000640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 0x7f7760151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.155+0000 7f775e000640 1 -- 172.21.15.37:0/1613458386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77601fb6a0 con 0x7f77600cf550 2024-09-21T06:27:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.155+0000 7f775ea00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 0x7f7760151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:33.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f2fa1200640 1 -- 172.21.15.37:0/305655352 reap_dead start 2024-09-21T06:27:33.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f2fa3200640 1 -- 172.21.15.37:0/305655352 shutdown_connections 2024-09-21T06:27:33.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f2fa3200640 1 -- 172.21.15.37:0/305655352 >> 172.21.15.37:0/305655352 conn(0x7f2f9401fbd0 msgr2=0x7f2f94056930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.157+0000 7f775e000640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f775000e630 tx=0x7f775000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f2fa3200640 1 -- 172.21.15.37:0/305655352 shutdown_connections 2024-09-21T06:27:33.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f2fa3200640 1 -- 172.21.15.37:0/305655352 wait complete. 2024-09-21T06:27:33.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.159+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775000ca10 con 0x7f77600cf550 2024-09-21T06:27:33.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.160+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f775000cbd0 con 0x7f77600cf550 2024-09-21T06:27:33.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.160+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750010430 con 0x7f77600cf550 2024-09-21T06:27:33.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.162+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776014ffe0 con 0x7f77600cf550 2024-09-21T06:27:33.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.162+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7760150cb0 con 0x7f77600cf550 2024-09-21T06:27:33.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.163+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f775001f020 con 0x7f77600cf550 2024-09-21T06:27:33.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.163+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77500036b0 con 0x7f77600cf550 2024-09-21T06:27:33.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.167+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7760073bd0 con 0x7f77600cf550 2024-09-21T06:27:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.173+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7750026070 con 0x7f77600cf550 2024-09-21T06:27:33.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.170+0000 7f2fa3200640 1 Processor -- start 2024-09-21T06:27:33.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.181+0000 7f2fa3200640 1 -- start start 2024-09-21T06:27:33.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.197+0000 7fbf21000640 1 -- 172.21.15.37:0/3926177402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 msgr2=0x7fbf00003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.197+0000 7fbf21000640 1 --2- 172.21.15.37:0/3926177402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf00003ea0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0041d0 tx=0x7fbf00005830 comp rx=0 tx=0).stop 2024-09-21T06:27:33.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.202+0000 7fbf21000640 1 -- 172.21.15.37:0/3926177402 shutdown_connections 2024-09-21T06:27:33.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.202+0000 7fbf21000640 1 --2- 172.21.15.37:0/3926177402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf00003ea0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.202+0000 7fbf21000640 1 -- 172.21.15.37:0/3926177402 >> 172.21.15.37:0/3926177402 conn(0x7fbf1c0d0350 msgr2=0x7fbf1c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.202+0000 7fbf21000640 1 -- 172.21.15.37:0/3926177402 shutdown_connections 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f94186dd0 0x7f2f94051000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 0x7f2f940da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94007e10 con 0x7f2f940d9c90 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94007c90 con 0x7f2f94033cc0 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa3200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2f94007f90 con 0x7f2f94186dd0 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50670/0 (socket says 172.21.15.37:50670) 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.203+0000 7f2fa1200640 1 -- 172.21.15.37:0/2518017075 learned_addr learned my addr 172.21.15.37:0/2518017075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.204+0000 7f2f9b400640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 0x7f2f940da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.204+0000 7f2f9be00640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f94186dd0 0x7f2f94051000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.204+0000 7f2fa1200640 1 -- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f94186dd0 msgr2=0x7f2f94051000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.204+0000 7f2fa1200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f94186dd0 0x7f2f94051000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.205+0000 7f2fa1200640 1 -- 172.21.15.37:0/2518017075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 msgr2=0x7f2f940da0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.206+0000 7f2fa1200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 0x7f2f940da0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.206+0000 7f2fa1200640 1 -- 172.21.15.37:0/2518017075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f94034610 con 0x7f2f94033cc0 2024-09-21T06:27:33.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.207+0000 7f2f9b400640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 0x7f2f940da0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.207+0000 7f2fa1200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c076070 tx=0x7f2f9c06af00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.210 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.208+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c075980 con 0x7f2f94033cc0 2024-09-21T06:27:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.208+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f9c075c10 con 0x7f2f94033cc0 2024-09-21T06:27:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.208+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c076380 con 0x7f2f94033cc0 2024-09-21T06:27:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.209+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f94039760 con 0x7f2f94033cc0 2024-09-21T06:27:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.202+0000 7fbf21000640 1 -- 172.21.15.37:0/3926177402 wait complete. 2024-09-21T06:27:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.214+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2f9c094440 con 0x7f2f94033cc0 2024-09-21T06:27:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.215+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f94039d20 con 0x7f2f94033cc0 2024-09-21T06:27:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.215+0000 7fbf21000640 1 Processor -- start 2024-09-21T06:27:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.215+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2f9c093860 con 0x7f2f94033cc0 2024-09-21T06:27:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.215+0000 7fbf21000640 1 -- start start 2024-09-21T06:27:33.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.222+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f94037490 con 0x7f2f94033cc0 2024-09-21T06:27:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.225+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2f9c09a060 con 0x7f2f94033cc0 2024-09-21T06:27:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.226+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f79cc005580 con 0x7f79f815c8e0 2024-09-21T06:27:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.227+0000 7fbf21000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.227+0000 7fbf21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.227+0000 7fbf21000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 0x7fbf1c156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf19e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf19e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50676/0 (socket says 172.21.15.37:50676) 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7f5f71a00640 1 -- 172.21.15.37:0/98584013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c15e3c0 msgr2=0x7f5f6c072f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7f5f71a00640 1 --2- 172.21.15.37:0/98584013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c15e3c0 0x7f5f6c072f50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005fd0 tx=0x7f5f5802a4f0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 -- 172.21.15.37:0/98584013 shutdown_connections 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 --2- 172.21.15.37:0/98584013 >> v2:172.21.15.37:3300/0 conn(0x7f5f6c073490 0x7f5f6c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 --2- 172.21.15.37:0/98584013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c15e3c0 0x7f5f6c072f50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 -- 172.21.15.37:0/98584013 >> 172.21.15.37:0/98584013 conn(0x7f5f6c0b1b90 msgr2=0x7f5f6c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 -- 172.21.15.37:0/98584013 shutdown_connections 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.230+0000 7f5f71a00640 1 -- 172.21.15.37:0/98584013 wait complete. 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf19400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf19400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34808/0 (socket says 172.21.15.37:34808) 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf1a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 0x7fbf1c156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.228+0000 7fbf1a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 0x7fbf1c156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44846/0 (socket says 172.21.15.37:44846) 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.232+0000 7fbf19e00640 1 -- 172.21.15.37:0/1167868700 learned_addr learned my addr 172.21.15.37:0/1167868700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.232+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c073ba0 con 0x7fbf1c151e30 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.232+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c073a20 con 0x7fbf00003a70 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.232+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c073d20 con 0x7fbf1c156560 2024-09-21T06:27:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19400640 1 -- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 msgr2=0x7fbf1c156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19400640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 0x7fbf1c156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19400640 1 -- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 msgr2=0x7fbf1c153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19400640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19400640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c202910 con 0x7fbf1c151e30 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.234+0000 7fbf19e00640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.235+0000 7fbf19400640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c15be20 tx=0x7fbf0800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.237+0000 7f5f71a00640 1 Processor -- start 2024-09-21T06:27:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.238+0000 7f5f71a00640 1 -- start start 2024-09-21T06:27:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.240+0000 7f8f80800640 1 -- 172.21.15.37:0/1292848881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f780d1120 msgr2=0x7f8f780ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 --2- 172.21.15.37:0/1292848881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f780d1120 0x7f8f780ceb90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005fd0 tx=0x7f8f7402a980 comp rx=0 tx=0).stop 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 -- 172.21.15.37:0/1292848881 shutdown_connections 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 --2- 172.21.15.37:0/1292848881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f780d1120 0x7f8f780ceb90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 --2- 172.21.15.37:0/1292848881 >> v2:172.21.15.37:3300/0 conn(0x7f8f780d0170 0x7f8f780d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 -- 172.21.15.37:0/1292848881 >> 172.21.15.37:0/1292848881 conn(0x7f8f7809d830 msgr2=0x7f8f780b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 -- 172.21.15.37:0/1292848881 shutdown_connections 2024-09-21T06:27:33.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7f8f80800640 1 -- 172.21.15.37:0/1292848881 wait complete. 2024-09-21T06:27:33.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.243+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08015020 con 0x7fbf1c151e30 2024-09-21T06:27:33.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.244+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf08011920 con 0x7fbf1c151e30 2024-09-21T06:27:33.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.244+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c154ac0 con 0x7fbf1c151e30 2024-09-21T06:27:33.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.244+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c155d60 con 0x7fbf1c151e30 2024-09-21T06:27:33.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.245+0000 7f5f71a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.245+0000 7f5f71a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 0x7f5f6c0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.246+0000 7f5f71a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 0x7f5f6c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.246+0000 7f5f71a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c06f310 con 0x7f5f6c071260 2024-09-21T06:27:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.246+0000 7f5f6a000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 0x7f5f6c0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.248+0000 7f5f6a000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 0x7f5f6c0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44858/0 (socket says 172.21.15.37:44858) 2024-09-21T06:27:33.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.251+0000 7f79eea00640 1 -- 172.21.15.37:0/483888575 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f79f003f9c0 con 0x7f79f815c8e0 2024-09-21T06:27:33.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.251+0000 7f5f6aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.251+0000 7f5f6aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34812/0 (socket says 172.21.15.37:34812) 2024-09-21T06:27:33.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.251+0000 7f5f6aa00640 1 -- 172.21.15.37:0/2000043871 learned_addr learned my addr 172.21.15.37:0/2000043871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.251+0000 7f5f6b400640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 0x7f5f6c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.253+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c06f190 con 0x7f5f6c15e3c0 2024-09-21T06:27:33.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.253+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c06f490 con 0x7f5f6c073490 2024-09-21T06:27:33.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.255+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011d70 con 0x7fbf1c151e30 2024-09-21T06:27:33.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.255+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef4005290 con 0x7fbf1c151e30 2024-09-21T06:27:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 msgr2=0x7f79f8209680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c730 tx=0x7f79f0004e70 comp rx=0 tx=0).stop 2024-09-21T06:27:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f5f6aa00640 1 -- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 msgr2=0x7f5f6c0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f5f6aa00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 0x7f5f6c0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 shutdown_connections 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f815c8e0 0x7f79f8209680 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f815bf90 0x7f79f815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 --2- 172.21.15.37:0/483888575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79dc005270 0x7f79f8170170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 >> 172.21.15.37:0/483888575 conn(0x7f79f806cde0 msgr2=0x7f79f80b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf08018020 con 0x7fbf1c151e30 2024-09-21T06:27:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.257+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 shutdown_connections 2024-09-21T06:27:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.259+0000 7f8f80800640 1 Processor -- start 2024-09-21T06:27:33.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.256+0000 7f5f6aa00640 1 -- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 msgr2=0x7f5f6c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.261+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf08019860 con 0x7fbf1c151e30 2024-09-21T06:27:33.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.261+0000 7f5f6aa00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 0x7f5f6c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.261+0000 7f5f6aa00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c0c4c80 con 0x7f5f6c071260 2024-09-21T06:27:33.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.261+0000 7f5f6b400640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 0x7f5f6c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.261+0000 7f8f80800640 1 -- start start 2024-09-21T06:27:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.257+0000 7f79e3e00640 1 -- 172.21.15.37:0/483888575 wait complete. 2024-09-21T06:27:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.265+0000 7f5f6aa00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c730 tx=0x7f5f5c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.268+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c017070 con 0x7f5f6c071260 2024-09-21T06:27:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.270+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f5c00ce40 con 0x7f5f6c071260 2024-09-21T06:27:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.271+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbf0801f020 con 0x7fbf1c151e30 2024-09-21T06:27:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.270+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c0c4f60 con 0x7f5f6c071260 2024-09-21T06:27:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.270+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c09a9e0 con 0x7f5f6c071260 2024-09-21T06:27:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.280+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c003850 con 0x7f5f6c071260 2024-09-21T06:27:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.281+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f5c01a070 con 0x7f5f6c071260 2024-09-21T06:27:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.281+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f5c00ae00 con 0x7f5f6c071260 2024-09-21T06:27:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.285+0000 7f8f80800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f44005290 con 0x7f5f6c071260 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.285+0000 7f8f80800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 0x7f8f7806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.285+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.285+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34818/0 (socket says 172.21.15.37:34818) 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.285+0000 7f8f7de00640 1 -- 172.21.15.37:0/3696764156 learned_addr learned my addr 172.21.15.37:0/3696764156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f80800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 0x7f8f78158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f7d400640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 0x7f8f7806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f780b6700 con 0x7f8f780d0170 2024-09-21T06:27:33.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f7e800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 0x7f8f78158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f780b6580 con 0x7f8f780d1120 2024-09-21T06:27:33.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.286+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8f780b6880 con 0x7f8f7815ee80 2024-09-21T06:27:33.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7e800640 1 -- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 msgr2=0x7f8f7806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7e800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 0x7f8f7806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7e800640 1 -- 172.21.15.37:0/3696764156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 msgr2=0x7f8f7806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7e800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7e800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7806e2f0 con 0x7f8f7815ee80 2024-09-21T06:27:33.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.292+0000 7f8f7de00640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.293+0000 7f8f7d400640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 0x7f8f7806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.293+0000 7f8f7e800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 0x7f8f78158610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f7815f630 tx=0x7f8f74003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.294+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74034040 con 0x7f8f7815ee80 2024-09-21T06:27:33.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.294+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7806e5d0 con 0x7f8f7815ee80 2024-09-21T06:27:33.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.295+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f780cbab0 con 0x7f8f7815ee80 2024-09-21T06:27:33.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.295+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5f5c021020 con 0x7f5f6c071260 2024-09-21T06:27:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.295+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f740045c0 con 0x7f8f7815ee80 2024-09-21T06:27:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.295+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74034040 con 0x7f8f7815ee80 2024-09-21T06:27:33.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.300+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8f74037020 con 0x7f8f7815ee80 2024-09-21T06:27:33.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.300+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8f74031660 con 0x7f8f7815ee80 2024-09-21T06:27:33.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.301+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54005290 con 0x7f8f7815ee80 2024-09-21T06:27:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.304+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8f7404a060 con 0x7f8f7815ee80 2024-09-21T06:27:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.304+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fc448005580 con 0x7fc46c06ec50 2024-09-21T06:27:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.338+0000 7fc469600640 1 -- 172.21.15.37:0/1113030347 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v11)=0 v11) ==== 115+0+0 (secure 0 0 0) 0x7fc45801f070 con 0x7fc46c06ec50 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 msgr2=0x7fc46c1539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc45800b410 tx=0x7fc45800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 shutdown_connections 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc46c15ac20 0x7fc46c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc46c06ec50 0x7fc46c1539a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 --2- 172.21.15.37:0/1113030347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46c06ddf0 0x7fc46c0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.342+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 >> 172.21.15.37:0/1113030347 conn(0x7fc46c0b1b90 msgr2=0x7fc46c158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.343+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 shutdown_connections 2024-09-21T06:27:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.343+0000 7fc474000640 1 -- 172.21.15.37:0/1113030347 wait complete. 2024-09-21T06:27:33.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.375+0000 7fa4f4a00640 1 -- 172.21.15.37:0/1107444284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8064810 con 0x7fa4dc004f40 2024-09-21T06:27:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.375+0000 7fa4fce00640 1 -- 172.21.15.37:0/1107444284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4dc004f40 msgr2=0x7fa4dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.375+0000 7fa4fce00640 1 --2- 172.21.15.37:0/1107444284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4dc004f40 0x7fa4dc00b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8052860 tx=0x7fa4dc005830 comp rx=0 tx=0).stop 2024-09-21T06:27:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.376+0000 7fa4f6800640 1 -- 172.21.15.37:0/1107444284 reap_dead start 2024-09-21T06:27:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.376+0000 7fa4fce00640 1 -- 172.21.15.37:0/1107444284 shutdown_connections 2024-09-21T06:27:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.376+0000 7fa4fce00640 1 -- 172.21.15.37:0/1107444284 >> 172.21.15.37:0/1107444284 conn(0x7fa4f001d9d0 msgr2=0x7fa4f0039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.381+0000 7fa4f5e00640 1 --2- 172.21.15.37:0/1107444284 >> v2:172.21.15.37:3300/0 conn(0x7fa4f0007310 0x7fa4f0035d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.382+0000 7fa4fce00640 1 -- 172.21.15.37:0/1107444284 shutdown_connections 2024-09-21T06:27:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.382+0000 7fa4fce00640 1 -- 172.21.15.37:0/1107444284 wait complete. 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.381+0000 7f8131800640 1 -- 172.21.15.37:0/13844212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8124036b60 msgr2=0x7f81240df1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.381+0000 7f8131800640 1 --2- 172.21.15.37:0/13844212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8124036b60 0x7f81240df1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8124054760 tx=0x7f811c02a540 comp rx=0 tx=0).stop 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.386+0000 7f8131800640 1 -- 172.21.15.37:0/13844212 shutdown_connections 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.386+0000 7f8131800640 1 --2- 172.21.15.37:0/13844212 >> v2:172.21.15.37:3300/0 conn(0x7f8124005c80 0x7f8124006680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.386+0000 7f8131800640 1 --2- 172.21.15.37:0/13844212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8124036b60 0x7f81240df1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.386+0000 7f8131800640 1 -- 172.21.15.37:0/13844212 >> 172.21.15.37:0/13844212 conn(0x7f8124051440 msgr2=0x7f8124051880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.386+0000 7f8131800640 1 -- 172.21.15.37:0/13844212 shutdown_connections 2024-09-21T06:27:33.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.395+0000 7fa4fce00640 1 Processor -- start 2024-09-21T06:27:33.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.395+0000 7fa4fce00640 1 -- start start 2024-09-21T06:27:33.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.399+0000 7fd076c00640 1 -- 172.21.15.37:0/645675143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 msgr2=0x7fd0700d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.399+0000 7fd076c00640 1 --2- 172.21.15.37:0/645675143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700d0170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005fd0 tx=0x7fd05c02a500 comp rx=0 tx=0).stop 2024-09-21T06:27:33.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.402+0000 7fa4fce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 0x7fa4f00e4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.393+0000 7f8131800640 1 -- 172.21.15.37:0/13844212 wait complete. 2024-09-21T06:27:33.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.403+0000 7f8131800640 1 Processor -- start 2024-09-21T06:27:33.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.403+0000 7fa4fce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 0x7fa4f00e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.403+0000 7f8131800640 1 -- start start 2024-09-21T06:27:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4fce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 0x7fa4f00decb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4fce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0031bf0 con 0x7fa4f00e1520 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4fce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0031a70 con 0x7fa4f017e4e0 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4f6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 0x7fa4f00decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4f6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 0x7fa4f00decb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44894/0 (socket says 172.21.15.37:44894) 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4f6800640 1 -- 172.21.15.37:0/2677571239 learned_addr learned my addr 172.21.15.37:0/2677571239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.407+0000 7fa4fce00640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f0031d70 con 0x7fa4f00de8a0 2024-09-21T06:27:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.408+0000 7fa4f5400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 0x7fa4f00e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.410+0000 7fa4f5400640 1 -- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 msgr2=0x7fa4f00decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.412+0000 7fa4f5400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 0x7fa4f00decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.411+0000 7fa4f5e00640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 0x7fa4f00e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.412+0000 7fa4f5400640 1 -- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 msgr2=0x7fa4f00e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.412+0000 7fa4f5400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 0x7fa4f00e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.412+0000 7fa4f5400640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00df1f0 con 0x7fa4f00e1520 2024-09-21T06:27:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.412+0000 7f8131800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 0x7f812401e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.416 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.416 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.413+0000 7fd076c00640 1 -- 172.21.15.37:0/645675143 shutdown_connections 2024-09-21T06:27:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.413+0000 7fa4f5e00640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 0x7fa4f00e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.414+0000 7fd076c00640 1 --2- 172.21.15.37:0/645675143 >> v2:172.21.15.150:3300/0 conn(0x7fd0700ceb90 0x7fd0700cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.414+0000 7fd076c00640 1 --2- 172.21.15.37:0/645675143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700d0170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.413+0000 7fa4f5400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 0x7fa4f00e28b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e5f0 tx=0x7fa4e400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.414+0000 7fd076c00640 1 -- 172.21.15.37:0/645675143 >> 172.21.15.37:0/645675143 conn(0x7fd07009d830 msgr2=0x7fd0700b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.414+0000 7fd076c00640 1 -- 172.21.15.37:0/645675143 shutdown_connections 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.414+0000 7fd076c00640 1 -- 172.21.15.37:0/645675143 wait complete. 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.415+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4015070 con 0x7fa4f00e1520 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.415+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e4004310 con 0x7fa4f00e1520 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.415+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400c9d0 con 0x7fa4f00e1520 2024-09-21T06:27:33.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.415+0000 7fa4fce00640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f00e1c90 con 0x7fa4f00e1520 2024-09-21T06:27:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.417+0000 7f8131800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.417+0000 7f8131800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 0x7f8124020c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.418+0000 7f812a000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.418+0000 7f812a000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44904/0 (socket says 172.21.15.37:44904) 2024-09-21T06:27:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.418+0000 7f812a000640 1 -- 172.21.15.37:0/3153254282 learned_addr learned my addr 172.21.15.37:0/3153254282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.418+0000 7f8131800640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8124006bc0 con 0x7f8124005c80 2024-09-21T06:27:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.418+0000 7f812aa00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 0x7f812401e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.419+0000 7fd076c00640 1 Processor -- start 2024-09-21T06:27:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.419+0000 7f8131800640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f812400be60 con 0x7f81240ced90 2024-09-21T06:27:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.419+0000 7fd076c00640 1 -- start start 2024-09-21T06:27:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.419+0000 7f8131800640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8124006d40 con 0x7f8124009bc0 2024-09-21T06:27:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.420+0000 7f812b400640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 0x7f8124020c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.420+0000 7fa4fce00640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f017dce0 con 0x7fa4f00e1520 2024-09-21T06:27:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.422+0000 7fd076c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7fd076c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0700ceb90 0x7fd0700cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7f812aa00640 1 -- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 msgr2=0x7f8124020710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7f812aa00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7f812aa00640 1 -- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 msgr2=0x7f8124020c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34868/0 (socket says 172.21.15.37:34868) 2024-09-21T06:27:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7fd06fe00640 1 -- 172.21.15.37:0/1989014860 learned_addr learned my addr 172.21.15.37:0/1989014860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7f812aa00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 0x7f8124020c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.423+0000 7fd076c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b6700 con 0x7fd0700b9ad0 2024-09-21T06:27:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7f812aa00640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812401f670 con 0x7f8124005c80 2024-09-21T06:27:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b6580 con 0x7fd0700ceb90 2024-09-21T06:27:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700b6880 con 0x7fd07015ee80 2024-09-21T06:27:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7f812a000640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7f812b400640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 0x7f8124020c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.424+0000 7fd074c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fd06fe00640 1 -- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 msgr2=0x7fd0700cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.425+0000 7f812aa00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 0x7f812401e940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f812c06b880 tx=0x7f812c06af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fd06fe00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fd06fe00640 1 -- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0700ceb90 msgr2=0x7fd0700cb610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:27:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4e400cb70 con 0x7fa4f00e1520 2024-09-21T06:27:33.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fd06fe00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0700ceb90 0x7fd0700cb610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c0753c0 con 0x7f8124005c80 2024-09-21T06:27:33.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.427+0000 7fd06fe00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700cc320 con 0x7fd0700b9ad0 2024-09-21T06:27:33.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.427+0000 7fd074c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.426+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc005290 con 0x7fa4f00e1520 2024-09-21T06:27:33.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.427+0000 7f8131800640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812401f950 con 0x7f8124005c80 2024-09-21T06:27:33.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.427+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4e4011680 con 0x7fa4f00e1520 2024-09-21T06:27:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.427+0000 7f8131800640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812401ff10 con 0x7f8124005c80 2024-09-21T06:27:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.428+0000 7fd06fe00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034040 tx=0x7fd05c003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.430+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f812c076270 con 0x7f8124005c80 2024-09-21T06:27:33.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.432+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c0753c0 con 0x7f8124005c80 2024-09-21T06:27:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.437+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa4f00df1f0 con 0x7fa4f00e1520 2024-09-21T06:27:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.437+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f812c076580 con 0x7f8124005c80 2024-09-21T06:27:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.437+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8100005290 con 0x7f8124005c80 2024-09-21T06:27:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.438+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f812c09c640 con 0x7f8124005c80 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 -- 172.21.15.37:0/3990783495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a81628e0 msgr2=0x7fe4a80cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a5600640 1 -- 172.21.15.37:0/3990783495 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494035020 con 0x7fe4a81628e0 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/3990783495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a81628e0 0x7fe4a80cf300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe494005fd0 tx=0x7fe49402a4b0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 -- 172.21.15.37:0/3990783495 shutdown_connections 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/3990783495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80cf840 0x7fe4a8154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/3990783495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a81628e0 0x7fe4a80cf300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.443+0000 7fe4a7e00640 1 -- 172.21.15.37:0/3990783495 >> 172.21.15.37:0/3990783495 conn(0x7fe4a80d1960 msgr2=0x7fe4a80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c040070 con 0x7fd0700b9ad0 2024-09-21T06:27:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd05c032730 con 0x7fd0700b9ad0 2024-09-21T06:27:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7f963b200640 1 -- 172.21.15.37:0/2816259030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634145ca0 msgr2=0x7f96340747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7f963b200640 1 --2- 172.21.15.37:0/2816259030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634145ca0 0x7f96340747a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9628009860 tx=0x7f962802a440 comp rx=0 tx=0).stop 2024-09-21T06:27:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0700cc600 con 0x7fd0700b9ad0 2024-09-21T06:27:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.444+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07006d160 con 0x7fd0700b9ad0 2024-09-21T06:27:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.446+0000 7f963b200640 1 -- 172.21.15.37:0/2816259030 shutdown_connections 2024-09-21T06:27:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.446+0000 7f963b200640 1 --2- 172.21.15.37:0/2816259030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634145ca0 0x7f96340747a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7f963b200640 1 --2- 172.21.15.37:0/2816259030 >> v2:172.21.15.37:3300/0 conn(0x7f9634070e70 0x7f9634074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7fe4a7e00640 1 -- 172.21.15.37:0/3990783495 shutdown_connections 2024-09-21T06:27:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7f963b200640 1 -- 172.21.15.37:0/2816259030 >> 172.21.15.37:0/2816259030 conn(0x7f96340ccb40 msgr2=0x7f96340ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7fe4a7e00640 1 -- 172.21.15.37:0/3990783495 wait complete. 2024-09-21T06:27:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f812c09a030 con 0x7f8124005c80 2024-09-21T06:27:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7f963b200640 1 -- 172.21.15.37:0/2816259030 shutdown_connections 2024-09-21T06:27:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.447+0000 7f963b200640 1 -- 172.21.15.37:0/2816259030 wait complete. 2024-09-21T06:27:33.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.452+0000 7f963b200640 1 Processor -- start 2024-09-21T06:27:33.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.453+0000 7f963b200640 1 -- start start 2024-09-21T06:27:33.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.456+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07006eae0 con 0x7fd0700b9ad0 2024-09-21T06:27:33.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.459+0000 7f963b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9634070e70 0x7f963415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.459+0000 7f963b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.459+0000 7f963b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634160040 0x7f96341f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f9633400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f9633400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34888/0 (socket says 172.21.15.37:34888) 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f9633400640 1 -- 172.21.15.37:0/396513371 learned_addr learned my addr 172.21.15.37:0/396513371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9634074fe0 con 0x7f9634145ca0 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9634145960 con 0x7f9634070e70 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9634075160 con 0x7f9634160040 2024-09-21T06:27:33.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f9633e00640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9634070e70 0x7f963415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7f9639200640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634160040 0x7f96341f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.461+0000 7f9633400640 1 -- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634160040 msgr2=0x7f96341f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.461+0000 7f9633400640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634160040 0x7f96341f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.461+0000 7f9633400640 1 -- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9634070e70 msgr2=0x7f963415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.461+0000 7f9633400640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9634070e70 0x7f963415bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.461+0000 7f9633400640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96341e74f0 con 0x7f9634145ca0 2024-09-21T06:27:33.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.462+0000 7f9633400640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f9628033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.460+0000 7fe4a7e00640 1 Processor -- start 2024-09-21T06:27:33.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.464+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628031d60 con 0x7f9634145ca0 2024-09-21T06:27:33.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.464+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9628033070 con 0x7f9634145ca0 2024-09-21T06:27:33.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.464+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628033480 con 0x7f9634145ca0 2024-09-21T06:27:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.464+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c032b20 con 0x7fd0700b9ad0 2024-09-21T06:27:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.464+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd05c043070 con 0x7fd0700b9ad0 2024-09-21T06:27:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.465+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd05c03c420 con 0x7fd0700b9ad0 2024-09-21T06:27:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.465+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd07006eae0 con 0x7fd0700b9ad0 2024-09-21T06:27:33.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.468+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341e77d0 con 0x7f9634145ca0 2024-09-21T06:27:33.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.468+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341e7d90 con 0x7f9634145ca0 2024-09-21T06:27:33.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.469+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9628033620 con 0x7f9634145ca0 2024-09-21T06:27:33.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.469+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f962803b750 con 0x7f9634145ca0 2024-09-21T06:27:33.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.470+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9634160e00 con 0x7f9634145ca0 2024-09-21T06:27:33.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.471+0000 7fe4a7e00640 1 -- start start 2024-09-21T06:27:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.471+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fbef4005850 con 0x7fbf1c151e30 2024-09-21T06:27:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.477+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 0x7fe4a814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.477+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9628052080 con 0x7f9634145ca0 2024-09-21T06:27:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.477+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 0x7fe4a8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.478+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 0x7fe4a8158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.478+0000 7fe4a7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81558b0 con 0x7fe4a80cf840 2024-09-21T06:27:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.478+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 0x7fe4a814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.479+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 0x7fe4a814dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50786/0 (socket says 172.21.15.37:50786) 2024-09-21T06:27:33.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.479+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f77600adcd0 con 0x7f77600cf550 2024-09-21T06:27:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.479+0000 7fe4a6a00640 1 -- 172.21.15.37:0/512040057 learned_addr learned my addr 172.21.15.37:0/512040057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.479+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8155730 con 0x7fe4a806d2b0 2024-09-21T06:27:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.480+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a8155a30 con 0x7fe4a80c9b40 2024-09-21T06:27:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.480+0000 7fe4a7400640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 0x7fe4a8158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.480+0000 7fe4a6000640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 0x7fe4a8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.481+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f8f54005580 con 0x7f8f7815ee80 2024-09-21T06:27:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a7400640 1 -- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 msgr2=0x7fe4a814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a7400640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 0x7fe4a814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a7400640 1 -- 172.21.15.37:0/512040057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 msgr2=0x7fe4a8157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a7400640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 0x7fe4a8157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a7400640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a814e8b0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.482+0000 7fe4a6000640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 0x7fe4a8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.483+0000 7fe4a7400640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 0x7fe4a8158370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b3c0 tx=0x7fe49800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.485+0000 7fbf12000640 1 -- 172.21.15.37:0/1167868700 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v12)=0 v12) ==== 120+0+0 (secure 0 0 0) 0x7fbf1c202910 con 0x7fbf1c151e30 2024-09-21T06:27:33.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.485+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980118f0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.486+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe498016070 con 0x7fe4a80c9b40 2024-09-21T06:27:33.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.486+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498011e40 con 0x7fe4a80c9b40 2024-09-21T06:27:33.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.488+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 msgr2=0x7fbf1c2023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.490+0000 7fbf21000640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c15be20 tx=0x7fbf0800b7a0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.487+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a80cc3d0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.487+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a814e5d0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.489+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe49801b460 con 0x7fe4a80c9b40 2024-09-21T06:27:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.489+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe49801a960 con 0x7fe4a80c9b40 2024-09-21T06:27:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.490+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a80cf300 con 0x7fe4a80c9b40 2024-09-21T06:27:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.493+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe49801a3c0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.495+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 shutdown_connections 2024-09-21T06:27:33.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.495+0000 7fbf21000640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf1c151e30 0x7fbf1c2023d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.495+0000 7fbf21000640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c156560 0x7fbf1c156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.495+0000 7fbf21000640 1 --2- 172.21.15.37:0/1167868700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf00003a70 0x7fbf1c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.495+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 >> 172.21.15.37:0/1167868700 conn(0x7fbf1c0d0350 msgr2=0x7fbf1c06f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.497+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 shutdown_connections 2024-09-21T06:27:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.496+0000 7f40c1e00640 1 -- 172.21.15.37:0/1261372975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a4003c80 msgr2=0x7f40a40040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.496+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1261372975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a4003c80 0x7f40a40040d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f40a8006e30 tx=0x7f40a4005830 comp rx=0 tx=0).stop 2024-09-21T06:27:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.497+0000 7fbf21000640 1 -- 172.21.15.37:0/1167868700 wait complete. 2024-09-21T06:27:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.498+0000 7f40c1e00640 1 -- 172.21.15.37:0/1261372975 shutdown_connections 2024-09-21T06:27:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.498+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1261372975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a4003c80 0x7f40a40040d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.498+0000 7f40c1e00640 1 -- 172.21.15.37:0/1261372975 >> 172.21.15.37:0/1261372975 conn(0x7f40bc0cfec0 msgr2=0x7f40bc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.501+0000 7f40c1e00640 1 -- 172.21.15.37:0/1261372975 shutdown_connections 2024-09-21T06:27:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.502+0000 7f40c1e00640 1 -- 172.21.15.37:0/1261372975 wait complete. 2024-09-21T06:27:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.506+0000 7f7756a00640 1 -- 172.21.15.37:0/1613458386 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f7750026070 con 0x7f77600cf550 2024-09-21T06:27:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 msgr2=0x7f77601fad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f775000e630 tx=0x7f775000eaf0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.511+0000 7f8f72000640 1 -- 172.21.15.37:0/3696764156 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f8f7404a060 con 0x7f8f7815ee80 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 shutdown_connections 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77601fb290 0x7f77601fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77600cf550 0x7f77601fad50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 --2- 172.21.15.37:0/1613458386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776014f7e0 0x7f7760151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.512+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 >> 172.21.15.37:0/1613458386 conn(0x7f776006cde0 msgr2=0x7f77600b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.514+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 shutdown_connections 2024-09-21T06:27:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.514+0000 7f7765a00640 1 -- 172.21.15.37:0/1613458386 wait complete. 2024-09-21T06:27:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 msgr2=0x7f8f78158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 0x7f8f78158610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f7815f630 tx=0x7f8f74003f10 comp rx=0 tx=0).stop 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 shutdown_connections 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f7815ee80 0x7f8f78158610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f780d1120 0x7f8f7806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 --2- 172.21.15.37:0/3696764156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f780d0170 0x7f8f7806d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 >> 172.21.15.37:0/3696764156 conn(0x7f8f7809d830 msgr2=0x7f8f780b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.516+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 shutdown_connections 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.517+0000 7f8f80800640 1 -- 172.21.15.37:0/3696764156 wait complete. 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.517+0000 7f40c1e00640 1 Processor -- start 2024-09-21T06:27:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.518+0000 7f40c1e00640 1 -- start start 2024-09-21T06:27:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.517+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f2f94024830 con 0x7f2f94033cc0 2024-09-21T06:27:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.523+0000 7f40c1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.523+0000 7f40c1e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 0x7f40bc157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.524+0000 7f40c0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.524+0000 7f40c0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50798/0 (socket says 172.21.15.37:50798) 2024-09-21T06:27:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.524+0000 7f40c0a00640 1 -- 172.21.15.37:0/1871154157 learned_addr learned my addr 172.21.15.37:0/1871154157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.524+0000 7f40bbe00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 0x7f40bc157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.525+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 0x7f40bc1fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.526+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc15f710 con 0x7f40bc157f50 2024-09-21T06:27:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.526+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc15f590 con 0x7f40a4003c80 2024-09-21T06:27:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.526+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc15f890 con 0x7f40bc157600 2024-09-21T06:27:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.526+0000 7f40c1400640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 0x7f40bc1fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.527+0000 7f40bbe00640 1 -- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 msgr2=0x7f40bc0742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.527+0000 7f40bbe00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.527+0000 7f40bbe00640 1 -- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 msgr2=0x7f40bc1fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.527+0000 7f40bbe00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 0x7f40bc1fd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.528+0000 7f40bbe00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc158940 con 0x7f40bc157600 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.528+0000 7f40c1400640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 0x7f40bc1fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.528+0000 7f40c0a00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.529+0000 7f40bbe00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 0x7f40bc157a10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0770f0 tx=0x7f40b000fb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.530+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0015070 con 0x7f40bc157600 2024-09-21T06:27:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.534+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc0b5030 con 0x7f40bc157600 2024-09-21T06:27:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.534+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc0b4790 con 0x7f40bc157600 2024-09-21T06:27:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.534+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40b0004340 con 0x7f40bc157600 2024-09-21T06:27:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.534+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000bad0 con 0x7f40bc157600 2024-09-21T06:27:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.537+0000 7f172ec00640 1 -- 172.21.15.37:0/254026929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 msgr2=0x7f1728155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.537+0000 7f1727e00640 1 -- 172.21.15.37:0/254026929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c031600 con 0x7f17280aef90 2024-09-21T06:27:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.537+0000 7f172ec00640 1 --2- 172.21.15.37:0/254026929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f1728155780 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f17280729d0 tx=0x7f171c02a4f0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.537+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f5f44005580 con 0x7f5f6c071260 2024-09-21T06:27:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 -- 172.21.15.37:0/254026929 shutdown_connections 2024-09-21T06:27:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 --2- 172.21.15.37:0/254026929 >> v2:172.21.15.150:3300/0 conn(0x7f172815f670 0x7f172806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 --2- 172.21.15.37:0/254026929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f1728155780 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 -- 172.21.15.37:0/254026929 >> 172.21.15.37:0/254026929 conn(0x7f17280cfec0 msgr2=0x7f17280d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40b000bc70 con 0x7f40bc157600 2024-09-21T06:27:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40b001a6b0 con 0x7f40bc157600 2024-09-21T06:27:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 -- 172.21.15.37:0/254026929 shutdown_connections 2024-09-21T06:27:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.541+0000 7f2f8fe00640 1 -- 172.21.15.37:0/2518017075 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f2f9c093b10 con 0x7f2f94033cc0 2024-09-21T06:27:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.540+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4098005290 con 0x7f40bc157600 2024-09-21T06:27:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.538+0000 7f172ec00640 1 -- 172.21.15.37:0/254026929 wait complete. 2024-09-21T06:27:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.544+0000 7f5f62a00640 1 -- 172.21.15.37:0/2000043871 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v16)=0 v16) ==== 119+0+0 (secure 0 0 0) 0x7f5f6c15e7d0 con 0x7f5f6c071260 2024-09-21T06:27:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 msgr2=0x7f2f940340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c076070 tx=0x7f2f9c06af00 comp rx=0 tx=0).stop 2024-09-21T06:27:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 shutdown_connections 2024-09-21T06:27:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f940d9c90 0x7f2f940da0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f94033cc0 0x7f2f940340d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 --2- 172.21.15.37:0/2518017075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f94186dd0 0x7f2f94051000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.545+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 >> 172.21.15.37:0/2518017075 conn(0x7f2f9401fbd0 msgr2=0x7f2f940de390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 msgr2=0x7f5f6c0b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f5f71a00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c730 tx=0x7f5f5c00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 shutdown_connections 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f5f71a00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f6c15e3c0 0x7f5f6c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f5f71a00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f6c073490 0x7f5f6c0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.550+0000 7f5f71a00640 1 --2- 172.21.15.37:0/2000043871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f6c071260 0x7f5f6c0b0fc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.550+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 >> 172.21.15.37:0/2000043871 conn(0x7f5f6c0b1b90 msgr2=0x7f5f6c06d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 shutdown_connections 2024-09-21T06:27:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f2fa3200640 1 -- 172.21.15.37:0/2518017075 wait complete. 2024-09-21T06:27:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.549+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f40b0029080 con 0x7f40bc157600 2024-09-21T06:27:33.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.554+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 shutdown_connections 2024-09-21T06:27:33.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.554+0000 7f5f71a00640 1 -- 172.21.15.37:0/2000043871 wait complete. 2024-09-21T06:27:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.565+0000 7f172ec00640 1 Processor -- start 2024-09-21T06:27:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.565+0000 7f172ec00640 1 -- start start 2024-09-21T06:27:33.578 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.582+0000 7f172ec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 0x7f17281524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f17281427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172ec00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172ec00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17280ce200 con 0x7f17280aef90 2024-09-21T06:27:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172ec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f17280ce080 con 0x7f1728070050 2024-09-21T06:27:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172ec00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f17280ce380 con 0x7f172815f670 2024-09-21T06:27:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.583+0000 7f172d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 0x7f17281524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.584+0000 7f172ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f17281427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.586+0000 7f172e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.586+0000 7f172d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 0x7f17281524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:50804/0 (socket says 172.21.15.37:50804) 2024-09-21T06:27:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.587+0000 7f172e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44988/0 (socket says 172.21.15.37:44988) 2024-09-21T06:27:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.587+0000 7f172e200640 1 -- 172.21.15.37:0/1670000353 learned_addr learned my addr 172.21.15.37:0/1670000353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.588+0000 7f172e200640 1 -- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 msgr2=0x7f17281524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.588+0000 7f172e200640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 0x7f17281524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.588+0000 7f172e200640 1 -- 172.21.15.37:0/1670000353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 msgr2=0x7f17281427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.588+0000 7f172e200640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f17281427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.588+0000 7f172e200640 1 -- 172.21.15.37:0/1670000353 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17281f98b0 con 0x7f172815f670 2024-09-21T06:27:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.590+0000 7f172ce00640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f17281427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.590+0000 7f172e200640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1720002230 tx=0x7f172000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.593+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720003b70 con 0x7f172815f670 2024-09-21T06:27:33.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.593+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1720003d10 con 0x7f172815f670 2024-09-21T06:27:33.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.593+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720005330 con 0x7f172815f670 2024-09-21T06:27:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.601+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1728157220 con 0x7f172815f670 2024-09-21T06:27:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.601+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f172814fba0 con 0x7f172815f670 2024-09-21T06:27:33.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.603+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172814fe00 con 0x7f172815f670 2024-09-21T06:27:33.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.605+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1720012630 con 0x7f172815f670 2024-09-21T06:27:33.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.605+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f172001b9a0 con 0x7f172815f670 2024-09-21T06:27:33.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.606+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fa4cc005850 con 0x7fa4f00e1520 2024-09-21T06:27:33.611 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.611 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.612 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.609+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1720020060 con 0x7f172815f670 2024-09-21T06:27:33.616 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.615+0000 7fa4ee000640 1 -- 172.21.15.37:0/2677571239 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v17)=0 v17) ==== 111+0+0 (secure 0 0 0) 0x7fa4e4011880 con 0x7fa4f00e1520 2024-09-21T06:27:33.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.616+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 msgr2=0x7fa4f00e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.616+0000 7fa4e3400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 0x7fa4f00e28b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e5f0 tx=0x7fa4e400eab0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f963406e6e0 con 0x7f9634145ca0 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 shutdown_connections 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7fa4e3400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f00e1520 0x7fa4f00e28b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7fa4e3400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f00de8a0 0x7fa4f00decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7fa4e3400640 1 --2- 172.21.15.37:0/2677571239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f017e4e0 0x7fa4f00e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.617+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 >> 172.21.15.37:0/2677571239 conn(0x7fa4f001d9d0 msgr2=0x7fa4f0004a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.618+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 shutdown_connections 2024-09-21T06:27:33.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.618+0000 7fa4e3400640 1 -- 172.21.15.37:0/2677571239 wait complete. 2024-09-21T06:27:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.625+0000 7f9627e00640 1 -- 172.21.15.37:0/396513371 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f962803f030 con 0x7f9634145ca0 2024-09-21T06:27:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.625+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f8100005580 con 0x7f8124005c80 2024-09-21T06:27:33.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.627+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 msgr2=0x7f96341e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.627+0000 7f963b200640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f9628033ce0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 shutdown_connections 2024-09-21T06:27:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9634145ca0 0x7f96341e68e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9634160040 0x7f96341f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 --2- 172.21.15.37:0/396513371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9634070e70 0x7f963415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 >> 172.21.15.37:0/396513371 conn(0x7f96340ccb40 msgr2=0x7f96340afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.628+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 shutdown_connections 2024-09-21T06:27:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.629+0000 7f963b200640 1 -- 172.21.15.37:0/396513371 wait complete. 2024-09-21T06:27:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.634+0000 7f8122a00640 1 -- 172.21.15.37:0/3153254282 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v19)=0 v19) ==== 119+0+0 (secure 0 0 0) 0x7f812c08f030 con 0x7f8124005c80 2024-09-21T06:27:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.635+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 msgr2=0x7f812401e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.635+0000 7f8117e00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 0x7f812401e940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f812c06b880 tx=0x7f812c06af20 comp rx=0 tx=0).stop 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 shutdown_connections 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81240ced90 0x7f8124020c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8124009bc0 0x7f8124020710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 --2- 172.21.15.37:0/3153254282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8124005c80 0x7f812401e940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 >> 172.21.15.37:0/3153254282 conn(0x7f8124051440 msgr2=0x7f8124052510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 shutdown_connections 2024-09-21T06:27:33.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.636+0000 7f8117e00640 1 -- 172.21.15.37:0/3153254282 wait complete. 2024-09-21T06:27:33.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.663+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fe4a814e030 con 0x7fe4a80c9b40 2024-09-21T06:27:33.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.667+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fd0700b6d90 con 0x7fd0700b9ad0 2024-09-21T06:27:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.669+0000 7fe49f400640 1 -- 172.21.15.37:0/512040057 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v20)=0 v20) ==== 125+0+0 (secure 0 0 0) 0x7fe49801abb0 con 0x7fe4a80c9b40 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 msgr2=0x7fe4a8158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 0x7fe4a8158370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b3c0 tx=0x7fe49800b880 comp rx=0 tx=0).stop 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 shutdown_connections 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80cf840 0x7fe4a8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4a80c9b40 0x7fe4a8158370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 --2- 172.21.15.37:0/512040057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4a806d2b0 0x7fe4a814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.670+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 >> 172.21.15.37:0/512040057 conn(0x7fe4a80d1960 msgr2=0x7fe4a80ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.671+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 shutdown_connections 2024-09-21T06:27:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.671+0000 7fe4a7e00640 1 -- 172.21.15.37:0/512040057 wait complete. 2024-09-21T06:27:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.673+0000 7fd067e00640 1 -- 172.21.15.37:0/1989014860 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v21)=0 v21) ==== 108+0+0 (secure 0 0 0) 0x7fd0701592d0 con 0x7fd0700b9ad0 2024-09-21T06:27:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 msgr2=0x7fd0700ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034040 tx=0x7fd05c003bf0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 shutdown_connections 2024-09-21T06:27:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd07015ee80 0x7fd0700cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0700ceb90 0x7fd0700cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 --2- 172.21.15.37:0/1989014860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0700b9ad0 0x7fd0700ca540 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 >> 172.21.15.37:0/1989014860 conn(0x7fd07009d830 msgr2=0x7fd0701608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 shutdown_connections 2024-09-21T06:27:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.675+0000 7fd076c00640 1 -- 172.21.15.37:0/1989014860 wait complete. 2024-09-21T06:27:33.681 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.681 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.701 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.720+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f4098005580 con 0x7f40bc157600 2024-09-21T06:27:33.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.725+0000 7f40b8c00640 1 -- 172.21.15.37:0/1871154157 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v22)=0 v22) ==== 106+0+0 (secure 0 0 0) 0x7f40b001a900 con 0x7f40bc157600 2024-09-21T06:27:33.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 msgr2=0x7f40bc157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 0x7f40bc157a10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0770f0 tx=0x7f40b000fb60 comp rx=0 tx=0).stop 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 shutdown_connections 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40bc157f50 0x7f40bc1fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40bc157600 0x7f40bc157a10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 --2- 172.21.15.37:0/1871154157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003c80 0x7f40bc0742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 >> 172.21.15.37:0/1871154157 conn(0x7f40bc0cfec0 msgr2=0x7f40bc159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 shutdown_connections 2024-09-21T06:27:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.726+0000 7f40c1e00640 1 -- 172.21.15.37:0/1871154157 wait complete. 2024-09-21T06:27:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.735+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f1728158f80 con 0x7f172815f670 2024-09-21T06:27:33.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.739+0000 7f1725600640 1 -- 172.21.15.37:0/1670000353 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f172806e8b0 con 0x7f172815f670 2024-09-21T06:27:33.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 msgr2=0x7f17281f8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1720002230 tx=0x7f172000cad0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 shutdown_connections 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17280aef90 0x7f17281427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f172815f670 0x7f17281f8b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 --2- 172.21.15.37:0/1670000353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1728070050 0x7f17281524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 >> 172.21.15.37:0/1670000353 conn(0x7f17280cfec0 msgr2=0x7f1728070460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 shutdown_connections 2024-09-21T06:27:33.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:33.740+0000 7f172ec00640 1 -- 172.21.15.37:0/1670000353 wait complete. 2024-09-21T06:27:33.752 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.752 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.764 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-21T06:27:33.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config dump 2024-09-21T06:27:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.021+0000 7f2c85400640 1 -- 172.21.15.37:0/1714862917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c800cbcd0 msgr2=0x7f2c8006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.021+0000 7f2c85400640 1 --2- 172.21.15.37:0/1714862917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c800cbcd0 0x7f2c8006cde0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2c78005fd0 tx=0x7f2c7802aa00 comp rx=0 tx=0).stop 2024-09-21T06:27:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 -- 172.21.15.37:0/1714862917 shutdown_connections 2024-09-21T06:27:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 --2- 172.21.15.37:0/1714862917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80074a70 0x7f2c800cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 --2- 172.21.15.37:0/1714862917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c800cbcd0 0x7f2c8006cde0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 -- 172.21.15.37:0/1714862917 >> 172.21.15.37:0/1714862917 conn(0x7f2c800d1140 msgr2=0x7f2c800d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:34.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 -- 172.21.15.37:0/1714862917 shutdown_connections 2024-09-21T06:27:34.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.022+0000 7f2c85400640 1 -- 172.21.15.37:0/1714862917 wait complete. 2024-09-21T06:27:34.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.024+0000 7f2c85400640 1 Processor -- start 2024-09-21T06:27:34.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.024+0000 7f2c85400640 1 -- start start 2024-09-21T06:27:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80070f10 0x7f2c80150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 0x7f2c801f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8006d8a0 con 0x7f2c800cbcd0 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8006d720 con 0x7f2c80074a70 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c85400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8006da20 con 0x7f2c80070f10 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7e800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80070f10 0x7f2c80150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 0x7f2c801f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35276/0 (socket says 172.21.15.37:35276) 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 -- 172.21.15.37:0/3550399408 learned_addr learned my addr 172.21.15.37:0/3550399408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 -- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80070f10 msgr2=0x7f2c80150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80070f10 0x7f2c80150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 -- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 msgr2=0x7f2c801f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 0x7f2c801f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7f200640 1 -- 172.21.15.37:0/3550399408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c800c5290 con 0x7f2c800cbcd0 2024-09-21T06:27:34.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.026+0000 7f2c7de00640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 0x7f2c801f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c7f200640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78046070 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c78004540 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c800cee90 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78033070 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.027+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c800cf150 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.028+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c78049070 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.028+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c7804b780 con 0x7f2c800cbcd0 2024-09-21T06:27:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.029+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c800cb790 con 0x7f2c800cbcd0 2024-09-21T06:27:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.031+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2c78050080 con 0x7f2c800cbcd0 2024-09-21T06:27:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.098+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2c8006dd60 con 0x7f2c800cbcd0 2024-09-21T06:27:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.098+0000 7f2c76a00640 1 -- 172.21.15.37:0/3550399408 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f2c7803e070 con 0x7f2c800cbcd0 2024-09-21T06:27:34.100 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:27:34.101 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:27:34.102 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 msgr2=0x7f2c801f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78003f10 comp rx=0 tx=0).stop 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 shutdown_connections 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c800cbcd0 0x7f2c801f8fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c80074a70 0x7f2c801f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 --2- 172.21.15.37:0/3550399408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c80070f10 0x7f2c80150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 >> 172.21.15.37:0/3550399408 conn(0x7f2c800d1140 msgr2=0x7f2c800b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:34.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.099+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 shutdown_connections 2024-09-21T06:27:34.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.100+0000 7f2c85400640 1 -- 172.21.15.37:0/3550399408 wait complete. 2024-09-21T06:27:34.123 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T06:27:34.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 -- 172.21.15.37:0/2669313070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 msgr2=0x7ff0000d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 --2- 172.21.15.37:0/2669313070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff0000d0be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff0000cfba0 tx=0x7fefe802a4b0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 -- 172.21.15.37:0/2669313070 shutdown_connections 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 --2- 172.21.15.37:0/2669313070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 0x7ff0000ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 --2- 172.21.15.37:0/2669313070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff0000d0be0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 --2- 172.21.15.37:0/2669313070 >> v2:172.21.15.150:3300/0 conn(0x7ff00015ee80 0x7ff0000d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.342+0000 7ff006200640 1 -- 172.21.15.37:0/2669313070 >> 172.21.15.37:0/2669313070 conn(0x7ff00009d830 msgr2=0x7ff0000b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.343+0000 7ff006200640 1 -- 172.21.15.37:0/2669313070 shutdown_connections 2024-09-21T06:27:34.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.343+0000 7ff006200640 1 -- 172.21.15.37:0/2669313070 wait complete. 2024-09-21T06:27:34.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.345+0000 7ff006200640 1 Processor -- start 2024-09-21T06:27:34.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.345+0000 7ff006200640 1 -- start start 2024-09-21T06:27:34.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.346+0000 7ff006200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7ff006200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 0x7ff0000d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7ff006200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 0x7ff0000d4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:34.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7ff006200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000cf520 con 0x7ff0000b9ad0 2024-09-21T06:27:34.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7ff006200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000cf3a0 con 0x7ff00015ee80 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7ff006200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff0000b6580 con 0x7ff0000d1120 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35308/0 (socket says 172.21.15.37:35308) 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 -- 172.21.15.37:0/2067346967 learned_addr learned my addr 172.21.15.37:0/2067346967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7feff7e00640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 0x7ff0000d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefffe00640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 0x7ff0000d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 -- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 msgr2=0x7ff0000d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 0x7ff0000d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 -- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 msgr2=0x7ff0000d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 0x7ff0000d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefff400640 1 -- 172.21.15.37:0/2067346967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00006e750 con 0x7ff0000b9ad0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.347+0000 7fefffe00640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 0x7ff0000d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7fefff400640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feff000e790 tx=0x7feff000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000cab0 con 0x7ff0000b9ad0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feff000cd40 con 0x7ff0000b9ad0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00006ea30 con 0x7ff0000b9ad0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0010690 con 0x7ff0000b9ad0 2024-09-21T06:27:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00006eff0 con 0x7ff0000b9ad0 2024-09-21T06:27:34.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.348+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feff0010830 con 0x7ff0000b9ad0 2024-09-21T06:27:34.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.349+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feff0018c60 con 0x7ff0000b9ad0 2024-09-21T06:27:34.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.349+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0000b5bd0 con 0x7ff0000b9ad0 2024-09-21T06:27:34.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.352+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff0000b5bd0 con 0x7ff0000b9ad0 2024-09-21T06:27:34.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.420+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff0000ceb70 con 0x7ff0000b9ad0 2024-09-21T06:27:34.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.423+0000 7feff7400640 1 -- 172.21.15.37:0/2067346967 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7feff0010c60 con 0x7ff0000b9ad0 2024-09-21T06:27:34.425 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-09-21T06:27:34.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 msgr2=0x7ff00014fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:34.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feff000e790 tx=0x7feff000eb90 comp rx=0 tx=0).stop 2024-09-21T06:27:34.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 shutdown_connections 2024-09-21T06:27:34.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00015ee80 0x7ff0000d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0000d1120 0x7ff0000d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 --2- 172.21.15.37:0/2067346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0000b9ad0 0x7ff00014fa20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:34.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 >> 172.21.15.37:0/2067346967 conn(0x7ff00009d830 msgr2=0x7ff000161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:34.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 shutdown_connections 2024-09-21T06:27:34.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:34.424+0000 7ff006200640 1 -- 172.21.15.37:0/2067346967 wait complete. 2024-09-21T06:27:34.447 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T06:27:34.447 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T06:27:34.447 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:34.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T06:27:34.505 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:34.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T06:27:34.601 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:34.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T06:27:34.701 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:27:34.701 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T06:27:34.797 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:34.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T06:27:34.854 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:34.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T06:27:34.951 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:34.951 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T06:27:35.047 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:27:35.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T06:27:35.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234 0 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 -- 172.21.15.150:0/1269010666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf80d3420 msgr2=0x7fccf80d3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 --2- 172.21.15.150:0/1269010666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf80d3420 0x7fccf80d3830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005fd0 tx=0x7fcce402a560 comp rx=0 tx=0).stop 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 -- 172.21.15.150:0/1269010666 shutdown_connections 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 --2- 172.21.15.150:0/1269010666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf80d3420 0x7fccf80d3830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 -- 172.21.15.150:0/1269010666 >> 172.21.15.150:0/1269010666 conn(0x7fccf80d1980 msgr2=0x7fccf80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 -- 172.21.15.150:0/1269010666 shutdown_connections 2024-09-21T06:27:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.403+0000 7fccfe600640 1 -- 172.21.15.150:0/1269010666 wait complete. 2024-09-21T06:27:35.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.405+0000 7fccfe600640 1 Processor -- start 2024-09-21T06:27:35.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.405+0000 7fccfe600640 1 -- start start 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.406+0000 7fccfe600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccf80d3420 0x7fccf814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfe600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfe600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf81620c0 0x7fccf80b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfe600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccf806e0a0 con 0x7fccf80d3420 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfe600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccf806df20 con 0x7fccf81620c0 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfe600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fccf806e220 con 0x7fccf815e4d0 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfdc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf81620c0 0x7fccf80b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:48280/0 (socket says 172.21.15.150:48280) 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 -- 172.21.15.150:0/1375791961 learned_addr learned my addr 172.21.15.150:0/1375791961 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:35.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccfd200640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccf80d3420 0x7fccf814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 -- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf81620c0 msgr2=0x7fccf80b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf81620c0 0x7fccf80b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 -- 172.21.15.150:0/1375791961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccf80d3420 msgr2=0x7fccf814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccf80d3420 0x7fccf814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.407+0000 7fccf7e00640 1 -- 172.21.15.150:0/1375791961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf814b490 con 0x7fccf815e4d0 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccf7e00640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcce4034040 tx=0x7fcce40039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce403e030 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcce40047a0 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf814da20 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4035040 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf814dd00 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.408+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcce4041030 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.409+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fcce4043dd0 con 0x7fccf815e4d0 2024-09-21T06:27:35.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.409+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf814e700 con 0x7fccf815e4d0 2024-09-21T06:27:35.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.411+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcce4043410 con 0x7fccf815e4d0 2024-09-21T06:27:35.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.486+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234", "id": 0} v 0) -- 0x7fccf80d3830 con 0x7fccf815e4d0 2024-09-21T06:27:35.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.491+0000 7fccf4c00640 1 -- 172.21.15.150:0/1375791961 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fcce40435f0 con 0x7fccf815e4d0 2024-09-21T06:27:35.493 INFO:teuthology.orchestra.run.smithi150.stdout:0 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.492+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 msgr2=0x7fccf81f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.492+0000 7fccfe600640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcce4034040 tx=0x7fcce40039e0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 shutdown_connections 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccf81620c0 0x7fccf80b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccf815e4d0 0x7fccf81f93f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 --2- 172.21.15.150:0/1375791961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccf80d3420 0x7fccf814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 >> 172.21.15.150:0/1375791961 conn(0x7fccf80d1980 msgr2=0x7fccf815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 shutdown_connections 2024-09-21T06:27:35.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.493+0000 7fccfe600640 1 -- 172.21.15.150:0/1375791961 wait complete. 2024-09-21T06:27:35.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 737fcbb4-0a68-4b75-9ba3-1372051e3295 1 2024-09-21T06:27:35.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 -- 172.21.15.150:0/192086466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 msgr2=0x7f68a80afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 --2- 172.21.15.150:0/192086466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a80afd70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6898009830 tx=0x7f689802a530 comp rx=0 tx=0).stop 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 -- 172.21.15.150:0/192086466 shutdown_connections 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 --2- 172.21.15.150:0/192086466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a80afd70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 -- 172.21.15.150:0/192086466 >> 172.21.15.150:0/192086466 conn(0x7f68a80d0a30 msgr2=0x7f68a80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 -- 172.21.15.150:0/192086466 shutdown_connections 2024-09-21T06:27:35.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.734+0000 7f68acc00640 1 -- 172.21.15.150:0/192086466 wait complete. 2024-09-21T06:27:35.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.736+0000 7f68acc00640 1 Processor -- start 2024-09-21T06:27:35.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.736+0000 7f68acc00640 1 -- start start 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68a80b4b50 0x7f68a8153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68a815b3c0 con 0x7f68a806e1d0 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68a815b240 con 0x7f68a80b4740 2024-09-21T06:27:35.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68acc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f68a815b540 con 0x7f68a80b4b50 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f689fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.150:44360/0 (socket says 172.21.15.150:44360) 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68a6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68a80b4b50 0x7f68a8153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f689fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44842/0 (socket says 172.21.15.150:44842) 2024-09-21T06:27:35.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.738+0000 7f68a5e00640 1 -- 172.21.15.150:0/1020445742 learned_addr learned my addr 172.21.15.150:0/1020445742 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:35.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 -- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68a80b4b50 msgr2=0x7f68a8153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68a80b4b50 0x7f68a8153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 -- 172.21.15.150:0/1020445742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 msgr2=0x7f68a81555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 -- 172.21.15.150:0/1020445742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a815bf10 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f68a5e00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689fe00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f689000c3c0 tx=0x7f689000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011790 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a815c1f0 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011cc0 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.739+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a815c7b0 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.740+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f689001c460 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.740+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f6890012920 con 0x7f68a80b4740 2024-09-21T06:27:35.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.741+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a815bc80 con 0x7f68a80b4740 2024-09-21T06:27:35.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.743+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6890012420 con 0x7f68a80b4740 2024-09-21T06:27:35.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.816+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "737fcbb4-0a68-4b75-9ba3-1372051e3295", "id": 1} v 0) -- 0x7f68a80afd70 con 0x7f68a80b4740 2024-09-21T06:27:35.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.821+0000 7f689e000640 1 -- 172.21.15.150:0/1020445742 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "737fcbb4-0a68-4b75-9ba3-1372051e3295", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f6890012b20 con 0x7f68a80b4740 2024-09-21T06:27:35.824 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.822+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 msgr2=0x7f68a8159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f689000c3c0 tx=0x7f689000c7c0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 shutdown_connections 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68a80b4b50 0x7f68a8153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a80b4740 0x7f68a8159730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 --2- 172.21.15.150:0/1020445742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68a806e1d0 0x7f68a81555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 >> 172.21.15.150:0/1020445742 conn(0x7f68a80d0a30 msgr2=0x7f68a815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:35.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 shutdown_connections 2024-09-21T06:27:35.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:35.823+0000 7f68acc00640 1 -- 172.21.15.150:0/1020445742 wait complete. 2024-09-21T06:27:35.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 814b0688-5744-4302-a267-b7ed03896600 2 2024-09-21T06:27:36.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 -- 172.21.15.150:0/3213832330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a081620c0 msgr2=0x7f1a081624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 --2- 172.21.15.150:0/3213832330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a081620c0 0x7f1a081624d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b72b0 tx=0x7f1a0002a9a0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 -- 172.21.15.150:0/3213832330 shutdown_connections 2024-09-21T06:27:36.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 --2- 172.21.15.150:0/3213832330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a081620c0 0x7f1a081624d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 --2- 172.21.15.150:0/3213832330 >> v2:172.21.15.37:3300/0 conn(0x7f1a080d3420 0x7f1a080d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 -- 172.21.15.150:0/3213832330 >> 172.21.15.150:0/3213832330 conn(0x7f1a080d1980 msgr2=0x7f1a080d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 -- 172.21.15.150:0/3213832330 shutdown_connections 2024-09-21T06:27:36.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.064+0000 7f1a0e200640 1 -- 172.21.15.150:0/3213832330 wait complete. 2024-09-21T06:27:36.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.066+0000 7f1a0e200640 1 Processor -- start 2024-09-21T06:27:36.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.067+0000 7f1a0e200640 1 -- start start 2024-09-21T06:27:36.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 0x7f1a081f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 0x7f1a081f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0806e020 con 0x7f1a081620c0 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0806dea0 con 0x7f1a080d3420 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0e200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0806e1a0 con 0x7f1a0815e4d0 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 0x7f1a081f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44856/0 (socket says 172.21.15.150:44856) 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0ce00640 1 -- 172.21.15.150:0/4082854105 learned_addr learned my addr 172.21.15.150:0/4082854105 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.068+0000 7f1a0d800640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 0x7f1a081f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 -- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 msgr2=0x7f1a081f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 0x7f1a081f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 -- 172.21.15.150:0/4082854105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 msgr2=0x7f1a081f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 0x7f1a081f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 -- 172.21.15.150:0/4082854105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081f9430 con 0x7f1a080d3420 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a07e00640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 0x7f1a081f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0d800640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 0x7f1a081f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0ce00640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c3a0 tx=0x7f19fc00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc017070 con 0x7f1a080d3420 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19fc0053c0 con 0x7f1a080d3420 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a080b5970 con 0x7f1a080d3420 2024-09-21T06:27:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.069+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc003850 con 0x7f1a080d3420 2024-09-21T06:27:36.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.070+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a080b5f30 con 0x7f1a080d3420 2024-09-21T06:27:36.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.070+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19fc01a070 con 0x7f1a080d3420 2024-09-21T06:27:36.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.070+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f19fc011d60 con 0x7f1a080d3420 2024-09-21T06:27:36.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.071+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a080d3830 con 0x7f1a080d3420 2024-09-21T06:27:36.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.073+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f19fc021020 con 0x7f1a080d3420 2024-09-21T06:27:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.150+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "814b0688-5744-4302-a267-b7ed03896600", "id": 2} v 0) -- 0x7f1a080d42b0 con 0x7f1a080d3420 2024-09-21T06:27:36.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.154+0000 7f1a04c00640 1 -- 172.21.15.150:0/4082854105 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "814b0688-5744-4302-a267-b7ed03896600", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f1a08162ea0 con 0x7f1a080d3420 2024-09-21T06:27:36.156 INFO:teuthology.orchestra.run.smithi150.stdout:2 2024-09-21T06:27:36.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 msgr2=0x7f1a080b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c3a0 tx=0x7f19fc00c860 comp rx=0 tx=0).stop 2024-09-21T06:27:36.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 shutdown_connections 2024-09-21T06:27:36.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a081620c0 0x7f1a081f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a0815e4d0 0x7f1a081f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 --2- 172.21.15.150:0/4082854105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a080d3420 0x7f1a080b5340 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 >> 172.21.15.150:0/4082854105 conn(0x7f1a080d1980 msgr2=0x7f1a080ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 shutdown_connections 2024-09-21T06:27:36.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.155+0000 7f1a0e200640 1 -- 172.21.15.150:0/4082854105 wait complete. 2024-09-21T06:27:36.177 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0 3 2024-09-21T06:27:36.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 -- 172.21.15.150:0/3431083020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2100d3420 msgr2=0x7fb2100d3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 --2- 172.21.15.150:0/3431083020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2100d3420 0x7fb2100d3830 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2100cff30 tx=0x7fb1f802a4f0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 -- 172.21.15.150:0/3431083020 shutdown_connections 2024-09-21T06:27:36.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 --2- 172.21.15.150:0/3431083020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2100d3420 0x7fb2100d3830 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 -- 172.21.15.150:0/3431083020 >> 172.21.15.150:0/3431083020 conn(0x7fb2100d1980 msgr2=0x7fb2100d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 -- 172.21.15.150:0/3431083020 shutdown_connections 2024-09-21T06:27:36.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.394+0000 7fb214a00640 1 -- 172.21.15.150:0/3431083020 wait complete. 2024-09-21T06:27:36.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.396+0000 7fb214a00640 1 Processor -- start 2024-09-21T06:27:36.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.396+0000 7fb214a00640 1 -- start start 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2100d3420 0x7fb210149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 0x7fb21014a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb21006e060 con 0x7fb2100d3420 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb21006dee0 con 0x7fb2101620c0 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb214a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb21006e1e0 con 0x7fb21015e4d0 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb207e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb20f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 0x7fb21014a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb20e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2100d3420 0x7fb210149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb207e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:48310/0 (socket says 172.21.15.150:48310) 2024-09-21T06:27:36.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.398+0000 7fb207e00640 1 -- 172.21.15.150:0/3255092104 learned_addr learned my addr 172.21.15.150:0/3255092104 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 -- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 msgr2=0x7fb21014a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 0x7fb21014a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 -- 172.21.15.150:0/3255092104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2100d3420 msgr2=0x7fb210149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2100d3420 0x7fb210149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 -- 172.21.15.150:0/3255092104 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21014a480 con 0x7fb21015e4d0 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb20f200640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 0x7fb21014a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb207e00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009950 tx=0x7fb1f8033d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8031d60 con 0x7fb21015e4d0 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1f8005020 con 0x7fb21015e4d0 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2100c9270 con 0x7fb21015e4d0 2024-09-21T06:27:36.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8005450 con 0x7fb21015e4d0 2024-09-21T06:27:36.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.399+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2100c9830 con 0x7fb21015e4d0 2024-09-21T06:27:36.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.400+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb1f802ad10 con 0x7fb21015e4d0 2024-09-21T06:27:36.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.400+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb1f80055f0 con 0x7fb21015e4d0 2024-09-21T06:27:36.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.400+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2100c8fe0 con 0x7fb21015e4d0 2024-09-21T06:27:36.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.403+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb1f803f030 con 0x7fb21015e4d0 2024-09-21T06:27:36.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.477+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0", "id": 3} v 0) -- 0x7fb2100d3830 con 0x7fb21015e4d0 2024-09-21T06:27:36.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.481+0000 7fb206a00640 1 -- 172.21.15.150:0/3255092104 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb1f8031700 con 0x7fb21015e4d0 2024-09-21T06:27:36.483 INFO:teuthology.orchestra.run.smithi150.stdout:3 2024-09-21T06:27:36.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 msgr2=0x7fb2100ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009950 tx=0x7fb1f8033d60 comp rx=0 tx=0).stop 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 shutdown_connections 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2101620c0 0x7fb21014a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb21015e4d0 0x7fb2100ca310 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 --2- 172.21.15.150:0/3255092104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2100d3420 0x7fb210149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 >> 172.21.15.150:0/3255092104 conn(0x7fb2100d1980 msgr2=0x7fb210160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 shutdown_connections 2024-09-21T06:27:36.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.482+0000 7fb214a00640 1 -- 172.21.15.150:0/3255092104 wait complete. 2024-09-21T06:27:36.506 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 9d41f63c-f130-4048-8aea-3bb2e496b081 4 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 -- 172.21.15.150:0/2984752667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf94162280 msgr2=0x7faf94162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 --2- 172.21.15.150:0/2984752667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf94162280 0x7faf94162690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009830 tx=0x7faf7c02a510 comp rx=0 tx=0).stop 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 -- 172.21.15.150:0/2984752667 shutdown_connections 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 --2- 172.21.15.150:0/2984752667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 --2- 172.21.15.150:0/2984752667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf94162280 0x7faf94162690 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 -- 172.21.15.150:0/2984752667 >> 172.21.15.150:0/2984752667 conn(0x7faf940b6b20 msgr2=0x7faf940d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 -- 172.21.15.150:0/2984752667 shutdown_connections 2024-09-21T06:27:36.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.721+0000 7faf99000640 1 -- 172.21.15.150:0/2984752667 wait complete. 2024-09-21T06:27:36.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.724+0000 7faf99000640 1 Processor -- start 2024-09-21T06:27:36.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.724+0000 7faf99000640 1 -- start start 2024-09-21T06:27:36.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf941591b0 0x7faf94156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf9415e4b0 con 0x7faf941591b0 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf940b39d0 con 0x7faf94162280 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf99000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faf9415e630 con 0x7faf940b2e40 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.725+0000 7faf91e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44878/0 (socket says 172.21.15.150:44878) 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 -- 172.21.15.150:0/664348066 learned_addr learned my addr 172.21.15.150:0/664348066 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:36.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf92800640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf941591b0 0x7faf94156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:36.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 -- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 msgr2=0x7faf940b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 -- 172.21.15.150:0/664348066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf941591b0 msgr2=0x7faf94156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf941591b0 0x7faf94156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 -- 172.21.15.150:0/664348066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf940b2a40 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf91e00640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.726+0000 7faf8be00640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009950 tx=0x7faf7c033c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c031d60 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf7c033e10 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c033310 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf940aaf80 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf940ab540 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf7c0334b0 con 0x7faf94162280 2024-09-21T06:27:36.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.727+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7faf7c003cd0 con 0x7faf94162280 2024-09-21T06:27:36.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.728+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf94162f00 con 0x7faf94162280 2024-09-21T06:27:36.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.732+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faf7c03f030 con 0x7faf94162280 2024-09-21T06:27:36.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.806+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9d41f63c-f130-4048-8aea-3bb2e496b081", "id": 4} v 0) -- 0x7faf940b2760 con 0x7faf94162280 2024-09-21T06:27:36.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.812+0000 7faf8a000640 1 -- 172.21.15.150:0/664348066 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d41f63c-f130-4048-8aea-3bb2e496b081", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7faf7c0035f0 con 0x7faf94162280 2024-09-21T06:27:36.814 INFO:teuthology.orchestra.run.smithi150.stdout:4 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 msgr2=0x7faf940aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009950 tx=0x7faf7c033c20 comp rx=0 tx=0).stop 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 shutdown_connections 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf941591b0 0x7faf94156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf94162280 0x7faf940aba60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 --2- 172.21.15.150:0/664348066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf940b2e40 0x7faf940b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 >> 172.21.15.150:0/664348066 conn(0x7faf940b6b20 msgr2=0x7faf9415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 shutdown_connections 2024-09-21T06:27:36.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:36.813+0000 7faf99000640 1 -- 172.21.15.150:0/664348066 wait complete. 2024-09-21T06:27:36.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new fe3eab60-21a1-4841-baa9-a8c004fe90c7 5 2024-09-21T06:27:37.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 -- 172.21.15.150:0/134737599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 msgr2=0x7f5dbc0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 --2- 172.21.15.150:0/134737599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc0b4f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009be0 tx=0x7f5db402a920 comp rx=0 tx=0).stop 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 -- 172.21.15.150:0/134737599 shutdown_connections 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 --2- 172.21.15.150:0/134737599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc0b4f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 -- 172.21.15.150:0/134737599 >> 172.21.15.150:0/134737599 conn(0x7f5dbc06f920 msgr2=0x7f5dbc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 -- 172.21.15.150:0/134737599 shutdown_connections 2024-09-21T06:27:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.052+0000 7f5dc1600640 1 -- 172.21.15.150:0/134737599 wait complete. 2024-09-21T06:27:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.054+0000 7f5dc1600640 1 Processor -- start 2024-09-21T06:27:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.054+0000 7f5dc1600640 1 -- start start 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 0x7f5dbc1f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 0x7f5dbc15f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc06d950 con 0x7f5dbc06e370 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc06d7d0 con 0x7f5dbc15f070 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc1600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc06dad0 con 0x7f5dbc156c60 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dc0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 0x7f5dbc15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dbbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 0x7f5dbc1f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:48360/0 (socket says 172.21.15.150:48360) 2024-09-21T06:27:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.056+0000 7f5dbb400640 1 -- 172.21.15.150:0/699447624 learned_addr learned my addr 172.21.15.150:0/699447624 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 -- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 msgr2=0x7f5dbc143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 -- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 msgr2=0x7f5dbc15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 0x7f5dbc15f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 -- 172.21.15.150:0/699447624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc15f9c0 con 0x7f5dbc06e370 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dc0c00640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 0x7f5dbc15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbb400640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dbbe00640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 0x7f5dbc1f9730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c3c0 tx=0x7f5dac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac011790 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dac017070 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac011ce0 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.057+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1504d0 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.058+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1509d0 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.058+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5dac01c460 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.058+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5dac012cc0 con 0x7f5dbc06e370 2024-09-21T06:27:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.059+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc0b4f90 con 0x7f5dbc06e370 2024-09-21T06:27:37.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.061+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5dbc15f9c0 con 0x7f5dbc06e370 2024-09-21T06:27:37.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.076+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f5dac012740 con 0x7f5dbc06e370 2024-09-21T06:27:37.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.148+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe3eab60-21a1-4841-baa9-a8c004fe90c7", "id": 5} v 0) -- 0x7f5dbc074c10 con 0x7f5dbc06e370 2024-09-21T06:27:37.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.150+0000 7f5db3e00640 1 -- 172.21.15.150:0/699447624 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe3eab60-21a1-4841-baa9-a8c004fe90c7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5dac011e80 con 0x7f5dbc06e370 2024-09-21T06:27:37.152 INFO:teuthology.orchestra.run.smithi150.stdout:5 2024-09-21T06:27:37.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 msgr2=0x7f5dbc1f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 0x7f5dbc1f9730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c3c0 tx=0x7f5dac00c880 comp rx=0 tx=0).stop 2024-09-21T06:27:37.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 shutdown_connections 2024-09-21T06:27:37.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dbc15f070 0x7f5dbc15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5dbc156c60 0x7f5dbc143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 --2- 172.21.15.150:0/699447624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dbc06e370 0x7f5dbc1f9730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 >> 172.21.15.150:0/699447624 conn(0x7f5dbc06f920 msgr2=0x7f5dbc15e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 shutdown_connections 2024-09-21T06:27:37.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.151+0000 7f5dc1600640 1 -- 172.21.15.150:0/699447624 wait complete. 2024-09-21T06:27:37.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new af143beb-c0f0-4231-92ac-b08f9865c704 6 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.388+0000 7f76eb400640 1 -- 172.21.15.150:0/3510850634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e40cef00 msgr2=0x7f76e40cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.388+0000 7f76eb400640 1 --2- 172.21.15.150:0/3510850634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e40cef00 0x7f76e40cf310 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009830 tx=0x7f76dc02a520 comp rx=0 tx=0).stop 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 -- 172.21.15.150:0/3510850634 shutdown_connections 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 --2- 172.21.15.150:0/3510850634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e40cef00 0x7f76e40cf310 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 --2- 172.21.15.150:0/3510850634 >> v2:172.21.15.150:3300/0 conn(0x7f76e406ddf0 0x7f76e406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 -- 172.21.15.150:0/3510850634 >> 172.21.15.150:0/3510850634 conn(0x7f76e40b1b90 msgr2=0x7f76e40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 -- 172.21.15.150:0/3510850634 shutdown_connections 2024-09-21T06:27:37.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.389+0000 7f76eb400640 1 -- 172.21.15.150:0/3510850634 wait complete. 2024-09-21T06:27:37.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.391+0000 7f76eb400640 1 Processor -- start 2024-09-21T06:27:37.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.391+0000 7f76eb400640 1 -- start start 2024-09-21T06:27:37.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 0x7f76e4155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 0x7f76e409f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76e40cf850 con 0x7f76e406ddf0 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4161060 con 0x7f76e40cef00 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.392+0000 7f76eb400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f76e40cf9d0 con 0x7f76e4156a50 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e9400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 0x7f76e409f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44976/0 (socket says 172.21.15.150:44976) 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 -- 172.21.15.150:0/462548036 learned_addr learned my addr 172.21.15.150:0/462548036 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e9400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 0x7f76e409f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:48456/0 (socket says 172.21.15.150:48456) 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e8a00640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 0x7f76e4155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 -- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 msgr2=0x7f76e409f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 0x7f76e409f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 -- 172.21.15.150:0/462548036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 msgr2=0x7f76e4155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 0x7f76e4155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 -- 172.21.15.150:0/462548036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e4155da0 con 0x7f76e40cef00 2024-09-21T06:27:37.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e8a00640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 0x7f76e4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:37.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.393+0000 7f76e3e00640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f76d000ea20 tx=0x7f76d000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.394+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000cd50 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.394+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76d0004020 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.394+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e41f96d0 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.394+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004450 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.394+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e41575c0 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.395+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e41f9440 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.395+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f76d00035f0 con 0x7f76e40cef00 2024-09-21T06:27:37.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.396+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f76d0052c80 con 0x7f76e40cef00 2024-09-21T06:27:37.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.397+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f76e4155da0 con 0x7f76e40cef00 2024-09-21T06:27:37.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.469+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af143beb-c0f0-4231-92ac-b08f9865c704", "id": 6} v 0) -- 0x7f76e406e200 con 0x7f76e40cef00 2024-09-21T06:27:37.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.478+0000 7f76e0c00640 1 -- 172.21.15.150:0/462548036 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af143beb-c0f0-4231-92ac-b08f9865c704", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f76d00038e0 con 0x7f76e40cef00 2024-09-21T06:27:37.481 INFO:teuthology.orchestra.run.smithi150.stdout:6 2024-09-21T06:27:37.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 msgr2=0x7f76e41f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f76d000ea20 tx=0x7f76d000eee0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 shutdown_connections 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76e4156a50 0x7f76e409f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76e40cef00 0x7f76e41f89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 --2- 172.21.15.150:0/462548036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e406ddf0 0x7f76e4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 >> 172.21.15.150:0/462548036 conn(0x7f76e40b1b90 msgr2=0x7f76e40b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 shutdown_connections 2024-09-21T06:27:37.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.480+0000 7f76eb400640 1 -- 172.21.15.150:0/462548036 wait complete. 2024-09-21T06:27:37.502 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 00fa710d-4aed-4575-b6e8-91b3a4cb057a 7 2024-09-21T06:27:37.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 -- 172.21.15.150:0/3746913278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c156a50 msgr2=0x7fcb9c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 --2- 172.21.15.150:0/3746913278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c156a50 0x7fcb9c074620 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 -- 172.21.15.150:0/3746913278 shutdown_connections 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 --2- 172.21.15.150:0/3746913278 >> v2:172.21.15.150:3300/0 conn(0x7fcb9c06e500 0x7fcb9c0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 --2- 172.21.15.150:0/3746913278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c156a50 0x7fcb9c074620 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 -- 172.21.15.150:0/3746913278 >> 172.21.15.150:0/3746913278 conn(0x7fcb9c0b1b90 msgr2=0x7fcb9c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 -- 172.21.15.150:0/3746913278 shutdown_connections 2024-09-21T06:27:37.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.713+0000 7fcba1200640 1 -- 172.21.15.150:0/3746913278 wait complete. 2024-09-21T06:27:37.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.715+0000 7fcba1200640 1 Processor -- start 2024-09-21T06:27:37.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.716+0000 7fcba1200640 1 -- start start 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c074b60 0x7fcb9c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb9c156a50 0x7fcb9c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cf7d0 con 0x7fcb9c074b60 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cf650 con 0x7fcb9c156a50 2024-09-21T06:27:37.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcba1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cf950 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcb99400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb9c156a50 0x7fcb9c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:48462/0 (socket says 172.21.15.150:48462) 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcb9a800640 1 -- 172.21.15.150:0/2370789488 learned_addr learned my addr 172.21.15.150:0/2370789488 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.717+0000 7fcb99e00640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c074b60 0x7fcb9c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 -- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb9c156a50 msgr2=0x7fcb9c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb9c156a50 0x7fcb9c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 -- 172.21.15.150:0/2370789488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c074b60 msgr2=0x7fcb9c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c074b60 0x7fcb9c15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 -- 172.21.15.150:0/2370789488 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c0c7200 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb9a800640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400ec80 tx=0x7fcb9400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94003b70 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb94003e00 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c0c74e0 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004280 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.718+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c0b34a0 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.719+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c0cef00 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.720+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7fcb94004420 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.720+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fcb94052ff0 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.722+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcb94004710 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.793+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00fa710d-4aed-4575-b6e8-91b3a4cb057a", "id": 7} v 0) -- 0x7fcb9c06e2a0 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.803+0000 7fcb92000640 1 -- 172.21.15.150:0/2370789488 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00fa710d-4aed-4575-b6e8-91b3a4cb057a", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fcb94018bd0 con 0x7fcb9c0c8dd0 2024-09-21T06:27:37.806 INFO:teuthology.orchestra.run.smithi150.stdout:7 2024-09-21T06:27:37.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 msgr2=0x7fcb9c1f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400ec80 tx=0x7fcb9400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 shutdown_connections 2024-09-21T06:27:37.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb9c0c8dd0 0x7fcb9c1f7240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb9c156a50 0x7fcb9c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 --2- 172.21.15.150:0/2370789488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c074b60 0x7fcb9c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 >> 172.21.15.150:0/2370789488 conn(0x7fcb9c0b1b90 msgr2=0x7fcb9c1560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 shutdown_connections 2024-09-21T06:27:37.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:27:37.805+0000 7fcba1200640 1 -- 172.21.15.150:0/2370789488 wait complete. 2024-09-21T06:27:37.831 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T06:27:37.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T06:27:37.837 INFO:tasks.ceph.osd.0:Started 2024-09-21T06:27:37.837 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T06:27:37.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T06:27:37.842 INFO:tasks.ceph.osd.1:Started 2024-09-21T06:27:37.842 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T06:27:37.842 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T06:27:37.846 INFO:tasks.ceph.osd.2:Started 2024-09-21T06:27:37.846 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T06:27:37.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T06:27:37.851 INFO:tasks.ceph.osd.3:Started 2024-09-21T06:27:37.851 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T06:27:37.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T06:27:37.856 INFO:tasks.ceph.osd.4:Started 2024-09-21T06:27:37.856 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T06:27:37.856 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T06:27:37.860 INFO:tasks.ceph.osd.5:Started 2024-09-21T06:27:37.860 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T06:27:37.860 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T06:27:37.864 INFO:tasks.ceph.osd.6:Started 2024-09-21T06:27:37.864 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T06:27:37.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T06:27:37.868 INFO:tasks.ceph.osd.7:Started 2024-09-21T06:27:37.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:27:38.123 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-21T06:27:38.121+0000 7feb4d400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T06:27:38.123 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-21T06:27:38.121+0000 7feb4d400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T06:27:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 -- 172.21.15.37:0/3078862948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08070030 msgr2=0x7f5c080ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 --2- 172.21.15.37:0/3078862948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08070030 0x7f5c080ce2a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c000082a0 tx=0x7f5c0002a970 comp rx=0 tx=0).stop 2024-09-21T06:27:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 -- 172.21.15.37:0/3078862948 shutdown_connections 2024-09-21T06:27:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 --2- 172.21.15.37:0/3078862948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08070030 0x7f5c080ce2a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 --2- 172.21.15.37:0/3078862948 >> v2:172.21.15.150:3300/0 conn(0x7f5c08070e80 0x7f5c0806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 -- 172.21.15.37:0/3078862948 >> 172.21.15.37:0/3078862948 conn(0x7f5c080cca40 msgr2=0x7f5c080cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 -- 172.21.15.37:0/3078862948 shutdown_connections 2024-09-21T06:27:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.175+0000 7f5c10600640 1 -- 172.21.15.37:0/3078862948 wait complete. 2024-09-21T06:27:38.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.177+0000 7f5c10600640 1 Processor -- start 2024-09-21T06:27:38.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.177+0000 7f5c10600640 1 -- start start 2024-09-21T06:27:38.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 0x7f5c0809f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:38.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 0x7f5c0809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08074d90 0x7f5c080a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0815ad50 con 0x7f5c08070e80 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0815abd0 con 0x7f5c08070030 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c10600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0806ddf0 con 0x7f5c08074d90 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c0d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 0x7f5c0809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c0d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 0x7f5c0809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35344/0 (socket says 172.21.15.37:35344) 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c0d200640 1 -- 172.21.15.37:0/3389471174 learned_addr learned my addr 172.21.15.37:0/3389471174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c0dc00640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 0x7f5c0809f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.179+0000 7f5c0e600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08074d90 0x7f5c080a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 -- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08074d90 msgr2=0x7f5c080a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08074d90 0x7f5c080a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 -- 172.21.15.37:0/3389471174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 msgr2=0x7f5c0809a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 0x7f5c0809a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 -- 172.21.15.37:0/3389471174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0815d6b0 con 0x7f5c08070030 2024-09-21T06:27:38.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c0dc00640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 0x7f5c0809f8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0107f0 tx=0x7f5bfc010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc01a020 con 0x7f5c08070030 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bfc00eab0 con 0x7f5c08070030 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.180+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0809e810 con 0x7f5c08070030 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.181+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc016480 con 0x7f5c08070030 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.181+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0809edd0 con 0x7f5c08070030 2024-09-21T06:27:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.182+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0806f710 con 0x7f5c08070030 2024-09-21T06:27:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.182+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 3) ==== 50869+0+0 (secure 0 0 0) 0x7f5bfc01d020 con 0x7f5c08070030 2024-09-21T06:27:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.183+0000 7f5c05600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 0x7f5be8040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.183+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5bfc054f00 con 0x7f5c08070030 2024-09-21T06:27:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.183+0000 7f5c0d200640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 0x7f5be8040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.184+0000 7f5c0d200640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 0x7f5be8040bb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c08074a90 tx=0x7f5bf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.188+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bfc024020 con 0x7f5c08070030 2024-09-21T06:27:38.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.391+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c080b5f40 con 0x7f5c08070030 2024-09-21T06:27:38.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.392+0000 7f5c05600640 1 -- 172.21.15.37:0/3389471174 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5c0815d6b0 con 0x7f5c08070030 2024-09-21T06:27:38.394 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:27:38.394 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:27:37.795248+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:38.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 msgr2=0x7f5be8040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 0x7f5be8040bb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c08074a90 tx=0x7f5bf4009040 comp rx=0 tx=0).stop 2024-09-21T06:27:38.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 msgr2=0x7f5c0809f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 0x7f5c0809f8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0107f0 tx=0x7f5bfc010cb0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 shutdown_connections 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5be803e6a0 0x7f5be8040bb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08070e80 0x7f5c0809a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c08074d90 0x7f5c080a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 --2- 172.21.15.37:0/3389471174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c08070030 0x7f5c0809f8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 >> 172.21.15.37:0/3389471174 conn(0x7f5c080cca40 msgr2=0x7f5c080af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 shutdown_connections 2024-09-21T06:27:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:38.394+0000 7f5c10600640 1 -- 172.21.15.37:0/3389471174 wait complete. 2024-09-21T06:27:38.418 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T06:27:38.418 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T06:27:39.220 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-21T06:27:39.218+0000 7f2a92631640 -1 Falling back to public interface 2024-09-21T06:27:39.224 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 -- 172.21.15.37:0/919844983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554074cb0 msgr2=0x7ff554073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 --2- 172.21.15.37:0/919844983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554074cb0 0x7ff554073dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff544005fd0 tx=0x7ff54402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 -- 172.21.15.37:0/919844983 shutdown_connections 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 --2- 172.21.15.37:0/919844983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff55406e500 0x7ff5540cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 --2- 172.21.15.37:0/919844983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554074cb0 0x7ff554073dd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 -- 172.21.15.37:0/919844983 >> 172.21.15.37:0/919844983 conn(0x7ff5540cca40 msgr2=0x7ff5540cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 -- 172.21.15.37:0/919844983 shutdown_connections 2024-09-21T06:27:39.441 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.438+0000 7ff559000640 1 -- 172.21.15.37:0/919844983 wait complete. 2024-09-21T06:27:39.443 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.441+0000 7ff559000640 1 Processor -- start 2024-09-21T06:27:39.443 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.441+0000 7ff559000640 1 -- start start 2024-09-21T06:27:39.445 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff554074cb0 0x7ff5540b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.445 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5540cbc10 0x7ff55409c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.445 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.445 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff55406f3f0 con 0x7ff55415dcc0 2024-09-21T06:27:39.445 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff55406f270 con 0x7ff5540cbc10 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff559000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff55406f570 con 0x7ff554074cb0 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff552800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff552800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35364/0 (socket says 172.21.15.37:35364) 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.442+0000 7ff552800640 1 -- 172.21.15.37:0/4233062333 learned_addr learned my addr 172.21.15.37:0/4233062333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff551e00640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff554074cb0 0x7ff5540b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff551400640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5540cbc10 0x7ff55409c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 -- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff554074cb0 msgr2=0x7ff5540b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff554074cb0 0x7ff5540b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 -- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5540cbc10 msgr2=0x7ff55409c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5540cbc10 0x7ff55409c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 -- 172.21.15.37:0/4233062333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55415e610 con 0x7ff55415dcc0 2024-09-21T06:27:39.446 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff552800640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00ea70 tx=0x7ff54c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00ce40 con 0x7ff55415dcc0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff54c004c10 con 0x7ff55415dcc0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c005020 con 0x7ff55415dcc0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.443+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5541f4300 con 0x7ff55415dcc0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.444+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541f4b10 con 0x7ff55415dcc0 2024-09-21T06:27:39.447 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.445+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55406fa90 con 0x7ff55415dcc0 2024-09-21T06:27:39.448 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.446+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7ff54c0051c0 con 0x7ff55415dcc0 2024-09-21T06:27:39.449 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.447+0000 7ff54a000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 0x7ff53407b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.449 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.447+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff54c09aad0 con 0x7ff55415dcc0 2024-09-21T06:27:39.449 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.447+0000 7ff551e00640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 0x7ff53407b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.450 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.448+0000 7ff551e00640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 0x7ff53407b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff544033000 tx=0x7ff544004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:39.454 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.452+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff54c09ce20 con 0x7ff55415dcc0 2024-09-21T06:27:39.489 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-21T06:27:39.487+0000 7fb3c7615640 -1 Falling back to public interface 2024-09-21T06:27:39.494 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-21T06:27:39.492+0000 7fdfbb215640 -1 Falling back to public interface 2024-09-21T06:27:39.539 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-21T06:27:39.537+0000 7f0e60411640 -1 Falling back to public interface 2024-09-21T06:27:39.543 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-21T06:27:39.541+0000 7f5515e18640 -1 Falling back to public interface 2024-09-21T06:27:39.660 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.657+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff55406d9d0 con 0x7ff55415dcc0 2024-09-21T06:27:39.660 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.658+0000 7ff54a000640 1 -- 172.21.15.37:0/4233062333 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff55415e610 con 0x7ff55415dcc0 2024-09-21T06:27:39.660 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-21T06:27:39.661 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:27:37.795248+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.660+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 msgr2=0x7ff53407b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.660+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 0x7ff53407b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff544033000 tx=0x7ff544004730 comp rx=0 tx=0).stop 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.660+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 msgr2=0x7ff55415e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.660+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00ea70 tx=0x7ff54c00ef30 comp rx=0 tx=0).stop 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 shutdown_connections 2024-09-21T06:27:39.663 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b00 0x7ff53407b010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55415dcc0 0x7ff55415e0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5540cbc10 0x7ff55409c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 --2- 172.21.15.37:0/4233062333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff554074cb0 0x7ff5540b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 >> 172.21.15.37:0/4233062333 conn(0x7ff5540cca40 msgr2=0x7ff55406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 shutdown_connections 2024-09-21T06:27:39.664 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:39.661+0000 7ff559000640 1 -- 172.21.15.37:0/4233062333 wait complete. 2024-09-21T06:27:39.681 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T06:27:39.701 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-21T06:27:39.699+0000 7f902de18640 -1 Falling back to public interface 2024-09-21T06:27:39.744 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-21T06:27:39.742+0000 7fecdfd2c640 -1 Falling back to public interface 2024-09-21T06:27:39.779 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-21T06:27:39.777+0000 7f9b090fe640 -1 Falling back to public interface 2024-09-21T06:27:42.902 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-21T06:27:42.900+0000 7f2a92631640 -1 osd.5 0 log_to_monitors true 2024-09-21T06:27:43.131 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-21T06:27:43.128+0000 7fb3c7615640 -1 osd.2 0 log_to_monitors true 2024-09-21T06:27:43.449 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-21T06:27:43.447+0000 7fdfbb215640 -1 osd.6 0 log_to_monitors true 2024-09-21T06:27:43.514 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-21T06:27:43.512+0000 7fecdfd2c640 -1 osd.7 0 log_to_monitors true 2024-09-21T06:27:43.608 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-21T06:27:43.606+0000 7f902de18640 -1 osd.4 0 log_to_monitors true 2024-09-21T06:27:43.703 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-21T06:27:43.701+0000 7f9b090fe640 -1 osd.1 0 log_to_monitors true 2024-09-21T06:27:43.758 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-21T06:27:43.756+0000 7f5515e18640 -1 osd.3 0 log_to_monitors true 2024-09-21T06:27:43.763 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-21T06:27:43.761+0000 7f0e60411640 -1 osd.0 0 log_to_monitors true 2024-09-21T06:27:44.953 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-21T06:27:44.951+0000 7f2a8d400640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T06:27:44.956 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-21T06:27:44.955+0000 7f2a83400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:46.488 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T06:27:46.829 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.827+0000 7f66af000640 1 -- 172.21.15.37:0/3199356278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f668c004c70 msgr2=0x7f668c0050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.830 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.827+0000 7f66af000640 1 --2- 172.21.15.37:0/3199356278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f668c004c70 0x7f668c0050c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6698001870 tx=0x7f668c005830 comp rx=0 tx=0).stop 2024-09-21T06:27:46.830 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.827+0000 7f66ae600640 1 -- 172.21.15.37:0/3199356278 reap_dead start 2024-09-21T06:27:46.830 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.827+0000 7f66af000640 1 -- 172.21.15.37:0/3199356278 shutdown_connections 2024-09-21T06:27:46.830 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.827+0000 7f66af000640 1 -- 172.21.15.37:0/3199356278 >> 172.21.15.37:0/3199356278 conn(0x7f66a806f920 msgr2=0x7f66a80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:46.831 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.828+0000 7f66af000640 1 -- 172.21.15.37:0/3199356278 shutdown_connections 2024-09-21T06:27:46.831 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.828+0000 7f66af000640 1 -- 172.21.15.37:0/3199356278 wait complete. 2024-09-21T06:27:46.837 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.835+0000 7f66af000640 1 Processor -- start 2024-09-21T06:27:46.837 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.835+0000 7f66af000640 1 -- start start 2024-09-21T06:27:46.840 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.838+0000 7f66af000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 0x7f66a81f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.840 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.839+0000 7f66af000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 0x7f66a80b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.841 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.839+0000 7f66af000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 0x7f66a81f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.841 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.839+0000 7f66ad200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 0x7f66a80b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.842 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.839+0000 7f66ad200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 0x7f66a80b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:58830/0 (socket says 172.21.15.37:58830) 2024-09-21T06:27:46.842 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66ae600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 0x7f66a81f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.842 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66ae600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 0x7f66a81f1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:56782/0 (socket says 172.21.15.37:56782) 2024-09-21T06:27:46.842 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66ae600640 1 -- 172.21.15.37:0/451078349 learned_addr learned my addr 172.21.15.37:0/451078349 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:46.842 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66a815b840 con 0x7f66a8156590 2024-09-21T06:27:46.843 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66adc00640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 0x7f66a81f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.843 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f66a815b6c0 con 0x7f66a8155d60 2024-09-21T06:27:46.843 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.840+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f66a806ddf0 con 0x7f66a80b18d0 2024-09-21T06:27:46.845 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.842+0000 7f66adc00640 1 -- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 msgr2=0x7f66a81f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.845 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.842+0000 7f66adc00640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 0x7f66a81f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.845 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.842+0000 7f66adc00640 1 -- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 msgr2=0x7f66a80b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.845 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.842+0000 7f66adc00640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 0x7f66a80b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.845 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.842+0000 7f66adc00640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a80b1ce0 con 0x7f66a8156590 2024-09-21T06:27:46.846 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.843+0000 7f66adc00640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 0x7f66a81f46e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6698004000 tx=0x7f66980015a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:46.846 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.844+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698043070 con 0x7f66a8156590 2024-09-21T06:27:46.846 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.844+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66980037e0 con 0x7f66a8156590 2024-09-21T06:27:46.846 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.844+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698003c30 con 0x7f66a8156590 2024-09-21T06:27:46.847 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.844+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81f4de0 con 0x7f66a8156590 2024-09-21T06:27:46.847 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.844+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81f53a0 con 0x7f66a8156590 2024-09-21T06:27:46.849 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.847+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a81f55a0 con 0x7f66a8156590 2024-09-21T06:27:46.852 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.848+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f6698033480 con 0x7f66a8156590 2024-09-21T06:27:46.852 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.850+0000 7f66a5600640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 0x7f668807b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.852 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.850+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f66980b99d0 con 0x7f66a8156590 2024-09-21T06:27:46.855 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.853+0000 7f66ad200640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 0x7f668807b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.861 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.859+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66a80b1ce0 con 0x7f66a8156590 2024-09-21T06:27:46.865 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:46.860+0000 7f66ad200640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 0x7f668807b010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66a80af680 tx=0x7f6694005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:46.870 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-21T06:27:46.867+0000 7fdfb5a00640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T06:27:46.880 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-21T06:27:46.879+0000 7fdfaba00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:46.885 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-21T06:27:46.883+0000 7f9029000640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T06:27:46.893 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-21T06:27:46.891+0000 7f901e600640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:46.920 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-21T06:27:46.918+0000 7fecdac00640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T06:27:46.932 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-21T06:27:46.931+0000 7fecd0c00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:47.293 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.289+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66a806e350 con 0x7f66a8156590 2024-09-21T06:27:47.294 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.291+0000 7f66a5600640 1 -- 172.21.15.37:0/451078349 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7f6698082510 con 0x7f66a8156590 2024-09-21T06:27:47.294 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-21T06:27:47.294 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":14,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:27:46.098937+0000","last_up_change":"2024-09-21T06:27:44.966787+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:47.296 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 msgr2=0x7f668807b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:47.296 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 0x7f668807b010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66a80af680 tx=0x7f6694005f90 comp rx=0 tx=0).stop 2024-09-21T06:27:47.296 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 msgr2=0x7f66a81f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:47.296 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 0x7f66a81f46e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6698004000 tx=0x7f66980015a0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 shutdown_connections 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6688078b00 0x7f668807b010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66a80b18d0 0x7f66a81f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66a8155d60 0x7f66a80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 --2- 172.21.15.37:0/451078349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a8156590 0x7f66a81f46e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.294+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 >> 172.21.15.37:0/451078349 conn(0x7f66a806f920 msgr2=0x7f66a80ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:47.297 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.295+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 shutdown_connections 2024-09-21T06:27:47.298 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:47.295+0000 7f66af000640 1 -- 172.21.15.37:0/451078349 wait complete. 2024-09-21T06:27:47.325 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T06:27:47.351 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-21T06:27:47.348+0000 7fb3c1e00640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T06:27:47.361 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-21T06:27:47.359+0000 7fb3b7e00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:47.512 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-21T06:27:47.510+0000 7f9b04a00640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T06:27:47.518 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-21T06:27:47.516+0000 7f9afa000640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:47.532 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-21T06:27:47.530+0000 7f5511000640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T06:27:47.539 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-21T06:27:47.537+0000 7f5506600640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:47.549 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-21T06:27:47.547+0000 7f0e5ac00640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T06:27:47.558 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-21T06:27:47.556+0000 7f0e50c00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:27:48.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:27:48.875 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:27:54.132 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 -- 172.21.15.37:0/1819994844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c073dd0 msgr2=0x7ff27c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 --2- 172.21.15.37:0/1819994844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c073dd0 0x7ff27c06ddf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff268005fd0 tx=0x7ff26802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 -- 172.21.15.37:0/1819994844 shutdown_connections 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 --2- 172.21.15.37:0/1819994844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c073dd0 0x7ff27c06ddf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 --2- 172.21.15.37:0/1819994844 >> v2:172.21.15.150:3300/0 conn(0x7ff27c06e330 0x7ff27c06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 -- 172.21.15.37:0/1819994844 >> 172.21.15.37:0/1819994844 conn(0x7ff27c0cca40 msgr2=0x7ff27c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 -- 172.21.15.37:0/1819994844 shutdown_connections 2024-09-21T06:27:54.350 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.347+0000 7ff280e00640 1 -- 172.21.15.37:0/1819994844 wait complete. 2024-09-21T06:27:54.351 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.350+0000 7ff280e00640 1 Processor -- start 2024-09-21T06:27:54.352 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.350+0000 7ff280e00640 1 -- start start 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff27c073dd0 0x7ff27c14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c074cb0 0x7ff27c14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c06f4a0 con 0x7ff27c06e330 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c06f320 con 0x7ff27c073dd0 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff280e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff27c06f620 con 0x7ff27c074cb0 2024-09-21T06:27:54.354 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff279e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff279e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36470/0 (socket says 172.21.15.37:36470) 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.351+0000 7ff279e00640 1 -- 172.21.15.37:0/2062201282 learned_addr learned my addr 172.21.15.37:0/2062201282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279400640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff27c073dd0 0x7ff27c14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff27a800640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c074cb0 0x7ff27c14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 -- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c074cb0 msgr2=0x7ff27c14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c074cb0 0x7ff27c14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 -- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff27c073dd0 msgr2=0x7ff27c14f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff27c073dd0 0x7ff27c14f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.355 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 -- 172.21.15.37:0/2062201282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c0ca830 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.352+0000 7ff279e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c370 tx=0x7ff26c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.353+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c017070 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.353+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26c0053c0 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.353+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c0b3f00 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.353+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003850 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.353+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c0cac30 con 0x7ff27c06e330 2024-09-21T06:27:54.356 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.354+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c06fb60 con 0x7ff27c06e330 2024-09-21T06:27:54.357 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.355+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff26c01a070 con 0x7ff27c06e330 2024-09-21T06:27:54.358 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.356+0000 7ff272000640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 0x7ff25c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.358 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.356+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7ff26c099d20 con 0x7ff27c06e330 2024-09-21T06:27:54.359 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.357+0000 7ff279400640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 0x7ff25c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.359 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.357+0000 7ff279400640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 0x7ff25c07b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff268033000 tx=0x7ff268004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:54.363 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.361+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff26c061e60 con 0x7ff27c06e330 2024-09-21T06:27:54.561 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.559+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff27c0cc0d0 con 0x7ff27c06e330 2024-09-21T06:27:54.562 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.560+0000 7ff272000640 1 -- 172.21.15.37:0/2062201282 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12908 (secure 0 0 0) 0x7ff26c066890 con 0x7ff27c06e330 2024-09-21T06:27:54.563 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-21T06:27:54.563 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":19,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:27:51.896070+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 msgr2=0x7ff25c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 0x7ff25c07b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff268033000 tx=0x7ff268004730 comp rx=0 tx=0).stop 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 msgr2=0x7ff27c1f9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c370 tx=0x7ff26c00c830 comp rx=0 tx=0).stop 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 shutdown_connections 2024-09-21T06:27:54.565 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff25c078bc0 0x7ff25c07b0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff27c074cb0 0x7ff27c14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff27c073dd0 0x7ff27c14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 --2- 172.21.15.37:0/2062201282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff27c06e330 0x7ff27c1f9aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 >> 172.21.15.37:0/2062201282 conn(0x7ff27c0cca40 msgr2=0x7ff27c0a0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 shutdown_connections 2024-09-21T06:27:54.566 INFO:teuthology.misc.health.smithi037.stderr:2024-09-21T06:27:54.563+0000 7ff280e00640 1 -- 172.21.15.37:0/2062201282 wait complete. 2024-09-21T06:27:54.584 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T06:27:54.584 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T06:27:54.584 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:27:54.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:27:54.588 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:27:54.588 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:27:54.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:27:54.591 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:27:54.592 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:27:54.592 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:27:54.596 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:27:54.596 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:27:54.597 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:27:54.600 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:27:54.601 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T06:27:54.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:27:54.785 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:27:54.792 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:27:54.800 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:27:54.801 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:27:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 -- 172.21.15.37:0/3239029344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 msgr2=0x7fb2bc0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/3239029344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc0cde00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005fd0 tx=0x7fb2b402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 -- 172.21.15.37:0/3239029344 shutdown_connections 2024-09-21T06:27:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/3239029344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc06e500 0x7fb2bc0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/3239029344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc0cde00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.837+0000 7fb2c3a00640 1 -- 172.21.15.37:0/3239029344 >> 172.21.15.37:0/3239029344 conn(0x7fb2bc0cca40 msgr2=0x7fb2bc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.838+0000 7fb2c3a00640 1 -- 172.21.15.37:0/3239029344 shutdown_connections 2024-09-21T06:27:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.838+0000 7fb2c3a00640 1 -- 172.21.15.37:0/3239029344 wait complete. 2024-09-21T06:27:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.840+0000 7fb2c3a00640 1 Processor -- start 2024-09-21T06:27:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.840+0000 7fb2c3a00640 1 -- start start 2024-09-21T06:27:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2bc1f85d0 0x7fb2bc15f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc161a60 0x7fb2bc1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0ce6d0 con 0x7fb2bc1f85d0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0ce550 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.841+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc0ce850 con 0x7fb2bc161a60 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2bc1f85d0 0x7fb2bc15f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:40412/0 (socket says 172.21.15.37:40412) 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 -- 172.21.15.37:0/107086045 learned_addr learned my addr 172.21.15.37:0/107086045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc161a60 0x7fb2bc1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 -- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc161a60 msgr2=0x7fb2bc1604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc161a60 0x7fb2bc1604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 -- 172.21.15.37:0/107086045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2bc1f85d0 msgr2=0x7fb2bc15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2bc1f85d0 0x7fb2bc15f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 -- 172.21.15.37:0/107086045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1f5930 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.842+0000 7fb2c1000640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c430 tx=0x7fb2b000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:54.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.843+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0017070 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.843+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b00053e0 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.843+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003850 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.843+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc1f5c10 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.843+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc15f320 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.844+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc0ced00 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.845+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2b001a070 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.846+0000 7fb2b8c00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 0x7fb29c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.846+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb2b0099cf0 con 0x7fb2bc15e7c0 2024-09-21T06:27:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.846+0000 7fb2bbe00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 0x7fb29c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:54.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.847+0000 7fb2bbe00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 0x7fb29c07b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80096c0 tx=0x7fb2a8009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:54.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:54.852+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2b009e070 con 0x7fb2bc15e7c0 2024-09-21T06:27:55.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.091+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2bc06f880 con 0x7fb2bc15e7c0 2024-09-21T06:27:55.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.092+0000 7fb2b8c00640 1 -- 172.21.15.37:0/107086045 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fb2b0061e30 con 0x7fb2bc15e7c0 2024-09-21T06:27:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:27:55.095 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"btime":"2024-09-21T06:27:54:793338+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:27:55.095 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-09-21T06:27:55.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.094+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 msgr2=0x7fb29c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:55.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.094+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 0x7fb29c07b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80096c0 tx=0x7fb2a8009260 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.094+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 msgr2=0x7fb2bc1f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.094+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c430 tx=0x7fb2b000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 shutdown_connections 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb29c078bc0 0x7fb29c07b0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2bc1f85d0 0x7fb2bc15f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2bc161a60 0x7fb2bc1604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 --2- 172.21.15.37:0/107086045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2bc15e7c0 0x7fb2bc1f4f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 >> 172.21.15.37:0/107086045 conn(0x7fb2bc0cca40 msgr2=0x7fb2bc06fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 shutdown_connections 2024-09-21T06:27:55.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:55.095+0000 7fb2c3a00640 1 -- 172.21.15.37:0/107086045 wait complete. 2024-09-21T06:27:57.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:27:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 -- 172.21.15.37:0/1360474817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03981629c0 msgr2=0x7f039806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 --2- 172.21.15.37:0/1360474817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03981629c0 0x7f039806d2b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0384005fd0 tx=0x7f038402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 -- 172.21.15.37:0/1360474817 shutdown_connections 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 --2- 172.21.15.37:0/1360474817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03981629c0 0x7f039806d2b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 -- 172.21.15.37:0/1360474817 >> 172.21.15.37:0/1360474817 conn(0x7f03980d1930 msgr2=0x7f03980d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 -- 172.21.15.37:0/1360474817 shutdown_connections 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.355+0000 7f039e400640 1 -- 172.21.15.37:0/1360474817 wait complete. 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.357+0000 7f039e400640 1 Processor -- start 2024-09-21T06:27:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.357+0000 7f039e400640 1 -- start start 2024-09-21T06:27:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03980b9e40 0x7f0398158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03981589a0 0x7f03981f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03981555b0 con 0x7f039806d7f0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f03980d3890 con 0x7f03980b9e40 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039e400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0398155730 con 0x7f03981589a0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36538/0 (socket says 172.21.15.37:36538) 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 -- 172.21.15.37:0/3771801627 learned_addr learned my addr 172.21.15.37:0/3771801627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039da00640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03981589a0 0x7f03981f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f0397e00640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03980b9e40 0x7f0398158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 -- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03981589a0 msgr2=0x7f03981f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03981589a0 0x7f03981f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 -- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03980b9e40 msgr2=0x7f0398158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03980b9e40 0x7f0398158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.359+0000 7f039d000640 1 -- 172.21.15.37:0/3771801627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03981441a0 con 0x7f039806d7f0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f039d000640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f038c00b3f0 tx=0x7f038c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c015020 con 0x7f039806d7f0 2024-09-21T06:27:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f038c011920 con 0x7f039806d7f0 2024-09-21T06:27:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981576a0 con 0x7f039806d7f0 2024-09-21T06:27:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c011d70 con 0x7f039806d7f0 2024-09-21T06:27:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.360+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0398157c60 con 0x7f039806d7f0 2024-09-21T06:27:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.361+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039806d2b0 con 0x7f039806d7f0 2024-09-21T06:27:57.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.364+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f038c018020 con 0x7f039806d7f0 2024-09-21T06:27:57.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.365+0000 7f0394c00640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 0x7f037807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.365+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f038c09acd0 con 0x7f039806d7f0 2024-09-21T06:27:57.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.365+0000 7f0397e00640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 0x7f037807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.366+0000 7f0397e00640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 0x7f037807b0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0384033000 tx=0x7f0384004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.368+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f038c01f020 con 0x7f039806d7f0 2024-09-21T06:27:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.606+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0398154d70 con 0x7f039806d7f0 2024-09-21T06:27:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.607+0000 7f0394c00640 1 -- 172.21.15.37:0/3771801627 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7f038c063550 con 0x7f039806d7f0 2024-09-21T06:27:57.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:27:57.609 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"btime":"2024-09-21T06:27:55:794813+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:27:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 msgr2=0x7f037807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 0x7f037807b0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0384033000 tx=0x7f0384004730 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 msgr2=0x7f03980cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f038c00b3f0 tx=0x7f038c00b8b0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 shutdown_connections 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0378078bc0 0x7f037807b0d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03981589a0 0x7f03981f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03980b9e40 0x7f0398158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 --2- 172.21.15.37:0/3771801627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039806d7f0 0x7f03980cb390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 >> 172.21.15.37:0/3771801627 conn(0x7f03980d1930 msgr2=0x7f0398068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 shutdown_connections 2024-09-21T06:27:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.609+0000 7f039e400640 1 -- 172.21.15.37:0/3771801627 wait complete. 2024-09-21T06:27:57.634 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 -- 172.21.15.37:0/167143832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 msgr2=0x7febb80cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 --2- 172.21.15.37:0/167143832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80cddc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7febac009830 tx=0x7febac02a510 comp rx=0 tx=0).stop 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 -- 172.21.15.37:0/167143832 shutdown_connections 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 --2- 172.21.15.37:0/167143832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80cddc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 -- 172.21.15.37:0/167143832 >> 172.21.15.37:0/167143832 conn(0x7febb806f940 msgr2=0x7febb80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 -- 172.21.15.37:0/167143832 shutdown_connections 2024-09-21T06:27:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.857+0000 7febbde00640 1 -- 172.21.15.37:0/167143832 wait complete. 2024-09-21T06:27:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.860+0000 7febbde00640 1 Processor -- start 2024-09-21T06:27:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.860+0000 7febbde00640 1 -- start start 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febb80b4a20 0x7febb81f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febb80a9440 0x7febb815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febb806de00 con 0x7febb815cb20 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7febb806dc80 con 0x7febb80b4a20 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febbde00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7febb80b4e30 con 0x7febb80a9440 2024-09-21T06:27:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.861+0000 7febb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36546/0 (socket says 172.21.15.37:36546) 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 -- 172.21.15.37:0/447542728 learned_addr learned my addr 172.21.15.37:0/447542728 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febbca00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febb80b4a20 0x7febb81f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febbd400640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febb80a9440 0x7febb815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 -- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febb80a9440 msgr2=0x7febb815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febb80a9440 0x7febb815b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 -- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febb80b4a20 msgr2=0x7febb81f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febb80b4a20 0x7febb81f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 -- 172.21.15.37:0/447542728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8136500 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.862+0000 7febb7e00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feba400b3e0 tx=0x7feba400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.863+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4015020 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.863+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feba4011920 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.863+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4011d70 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.863+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb81f95c0 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.863+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb8138ee0 con 0x7febb815cb20 2024-09-21T06:27:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.864+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb80cddc0 con 0x7febb815cb20 2024-09-21T06:27:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.865+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feba4018020 con 0x7febb815cb20 2024-09-21T06:27:57.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.866+0000 7febb4c00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.867+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7feba409a9b0 con 0x7febb815cb20 2024-09-21T06:27:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.867+0000 7febbca00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.868+0000 7febbca00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:57.871+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feba401f020 con 0x7febb815cb20 2024-09-21T06:27:58.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.070+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7febb81f50d0 con 0x7febb815cb20 2024-09-21T06:27:58.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.070+0000 7febb4c00640 1 -- 172.21.15.37:0/447542728 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7feba40630a0 con 0x7febb815cb20 2024-09-21T06:27:58.072 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:27:58.073 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 msgr2=0x7feb9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac004730 comp rx=0 tx=0).stop 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 msgr2=0x7febb80a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feba400b3e0 tx=0x7feba400b8a0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 shutdown_connections 2024-09-21T06:27:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb815cb20 0x7febb80a9df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febb80a9440 0x7febb815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 --2- 172.21.15.37:0/447542728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febb80b4a20 0x7febb81f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 >> 172.21.15.37:0/447542728 conn(0x7febb806f940 msgr2=0x7febb81459f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 shutdown_connections 2024-09-21T06:27:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.073+0000 7febbde00640 1 -- 172.21.15.37:0/447542728 wait complete. 2024-09-21T06:27:58.098 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:27:58.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 -- 172.21.15.37:0/1360642647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c06e2f0 msgr2=0x7f743c160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 --2- 172.21.15.37:0/1360642647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c06e2f0 0x7f743c160a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f743c075dd0 tx=0x7f743402aac0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 -- 172.21.15.37:0/1360642647 shutdown_connections 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 --2- 172.21.15.37:0/1360642647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c06e2f0 0x7f743c160a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 --2- 172.21.15.37:0/1360642647 >> v2:172.21.15.150:3300/0 conn(0x7f743c0ca690 0x7f743c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.326+0000 7f7443400640 1 -- 172.21.15.37:0/1360642647 >> 172.21.15.37:0/1360642647 conn(0x7f743c0ccac0 msgr2=0x7f743c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.327+0000 7f7443400640 1 -- 172.21.15.37:0/1360642647 shutdown_connections 2024-09-21T06:27:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.327+0000 7f7443400640 1 -- 172.21.15.37:0/1360642647 wait complete. 2024-09-21T06:27:58.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.329+0000 7f7443400640 1 Processor -- start 2024-09-21T06:27:58.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.329+0000 7f7443400640 1 -- start start 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c06e2f0 0x7f743c1f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c160640 0x7f743c0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f743c06f7c0 con 0x7f743c0ca690 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f743c06f640 con 0x7f743c160640 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.330+0000 7f7443400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f743c06f940 con 0x7f743c06e2f0 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36578/0 (socket says 172.21.15.37:36578) 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 -- 172.21.15.37:0/756915454 learned_addr learned my addr 172.21.15.37:0/756915454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f7440a00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c06e2f0 0x7f743c1f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f7441400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c160640 0x7f743c0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 -- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c06e2f0 msgr2=0x7f743c1f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c06e2f0 0x7f743c1f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 -- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c160640 msgr2=0x7f743c0c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c160640 0x7f743c0c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 -- 172.21.15.37:0/756915454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c1f3fd0 con 0x7f743c0ca690 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.331+0000 7f743be00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f742800c3c0 tx=0x7f742800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.332+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428017070 con 0x7f743c0ca690 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.332+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74280035f0 con 0x7f743c0ca690 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.332+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428003a80 con 0x7f743c0ca690 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.332+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c1f42b0 con 0x7f743c0ca690 2024-09-21T06:27:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.332+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1f4590 con 0x7f743c0ca690 2024-09-21T06:27:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.333+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c1f3d40 con 0x7f743c0ca690 2024-09-21T06:27:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.334+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f742801a070 con 0x7f743c0ca690 2024-09-21T06:27:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.335+0000 7f7438c00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 0x7f741c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.335+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7428099ba0 con 0x7f743c0ca690 2024-09-21T06:27:58.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.336+0000 7f7440a00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 0x7f741c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:58.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.336+0000 7f7440a00640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 0x7f741c07b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f743000bb50 tx=0x7f743000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:58.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.340+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7428061d70 con 0x7f743c0ca690 2024-09-21T06:27:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:58.537+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f743c160a50 con 0x7f743c0ca690 2024-09-21T06:27:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.186+0000 7f7438c00640 1 -- 172.21.15.37:0/756915454 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f74280667a0 con 0x7f743c0ca690 2024-09-21T06:27:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:27:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.189+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 msgr2=0x7f741c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 0x7f741c07b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f743000bb50 tx=0x7f743000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 msgr2=0x7f743c15f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f742800c3c0 tx=0x7f742800c880 comp rx=0 tx=0).stop 2024-09-21T06:27:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 shutdown_connections 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f741c078bc0 0x7f741c07b0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f743c0ca690 0x7f743c15f860 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c160640 0x7f743c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 --2- 172.21.15.37:0/756915454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c06e2f0 0x7f743c1f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 >> 172.21.15.37:0/756915454 conn(0x7f743c0ccac0 msgr2=0x7f743c073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 shutdown_connections 2024-09-21T06:27:59.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.190+0000 7f7443400640 1 -- 172.21.15.37:0/756915454 wait complete. 2024-09-21T06:27:59.222 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T06:27:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 -- 172.21.15.37:0/4174937195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e015dfe0 msgr2=0x7f60e0160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 --2- 172.21.15.37:0/4174937195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e015dfe0 0x7f60e0160a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009830 tx=0x7f60d002a480 comp rx=0 tx=0).stop 2024-09-21T06:27:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 -- 172.21.15.37:0/4174937195 shutdown_connections 2024-09-21T06:27:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 --2- 172.21.15.37:0/4174937195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e006e160 0x7f60e006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 --2- 172.21.15.37:0/4174937195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e015dfe0 0x7f60e0160a40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 -- 172.21.15.37:0/4174937195 >> 172.21.15.37:0/4174937195 conn(0x7f60e006f940 msgr2=0x7f60e00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 -- 172.21.15.37:0/4174937195 shutdown_connections 2024-09-21T06:27:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.459+0000 7f60e4c00640 1 -- 172.21.15.37:0/4174937195 wait complete. 2024-09-21T06:27:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.461+0000 7f60e4c00640 1 Processor -- start 2024-09-21T06:27:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.461+0000 7f60e4c00640 1 -- start start 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e00cc8d0 0x7f60e0133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e015dfe0 0x7f60e0131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60e00b4c50 con 0x7f60e006e160 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60e00b4ad0 con 0x7f60e00cc8d0 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60e4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f60e00b4dd0 con 0x7f60e015dfe0 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36588/0 (socket says 172.21.15.37:36588) 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 -- 172.21.15.37:0/2801906527 learned_addr learned my addr 172.21.15.37:0/2801906527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:27:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60dfe00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e015dfe0 0x7f60e0131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60dea00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e00cc8d0 0x7f60e0133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 -- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e015dfe0 msgr2=0x7f60e0131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e015dfe0 0x7f60e0131b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 -- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e00cc8d0 msgr2=0x7f60e0133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.463+0000 7f60df400640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e00cc8d0 0x7f60e0133bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60df400640 1 -- 172.21.15.37:0/2801906527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e0137f50 con 0x7f60e006e160 2024-09-21T06:27:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60df400640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009950 tx=0x7f60d0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0005c90 con 0x7f60e006e160 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d0031e50 con 0x7f60e006e160 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01347b0 con 0x7f60e006e160 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0005260 con 0x7f60e006e160 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.464+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01f6fa0 con 0x7f60e006e160 2024-09-21T06:27:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.465+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e00b52f0 con 0x7f60e006e160 2024-09-21T06:27:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.467+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60d0005400 con 0x7f60e006e160 2024-09-21T06:27:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.467+0000 7f60d7400640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 0x7f60c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.468+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f60d00bbc40 con 0x7f60e006e160 2024-09-21T06:27:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.468+0000 7f60dea00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 0x7f60c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.469+0000 7f60dea00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 0x7f60c007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60e00af5c0 tx=0x7f60cc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.472+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60e0137f50 con 0x7f60e006e160 2024-09-21T06:27:59.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:27:59.670+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f60e00cc390 con 0x7f60e006e160 2024-09-21T06:28:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.189+0000 7f60d7400640 1 -- 172.21.15.37:0/2801906527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7f60d003f030 con 0x7f60e006e160 2024-09-21T06:28:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:28:00.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 msgr2=0x7f60c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:00.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 0x7f60c007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60e00af5c0 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:00.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 msgr2=0x7f60e01f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:00.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009950 tx=0x7f60d0033c20 comp rx=0 tx=0).stop 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 shutdown_connections 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60c0078b90 0x7f60c007b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60e015dfe0 0x7f60e0131b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60e00cc8d0 0x7f60e0133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 --2- 172.21.15.37:0/2801906527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e006e160 0x7f60e01f7a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 >> 172.21.15.37:0/2801906527 conn(0x7f60e006f940 msgr2=0x7f60e015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 shutdown_connections 2024-09-21T06:28:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.192+0000 7f60e4c00640 1 -- 172.21.15.37:0/2801906527 wait complete. 2024-09-21T06:28:00.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:28:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 -- 172.21.15.37:0/3549782125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7406e500 msgr2=0x7f3e740cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 --2- 172.21.15.37:0/3549782125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7406e500 0x7f3e740cef00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e740b2c70 tx=0x7f3e6c02aa10 comp rx=0 tx=0).stop 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 -- 172.21.15.37:0/3549782125 shutdown_connections 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 --2- 172.21.15.37:0/3549782125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7406e500 0x7f3e740cef00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 --2- 172.21.15.37:0/3549782125 >> v2:172.21.15.150:3300/0 conn(0x7f3e74156a50 0x7f3e74074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 -- 172.21.15.37:0/3549782125 >> 172.21.15.37:0/3549782125 conn(0x7f3e740b1b90 msgr2=0x7f3e740b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 -- 172.21.15.37:0/3549782125 shutdown_connections 2024-09-21T06:28:00.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.466+0000 7f3e7c200640 1 -- 172.21.15.37:0/3549782125 wait complete. 2024-09-21T06:28:00.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.468+0000 7f3e7c200640 1 Processor -- start 2024-09-21T06:28:00.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.469+0000 7f3e7c200640 1 -- start start 2024-09-21T06:28:00.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74074b60 0x7f3e741571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e74156a50 0x7f3e741f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740cf7d0 con 0x7f3e740c9f30 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740cf650 con 0x7f3e74074b60 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7c200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740cf950 con 0x7f3e74156a50 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36604/0 (socket says 172.21.15.37:36604) 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e7a200640 1 -- 172.21.15.37:0/4232982702 learned_addr learned my addr 172.21.15.37:0/4232982702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e79800640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74074b60 0x7f3e741571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.470+0000 7f3e78e00640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e74156a50 0x7f3e741f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 -- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e74156a50 msgr2=0x7f3e741f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e74156a50 0x7f3e741f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 -- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74074b60 msgr2=0x7f3e741571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74074b60 0x7f3e741571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 -- 172.21.15.37:0/4232982702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e74157910 con 0x7f3e740c9f30 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e7a200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d60 tx=0x7f3e6c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.471+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c033050 con 0x7f3e740c9f30 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.472+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e6c004450 con 0x7f3e740c9f30 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.472+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e74157bf0 con 0x7f3e740c9f30 2024-09-21T06:28:00.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.472+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004860 con 0x7f3e740c9f30 2024-09-21T06:28:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.472+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741f7310 con 0x7f3e740c9f30 2024-09-21T06:28:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.473+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e740c6a10 con 0x7f3e740c9f30 2024-09-21T06:28:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.474+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e6c036070 con 0x7f3e740c9f30 2024-09-21T06:28:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.475+0000 7f3e71600640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 0x7f3e5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.475+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f3e6c0bc0c0 con 0x7f3e740c9f30 2024-09-21T06:28:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.476+0000 7f3e79800640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 0x7f3e5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.476+0000 7f3e79800640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 0x7f3e5407b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e68004630 tx=0x7f3e6800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.480+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e740c6a10 con 0x7f3e740c9f30 2024-09-21T06:28:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:00.714+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3e740cef00 con 0x7f3e740c9f30 2024-09-21T06:28:01.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.217+0000 7f3e71600640 1 -- 172.21.15.37:0/4232982702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f3e6c083e70 con 0x7f3e740c9f30 2024-09-21T06:28:01.220 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T06:28:01.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.222+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 msgr2=0x7f3e5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.222+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 0x7f3e5407b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e68004630 tx=0x7f3e6800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 msgr2=0x7f3e740c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d60 tx=0x7f3e6c003f10 comp rx=0 tx=0).stop 2024-09-21T06:28:01.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 shutdown_connections 2024-09-21T06:28:01.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e54078bc0 0x7f3e5407b0d0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e68004630 tx=0x7f3e6800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740c9f30 0x7f3e740c5a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e74156a50 0x7f3e741f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 --2- 172.21.15.37:0/4232982702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74074b60 0x7f3e741571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.223+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 >> 172.21.15.37:0/4232982702 conn(0x7f3e740b1b90 msgr2=0x7f3e740b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.224+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 shutdown_connections 2024-09-21T06:28:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.224+0000 7f3e7c200640 1 -- 172.21.15.37:0/4232982702 wait complete. 2024-09-21T06:28:01.273 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T06:28:01.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 -- 172.21.15.37:0/3188364799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f44070940 msgr2=0x7f4f440ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 --2- 172.21.15.37:0/3188364799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f44070940 0x7f4f440ce8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f440cd700 tx=0x7f4f2c02a560 comp rx=0 tx=0).stop 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 -- 172.21.15.37:0/3188364799 shutdown_connections 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 --2- 172.21.15.37:0/3188364799 >> v2:172.21.15.150:3300/0 conn(0x7f4f4406def0 0x7f4f4406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 --2- 172.21.15.37:0/3188364799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f44070940 0x7f4f440ce8a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 --2- 172.21.15.37:0/3188364799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 0x7f4f440cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.503+0000 7f4f49e00640 1 -- 172.21.15.37:0/3188364799 >> 172.21.15.37:0/3188364799 conn(0x7f4f440ccac0 msgr2=0x7f4f440cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:01.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.504+0000 7f4f49e00640 1 -- 172.21.15.37:0/3188364799 shutdown_connections 2024-09-21T06:28:01.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.504+0000 7f4f49e00640 1 -- 172.21.15.37:0/3188364799 wait complete. 2024-09-21T06:28:01.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.506+0000 7f4f49e00640 1 Processor -- start 2024-09-21T06:28:01.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.506+0000 7f4f49e00640 1 -- start start 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f44070940 0x7f4f4414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 0x7f4f440afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4406f400 con 0x7f4f4406def0 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4406f280 con 0x7f4f44070940 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.507+0000 7f4f49e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4406f580 con 0x7f4f440cd790 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36632/0 (socket says 172.21.15.37:36632) 2024-09-21T06:28:01.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 -- 172.21.15.37:0/2635251247 learned_addr learned my addr 172.21.15.37:0/2635251247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f3be00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f44070940 0x7f4f4414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f43400640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 0x7f4f440afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 -- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 msgr2=0x7f4f440afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 0x7f4f440afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 -- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f44070940 msgr2=0x7f4f4414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f44070940 0x7f4f4414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 -- 172.21.15.37:0/2635251247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f44158310 con 0x7f4f4406def0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.508+0000 7f4f42a00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b410 tx=0x7f4f3400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.509+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34015020 con 0x7f4f4406def0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.509+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f34011940 con 0x7f4f4406def0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.509+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011d90 con 0x7f4f4406def0 2024-09-21T06:28:01.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.509+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f441fa2b0 con 0x7f4f4406def0 2024-09-21T06:28:01.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.509+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f44158890 con 0x7f4f4406def0 2024-09-21T06:28:01.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.510+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4406fb90 con 0x7f4f4406def0 2024-09-21T06:28:01.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.511+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f34018020 con 0x7f4f4406def0 2024-09-21T06:28:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.512+0000 7f4f3b400640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 0x7f4f2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.512+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4f3409abe0 con 0x7f4f4406def0 2024-09-21T06:28:01.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.513+0000 7f4f3be00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 0x7f4f2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.513+0000 7f4f3be00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 0x7f4f2007b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c033000 tx=0x7f4f2c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:01.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.517+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f3401f020 con 0x7f4f4406def0 2024-09-21T06:28:01.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:01.721+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f4f4406ffd0 con 0x7f4f4406def0 2024-09-21T06:28:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.205+0000 7f4f3b400640 1 -- 172.21.15.37:0/2635251247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7f4f340629c0 con 0x7f4f4406def0 2024-09-21T06:28:02.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.207+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 msgr2=0x7f4f2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:02.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.207+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 0x7f4f2007b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c033000 tx=0x7f4f2c004730 comp rx=0 tx=0).stop 2024-09-21T06:28:02.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.207+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 msgr2=0x7f4f44158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:02.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.207+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b410 tx=0x7f4f3400b8d0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 shutdown_connections 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f20078bc0 0x7f4f2007b0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f440cd790 0x7f4f440afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f44070940 0x7f4f4414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 --2- 172.21.15.37:0/2635251247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4406def0 0x7f4f44158db0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 >> 172.21.15.37:0/2635251247 conn(0x7f4f440ccac0 msgr2=0x7f4f4415ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 shutdown_connections 2024-09-21T06:28:02.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.208+0000 7f4f49e00640 1 -- 172.21.15.37:0/2635251247 wait complete. 2024-09-21T06:28:02.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 -- 172.21.15.37:0/2323442176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c162260 msgr2=0x7f070c162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 --2- 172.21.15.37:0/2323442176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c162260 0x7f070c162670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06fc005fd0 tx=0x7f06fc02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 -- 172.21.15.37:0/2323442176 shutdown_connections 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 --2- 172.21.15.37:0/2323442176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c0cf070 0x7f070c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 --2- 172.21.15.37:0/2323442176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c162260 0x7f070c162670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 -- 172.21.15.37:0/2323442176 >> 172.21.15.37:0/2323442176 conn(0x7f070c0d43a0 msgr2=0x7f070c0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.471+0000 7f0711600640 1 -- 172.21.15.37:0/2323442176 shutdown_connections 2024-09-21T06:28:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.472+0000 7f0711600640 1 -- 172.21.15.37:0/2323442176 wait complete. 2024-09-21T06:28:02.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.474+0000 7f0711600640 1 Processor -- start 2024-09-21T06:28:02.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.474+0000 7f0711600640 1 -- start start 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f070c0cf070 0x7f070c159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c162260 0x7f070c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f070c0cf9c0 con 0x7f070c154f60 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1630c0 con 0x7f070c0cf070 2024-09-21T06:28:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.475+0000 7f0711600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f070c0cfb40 con 0x7f070c162260 2024-09-21T06:28:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f070c0cf070 0x7f070c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36658/0 (socket says 172.21.15.37:36658) 2024-09-21T06:28:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 -- 172.21.15.37:0/3096576666 learned_addr learned my addr 172.21.15.37:0/3096576666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f0710c00640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c162260 0x7f070c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 -- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c162260 msgr2=0x7f070c153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c162260 0x7f070c153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 -- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f070c0cf070 msgr2=0x7f070c159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f070c0cf070 0x7f070c159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 -- 172.21.15.37:0/3096576666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c0cb130 con 0x7f070c154f60 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.476+0000 7f070b400640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c3a0 tx=0x7f06f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.477+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8017070 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.477+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06f80053c0 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.477+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c0ca890 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.477+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8003850 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.477+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c0cae50 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.478+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0cf480 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.479+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06f801a070 con 0x7f070c154f60 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.480+0000 7f0703e00640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 0x7f06ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.480+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7f06f809a1b0 con 0x7f070c154f60 2024-09-21T06:28:02.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.481+0000 7f070be00640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 0x7f06ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:02.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.481+0000 7f070be00640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 0x7f06ec07b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06fc033000 tx=0x7f06fc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.484+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f070c162bb0 con 0x7f070c154f60 2024-09-21T06:28:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:02.682+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f070c0ca610 con 0x7f070c154f60 2024-09-21T06:28:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.234+0000 7f0703e00640 1 -- 172.21.15.37:0/3096576666 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7f070c0ca610 con 0x7f070c154f60 2024-09-21T06:28:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' created 2024-09-21T06:28:04.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.237+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 msgr2=0x7f06ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.237+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 0x7f06ec07b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06fc033000 tx=0x7f06fc004730 comp rx=0 tx=0).stop 2024-09-21T06:28:04.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.237+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 msgr2=0x7f070c158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.237+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c3a0 tx=0x7f06f800c860 comp rx=0 tx=0).stop 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 shutdown_connections 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06ec078bc0 0x7f06ec07b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f070c154f60 0x7f070c158a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f070c162260 0x7f070c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 --2- 172.21.15.37:0/3096576666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f070c0cf070 0x7f070c159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 >> 172.21.15.37:0/3096576666 conn(0x7f070c0d43a0 msgr2=0x7f070c155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 shutdown_connections 2024-09-21T06:28:04.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.238+0000 7f0711600640 1 -- 172.21.15.37:0/3096576666 wait complete. 2024-09-21T06:28:04.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.507+0000 7f3f55c00640 1 -- 172.21.15.37:0/2394895114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500732a0 msgr2=0x7f3f500b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.507+0000 7f3f55c00640 1 --2- 172.21.15.37:0/2394895114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500732a0 0x7f3f500b3ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009830 tx=0x7f3f3802a510 comp rx=0 tx=0).stop 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.507+0000 7f3f55c00640 1 -- 172.21.15.37:0/2394895114 shutdown_connections 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.508+0000 7f3f55c00640 1 --2- 172.21.15.37:0/2394895114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500732a0 0x7f3f500b3ee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.508+0000 7f3f55c00640 1 --2- 172.21.15.37:0/2394895114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f5015f720 0x7f3f50155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.508+0000 7f3f55c00640 1 -- 172.21.15.37:0/2394895114 >> 172.21.15.37:0/2394895114 conn(0x7f3f500d0460 msgr2=0x7f3f5006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.508+0000 7f3f55c00640 1 -- 172.21.15.37:0/2394895114 shutdown_connections 2024-09-21T06:28:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.508+0000 7f3f55c00640 1 -- 172.21.15.37:0/2394895114 wait complete. 2024-09-21T06:28:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.510+0000 7f3f55c00640 1 Processor -- start 2024-09-21T06:28:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.510+0000 7f3f55c00640 1 -- start start 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f500732a0 0x7f3f50151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f5015f720 0x7f3f50150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500ae180 con 0x7f3f500b49b0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500ae000 con 0x7f3f5015f720 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500ae300 con 0x7f3f500732a0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54142/0 (socket says 172.21.15.37:54142) 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 -- 172.21.15.37:0/3610408811 learned_addr learned my addr 172.21.15.37:0/3610408811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f4fe00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f500732a0 0x7f3f50151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f55200640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f5015f720 0x7f3f50150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 -- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f500732a0 msgr2=0x7f3f50151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f500732a0 0x7f3f50151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 -- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f5015f720 msgr2=0x7f3f50150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f5015f720 0x7f3f50150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.512+0000 7f3f47400640 1 -- 172.21.15.37:0/3610408811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500b0470 con 0x7f3f500b49b0 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f47400640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009950 tx=0x7f3f38033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38005c90 con 0x7f3f500b49b0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f38031e50 con 0x7f3f500b49b0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38005260 con 0x7f3f500b49b0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f500b08e0 con 0x7f3f500b49b0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.513+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f500af710 con 0x7f3f500b49b0 2024-09-21T06:28:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.514+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50074070 con 0x7f3f500b49b0 2024-09-21T06:28:04.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.516+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f38005400 con 0x7f3f500b49b0 2024-09-21T06:28:04.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.517+0000 7f3f4cc00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 0x7f3f2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.517+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f3f380bc020 con 0x7f3f500b49b0 2024-09-21T06:28:04.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.517+0000 7f3f4fe00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 0x7f3f2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.518+0000 7f3f4fe00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 0x7f3f2c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009b30 tx=0x7f3f40005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.522+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f50074070 con 0x7f3f500b49b0 2024-09-21T06:28:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:04.717+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f3f500b0640 con 0x7f3f500b49b0 2024-09-21T06:28:05.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.236+0000 7f3f4cc00640 1 -- 172.21.15.37:0/3610408811 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7f3f3803f030 con 0x7f3f500b49b0 2024-09-21T06:28:05.238 INFO:teuthology.orchestra.run.smithi037.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T06:28:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.242+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 msgr2=0x7f3f2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.242+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 0x7f3f2c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009b30 tx=0x7f3f40005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.242+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 msgr2=0x7f3f50150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.242+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009950 tx=0x7f3f38033c20 comp rx=0 tx=0).stop 2024-09-21T06:28:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 shutdown_connections 2024-09-21T06:28:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3f2c078b90 0x7f3f2c07b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f500b49b0 0x7f3f50150430 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f5015f720 0x7f3f50150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 --2- 172.21.15.37:0/3610408811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f500732a0 0x7f3f50151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 >> 172.21.15.37:0/3610408811 conn(0x7f3f500d0460 msgr2=0x7f3f5006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.243+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 shutdown_connections 2024-09-21T06:28:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.244+0000 7f3f55c00640 1 -- 172.21.15.37:0/3610408811 wait complete. 2024-09-21T06:28:05.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T06:28:05.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.522+0000 7f4838a00640 1 -- 172.21.15.37:0/824506438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483406e500 msgr2=0x7f48340cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.522+0000 7f4838a00640 1 --2- 172.21.15.37:0/824506438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483406e500 0x7f48340cf290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48340cb1d0 tx=0x7f482002a560 comp rx=0 tx=0).stop 2024-09-21T06:28:05.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 -- 172.21.15.37:0/824506438 shutdown_connections 2024-09-21T06:28:05.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 --2- 172.21.15.37:0/824506438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483406e500 0x7f48340cf290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 --2- 172.21.15.37:0/824506438 >> v2:172.21.15.150:3300/0 conn(0x7f48340cf7d0 0x7f4834072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 -- 172.21.15.37:0/824506438 >> 172.21.15.37:0/824506438 conn(0x7f48340b1b90 msgr2=0x7f48340b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 -- 172.21.15.37:0/824506438 shutdown_connections 2024-09-21T06:28:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.523+0000 7f4838a00640 1 -- 172.21.15.37:0/824506438 wait complete. 2024-09-21T06:28:05.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.525+0000 7f4838a00640 1 Processor -- start 2024-09-21T06:28:05.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.525+0000 7f4838a00640 1 -- start start 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48340cf7d0 0x7f48340a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f483415fc50 0x7f4834159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4834070d20 con 0x7f48340a0830 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4834070ba0 con 0x7f48340cf7d0 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.526+0000 7f4838a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f48340732d0 con 0x7f483415fc50 2024-09-21T06:28:05.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54156/0 (socket says 172.21.15.37:54156) 2024-09-21T06:28:05.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 -- 172.21.15.37:0/2487633632 learned_addr learned my addr 172.21.15.37:0/2487633632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:05.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4831e00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48340cf7d0 0x7f48340a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4831400640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f483415fc50 0x7f4834159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 -- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f483415fc50 msgr2=0x7f4834159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f483415fc50 0x7f4834159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 -- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48340cf7d0 msgr2=0x7f48340a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48340cf7d0 0x7f48340a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.527+0000 7f4832800640 1 -- 172.21.15.37:0/2487633632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48340a05c0 con 0x7f48340a0830 2024-09-21T06:28:05.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f4832800640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f482c00cb00 tx=0x7f482c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c00f450 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f482c00f700 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c00fb50 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48340a1020 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.528+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483409f2c0 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.529+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48340737b0 con 0x7f48340a0830 2024-09-21T06:28:05.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.530+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f482c00fcf0 con 0x7f48340a0830 2024-09-21T06:28:05.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.531+0000 7f482a000640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 0x7f481407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.531+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7f482c09bbb0 con 0x7f48340a0830 2024-09-21T06:28:05.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.532+0000 7f4831e00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 0x7f481407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.533+0000 7f4831e00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 0x7f481407b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:05.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.536+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f482c014030 con 0x7f48340a0830 2024-09-21T06:28:05.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:05.775+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4834070470 con 0x7f48340a0830 2024-09-21T06:28:05.808 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:28:05.806+0000 7f8fc3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:28:06.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.277+0000 7f482a000640 1 -- 172.21.15.37:0/2487633632 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f482c0635e0 con 0x7f48340a0830 2024-09-21T06:28:06.279 INFO:teuthology.orchestra.run.smithi037.stderr:added data pool 4 to fsmap 2024-09-21T06:28:06.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 msgr2=0x7f481407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 0x7f481407b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824009060 comp rx=0 tx=0).stop 2024-09-21T06:28:06.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 msgr2=0x7f48340c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f482c00cb00 tx=0x7f482c00a250 comp rx=0 tx=0).stop 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 shutdown_connections 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4814078bc0 0x7f481407b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48340a0830 0x7f48340c3bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f483415fc50 0x7f4834159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 --2- 172.21.15.37:0/2487633632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48340cf7d0 0x7f48340a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 >> 172.21.15.37:0/2487633632 conn(0x7f48340b1b90 msgr2=0x7f483415aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 shutdown_connections 2024-09-21T06:28:06.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.280+0000 7f4838a00640 1 -- 172.21.15.37:0/2487633632 wait complete. 2024-09-21T06:28:06.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 -- 172.21.15.37:0/17193345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15b9f0 msgr2=0x7f21cc0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 --2- 172.21.15.37:0/17193345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15b9f0 0x7f21cc0cf930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009860 tx=0x7f21c002a530 comp rx=0 tx=0).stop 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 -- 172.21.15.37:0/17193345 shutdown_connections 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 --2- 172.21.15.37:0/17193345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15b9f0 0x7f21cc0cf930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 -- 172.21.15.37:0/17193345 >> 172.21.15.37:0/17193345 conn(0x7f21cc0cca40 msgr2=0x7f21cc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 -- 172.21.15.37:0/17193345 shutdown_connections 2024-09-21T06:28:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.532+0000 7f21d3a00640 1 -- 172.21.15.37:0/17193345 wait complete. 2024-09-21T06:28:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.534+0000 7f21d3a00640 1 Processor -- start 2024-09-21T06:28:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.535+0000 7f21d3a00640 1 -- start start 2024-09-21T06:28:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21cc15b9f0 0x7f21cc0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15f5a0 0x7f21cc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc15f9b0 con 0x7f21cc0cefa0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc06e710 con 0x7f21cc15b9f0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d3a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc15fb30 con 0x7f21cc15f5a0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54162/0 (socket says 172.21.15.37:54162) 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d1000640 1 -- 172.21.15.37:0/2647267616 learned_addr learned my addr 172.21.15.37:0/2647267616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.536+0000 7f21d1a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15f5a0 0x7f21cc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21cbe00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21cc15b9f0 0x7f21cc0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 -- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15f5a0 msgr2=0x7f21cc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15f5a0 0x7f21cc0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 -- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21cc15b9f0 msgr2=0x7f21cc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21cc15b9f0 0x7f21cc0b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 -- 172.21.15.37:0/2647267616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc1f74b0 con 0x7f21cc0cefa0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d1000640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009980 tx=0x7f21c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0005c90 con 0x7f21cc0cefa0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21c0005020 con 0x7f21cc0cefa0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc0c7070 con 0x7f21cc0cefa0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0005410 con 0x7f21cc0cefa0 2024-09-21T06:28:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.537+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1f7730 con 0x7f21cc0cefa0 2024-09-21T06:28:06.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.538+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc0cf930 con 0x7f21cc0cefa0 2024-09-21T06:28:06.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.540+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21c00317c0 con 0x7f21cc0cefa0 2024-09-21T06:28:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.541+0000 7f21c8c00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 0x7f21ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.541+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f21c00bc260 con 0x7f21cc0cefa0 2024-09-21T06:28:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.541+0000 7f21cbe00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 0x7f21ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.542+0000 7f21cbe00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 0x7f21ac07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001240 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.544+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21cc1f74b0 con 0x7f21cc0cefa0 2024-09-21T06:28:06.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.781+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21cc0cd0a0 con 0x7f21cc0cefa0 2024-09-21T06:28:06.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.781+0000 7f21c8c00640 1 -- 172.21.15.37:0/2647267616 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5225 (secure 0 0 0) 0x7f21c003f030 con 0x7f21cc0cefa0 2024-09-21T06:28:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:06.784 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"btime":"2024-09-21T06:28:06:266663+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:05.777993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:06.784 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-09-21T06:28:06.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 msgr2=0x7f21ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 0x7f21ac07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001240 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 msgr2=0x7f21cc1f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009980 tx=0x7f21c0003c10 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 shutdown_connections 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21ac078bc0 0x7f21ac07b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21cc15f5a0 0x7f21cc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21cc15b9f0 0x7f21cc0b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 --2- 172.21.15.37:0/2647267616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21cc0cefa0 0x7f21cc1f2f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 >> 172.21.15.37:0/2647267616 conn(0x7f21cc0cca40 msgr2=0x7f21cc09cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 shutdown_connections 2024-09-21T06:28:06.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:06.784+0000 7f21d3a00640 1 -- 172.21.15.37:0/2647267616 wait complete. 2024-09-21T06:28:06.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 -- 172.21.15.37:0/183393185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f792806e500 msgr2=0x7f79280cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 --2- 172.21.15.37:0/183393185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f792806e500 0x7f79280cc020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7920005fd0 tx=0x7f792002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 -- 172.21.15.37:0/183393185 shutdown_connections 2024-09-21T06:28:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 --2- 172.21.15.37:0/183393185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f792806e500 0x7f79280cc020 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 --2- 172.21.15.37:0/183393185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7928074cb0 0x7f7928073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 -- 172.21.15.37:0/183393185 >> 172.21.15.37:0/183393185 conn(0x7f79280cca40 msgr2=0x7f79280cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 -- 172.21.15.37:0/183393185 shutdown_connections 2024-09-21T06:28:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.031+0000 7f792ea00640 1 -- 172.21.15.37:0/183393185 wait complete. 2024-09-21T06:28:07.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.034+0000 7f792ea00640 1 Processor -- start 2024-09-21T06:28:07.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.034+0000 7f792ea00640 1 -- start start 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7928074cb0 0x7f79280b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f792809b870 0x7f7928156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f792806f360 con 0x7f79280cbc10 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f792806f1e0 con 0x7f7928074cb0 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f792ea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f792806f4e0 con 0x7f792809b870 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.035+0000 7f791fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54168/0 (socket says 172.21.15.37:54168) 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 -- 172.21.15.37:0/132765665 learned_addr learned my addr 172.21.15.37:0/132765665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f7927e00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7928074cb0 0x7f79280b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f792ca00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f792809b870 0x7f7928156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 -- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f792809b870 msgr2=0x7f7928156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f792809b870 0x7f7928156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 -- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7928074cb0 msgr2=0x7f79280b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7928074cb0 0x7f79280b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 -- 172.21.15.37:0/132765665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792815bf80 con 0x7f79280cbc10 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.036+0000 7f791fe00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f791000c3a0 tx=0x7f791000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.037+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910011790 con 0x7f79280cbc10 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.037+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7910017070 con 0x7f79280cbc10 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.037+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79281f3d50 con 0x7f79280cbc10 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.037+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910011cc0 con 0x7f79280cbc10 2024-09-21T06:28:07.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.037+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7928154f60 con 0x7f79280cbc10 2024-09-21T06:28:07.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.038+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792806fa00 con 0x7f79280cbc10 2024-09-21T06:28:07.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.039+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f791001c460 con 0x7f79280cbc10 2024-09-21T06:28:07.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.040+0000 7f7924c00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.040+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f791009adf0 con 0x7f79280cbc10 2024-09-21T06:28:07.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.041+0000 7f7927e00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.041+0000 7f7927e00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 0x7f790407b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f79180095f0 tx=0x7f7918005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.045+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7910062780 con 0x7f79280cbc10 2024-09-21T06:28:07.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.241+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f792809b0c0 con 0x7f79280cbc10 2024-09-21T06:28:07.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.243+0000 7f7924c00640 1 -- 172.21.15.37:0/132765665 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f79100671b0 con 0x7f79280cbc10 2024-09-21T06:28:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:07.245 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:07.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 msgr2=0x7f790407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 0x7f790407b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f79180095f0 tx=0x7f7918005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 msgr2=0x7f79281569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f791000c3a0 tx=0x7f791000c860 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 shutdown_connections 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280cbc10 0x7f79281569a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f792809b870 0x7f7928156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 --2- 172.21.15.37:0/132765665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7928074cb0 0x7f79280b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 >> 172.21.15.37:0/132765665 conn(0x7f79280cca40 msgr2=0x7f79281e4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 shutdown_connections 2024-09-21T06:28:07.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.245+0000 7f792ea00640 1 -- 172.21.15.37:0/132765665 wait complete. 2024-09-21T06:28:07.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:07.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 -- 172.21.15.37:0/2684310235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24c0004800 msgr2=0x7f24c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 --2- 172.21.15.37:0/2684310235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24c0004800 0x7f24c0004c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006f80 tx=0x7f24c0005830 comp rx=0 tx=0).stop 2024-09-21T06:28:07.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24de800640 1 -- 172.21.15.37:0/2684310235 reap_dead start 2024-09-21T06:28:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 -- 172.21.15.37:0/2684310235 shutdown_connections 2024-09-21T06:28:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 -- 172.21.15.37:0/2684310235 >> 172.21.15.37:0/2684310235 conn(0x7f24d806f940 msgr2=0x7f24d80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 -- 172.21.15.37:0/2684310235 shutdown_connections 2024-09-21T06:28:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.497+0000 7f24df200640 1 -- 172.21.15.37:0/2684310235 wait complete. 2024-09-21T06:28:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.499+0000 7f24df200640 1 Processor -- start 2024-09-21T06:28:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.499+0000 7f24df200640 1 -- start start 2024-09-21T06:28:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c0004800 0x7f24d8142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 0x7f24d81f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8070e70 con 0x7f24d8155260 2024-09-21T06:28:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8070cf0 con 0x7f24d80b1290 2024-09-21T06:28:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24df200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f24d815e8a0 con 0x7f24c0004800 2024-09-21T06:28:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24dd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24dd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54186/0 (socket says 172.21.15.37:54186) 2024-09-21T06:28:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24dd400640 1 -- 172.21.15.37:0/3697503828 learned_addr learned my addr 172.21.15.37:0/3697503828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24de800640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 0x7f24d81f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.501+0000 7f24dde00640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c0004800 0x7f24d8142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 -- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c0004800 msgr2=0x7f24d8142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c0004800 0x7f24d8142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 -- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 msgr2=0x7f24d81f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 0x7f24d81f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 -- 172.21.15.37:0/3697503828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81f8740 con 0x7f24d8155260 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24de800640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 0x7f24d81f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24dd400640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24c802b0a0 tx=0x7f24c8006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8003a00 con 0x7f24d8155260 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c8003c90 con 0x7f24d8155260 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d8155dc0 con 0x7f24d8155260 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c802e6e0 con 0x7f24d8155260 2024-09-21T06:28:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.502+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d8142bd0 con 0x7f24d8155260 2024-09-21T06:28:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.504+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d80cc820 con 0x7f24d8155260 2024-09-21T06:28:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.505+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24c8035020 con 0x7f24d8155260 2024-09-21T06:28:07.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.506+0000 7f24d2000640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 0x7f24bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.506+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f24c80bb540 con 0x7f24d8155260 2024-09-21T06:28:07.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.506+0000 7f24dde00640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 0x7f24bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.507+0000 7f24dde00640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 0x7f24bc07b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ab40 tx=0x7f24cc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.510+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24c8047030 con 0x7f24d8155260 2024-09-21T06:28:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.708+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24d80cdaa0 con 0x7f24d8155260 2024-09-21T06:28:07.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.710+0000 7f24d2000640 1 -- 172.21.15.37:0/3697503828 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f24c803e070 con 0x7f24d8155260 2024-09-21T06:28:07.712 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:07.712 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:07.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 msgr2=0x7f24bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 0x7f24bc07b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ab40 tx=0x7f24cc005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 msgr2=0x7f24d813e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24c802b0a0 tx=0x7f24c8006cc0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 shutdown_connections 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f24bc078b90 0x7f24bc07b0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24d8155260 0x7f24d813e890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d80b1290 0x7f24d81f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 --2- 172.21.15.37:0/3697503828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c0004800 0x7f24d8142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 >> 172.21.15.37:0/3697503828 conn(0x7f24d806f940 msgr2=0x7f24d80cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 shutdown_connections 2024-09-21T06:28:07.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.712+0000 7f24df200640 1 -- 172.21.15.37:0/3697503828 wait complete. 2024-09-21T06:28:07.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.974+0000 7ff26a200640 1 -- 172.21.15.37:0/3219333918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d0830 msgr2=0x7ff264152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.974+0000 7ff26a200640 1 --2- 172.21.15.37:0/3219333918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d0830 0x7ff264152c00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005fd0 tx=0x7ff25c02a940 comp rx=0 tx=0).stop 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.975+0000 7ff26a200640 1 -- 172.21.15.37:0/3219333918 shutdown_connections 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.975+0000 7ff26a200640 1 --2- 172.21.15.37:0/3219333918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d0830 0x7ff264152c00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.975+0000 7ff26a200640 1 -- 172.21.15.37:0/3219333918 >> 172.21.15.37:0/3219333918 conn(0x7ff2640b3b30 msgr2=0x7ff2640b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.975+0000 7ff26a200640 1 -- 172.21.15.37:0/3219333918 shutdown_connections 2024-09-21T06:28:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.975+0000 7ff26a200640 1 -- 172.21.15.37:0/3219333918 wait complete. 2024-09-21T06:28:07.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.977+0000 7ff26a200640 1 Processor -- start 2024-09-21T06:28:07.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.977+0000 7ff26a200640 1 -- start start 2024-09-21T06:28:07.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.978+0000 7ff26a200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 0x7ff26406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff26a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff26a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 0x7ff2641e2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff26a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641532c0 con 0x7ff2640d1840 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff26a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff264153140 con 0x7ff2640caf30 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff26a200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff264153440 con 0x7ff264071cf0 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54190/0 (socket says 172.21.15.37:54190) 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 -- 172.21.15.37:0/63799275 learned_addr learned my addr 172.21.15.37:0/63799275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff268e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 0x7ff26406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff269800640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 0x7ff2641e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 -- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 msgr2=0x7ff26406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 0x7ff26406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 -- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 msgr2=0x7ff2641e2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 0x7ff2641e2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff263e00640 1 -- 172.21.15.37:0/63799275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26406e000 con 0x7ff2640d1840 2024-09-21T06:28:07.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff268e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 0x7ff26406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:07.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.979+0000 7ff269800640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 0x7ff2641e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff263e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c400 tx=0x7ff25000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250017070 con 0x7ff2640d1840 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2500053c0 con 0x7ff2640d1840 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26406e2e0 con 0x7ff2640d1840 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250003850 con 0x7ff2640d1840 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.980+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26406e8a0 con 0x7ff2640d1840 2024-09-21T06:28:07.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.981+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264153860 con 0x7ff2640d1840 2024-09-21T06:28:07.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.982+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff25001a070 con 0x7ff2640d1840 2024-09-21T06:28:07.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.983+0000 7ff260c00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 0x7ff24407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.983+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff25009a490 con 0x7ff2640d1840 2024-09-21T06:28:07.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.984+0000 7ff268e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 0x7ff24407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.984+0000 7ff268e00640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 0x7ff24407b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff258000f60 tx=0x7ff258009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:07.988+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff250062580 con 0x7ff2640d1840 2024-09-21T06:28:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.227+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2640b65a0 con 0x7ff2640d1840 2024-09-21T06:28:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.228+0000 7ff260c00640 1 -- 172.21.15.37:0/63799275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5225 (secure 0 0 0) 0x7ff250061e80 con 0x7ff2640d1840 2024-09-21T06:28:08.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:08.230 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"btime":"2024-09-21T06:28:06:266663+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:05.777993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-09-21T06:28:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 msgr2=0x7ff24407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 0x7ff24407b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff258000f60 tx=0x7ff258009040 comp rx=0 tx=0).stop 2024-09-21T06:28:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 msgr2=0x7ff2640ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c400 tx=0x7ff25000c8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 shutdown_connections 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff244078b90 0x7ff24407b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2640d1840 0x7ff2640ca9f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640caf30 0x7ff2641e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 --2- 172.21.15.37:0/63799275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264071cf0 0x7ff26406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 >> 172.21.15.37:0/63799275 conn(0x7ff2640b3b30 msgr2=0x7ff2640b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 shutdown_connections 2024-09-21T06:28:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.230+0000 7ff26a200640 1 -- 172.21.15.37:0/63799275 wait complete. 2024-09-21T06:28:08.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 -- 172.21.15.37:0/3536967566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 msgr2=0x7f384406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 --2- 172.21.15.37:0/3536967566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f384406d2b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f382c009830 tx=0x7f382c02a500 comp rx=0 tx=0).stop 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 -- 172.21.15.37:0/3536967566 shutdown_connections 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 --2- 172.21.15.37:0/3536967566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f384406d2b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 --2- 172.21.15.37:0/3536967566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38440b9e40 0x7f38441622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 -- 172.21.15.37:0/3536967566 >> 172.21.15.37:0/3536967566 conn(0x7f38440d1930 msgr2=0x7f38440d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 -- 172.21.15.37:0/3536967566 shutdown_connections 2024-09-21T06:28:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.481+0000 7f3843e00640 1 -- 172.21.15.37:0/3536967566 wait complete. 2024-09-21T06:28:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.483+0000 7f3843e00640 1 Processor -- start 2024-09-21T06:28:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.483+0000 7f3843e00640 1 -- start start 2024-09-21T06:28:08.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f384406d7f0 0x7f38441531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f38441f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3844155940 con 0x7f38440b9e40 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38441557c0 con 0x7f38441627f0 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3844155ac0 con 0x7f384406d7f0 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f383be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f383be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54192/0 (socket says 172.21.15.37:54192) 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f383be00640 1 -- 172.21.15.37:0/1647507622 learned_addr learned my addr 172.21.15.37:0/1647507622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3842a00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f384406d7f0 0x7f38441531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.485+0000 7f3843400640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f38441f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 -- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f384406d7f0 msgr2=0x7f38441531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f384406d7f0 0x7f38441531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 -- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 msgr2=0x7f38441f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f38441f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 -- 172.21.15.37:0/1647507622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384415b4e0 con 0x7f38440b9e40 2024-09-21T06:28:08.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383be00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c005c90 con 0x7f38440b9e40 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f382c031e50 con 0x7f38440b9e40 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c033290 con 0x7f38440b9e40 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3844159d00 con 0x7f38440b9e40 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.486+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384415a2c0 con 0x7f38440b9e40 2024-09-21T06:28:08.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.487+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38441622b0 con 0x7f38440b9e40 2024-09-21T06:28:08.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.489+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f382c033430 con 0x7f38440b9e40 2024-09-21T06:28:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.489+0000 7f383aa00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 0x7f382007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.490+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f382c0bc1e0 con 0x7f38440b9e40 2024-09-21T06:28:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.490+0000 7f3842a00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 0x7f382007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.491+0000 7f3842a00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 0x7f382007b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3834005fd0 tx=0x7f3834009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.494+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f382c04a030 con 0x7f38440b9e40 2024-09-21T06:28:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.692+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3844160dc0 con 0x7f38440b9e40 2024-09-21T06:28:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.693+0000 7f383aa00640 1 -- 172.21.15.37:0/1647507622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f382c03f030 con 0x7f38440b9e40 2024-09-21T06:28:08.696 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:08.696 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 msgr2=0x7f382007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 0x7f382007b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3834005fd0 tx=0x7f3834009260 comp rx=0 tx=0).stop 2024-09-21T06:28:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 msgr2=0x7f3844153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c033c20 comp rx=0 tx=0).stop 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 shutdown_connections 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078b90 0x7f382007b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440b9e40 0x7f3844153710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38441627f0 0x7f38441f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 --2- 172.21.15.37:0/1647507622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f384406d7f0 0x7f38441531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 >> 172.21.15.37:0/1647507622 conn(0x7f38440d1930 msgr2=0x7f3844161160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 shutdown_connections 2024-09-21T06:28:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:08.696+0000 7f3843e00640 1 -- 172.21.15.37:0/1647507622 wait complete. 2024-09-21T06:28:09.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:28:00.716481+0000', 'modified': '2024-09-21T06:28:05.777993+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4399': {'gid': 4399, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/2075485628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 2075485628}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2075485628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4399, 'qdb_cluster': [4399]} 2024-09-21T06:28:09.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:28:09.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 -- 172.21.15.37:0/101515300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 msgr2=0x7f8a600cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 --2- 172.21.15.37:0/101515300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600cf310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a58005fd0 tx=0x7f8a5802aa00 comp rx=0 tx=0).stop 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 -- 172.21.15.37:0/101515300 shutdown_connections 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 --2- 172.21.15.37:0/101515300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600cf310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 --2- 172.21.15.37:0/101515300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 0x7f8a6006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 -- 172.21.15.37:0/101515300 >> 172.21.15.37:0/101515300 conn(0x7f8a600b1b90 msgr2=0x7f8a600b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 -- 172.21.15.37:0/101515300 shutdown_connections 2024-09-21T06:28:09.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.357+0000 7f8a66800640 1 -- 172.21.15.37:0/101515300 wait complete. 2024-09-21T06:28:09.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.360+0000 7f8a66800640 1 Processor -- start 2024-09-21T06:28:09.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.360+0000 7f8a66800640 1 -- start start 2024-09-21T06:28:09.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 0x7f8a601f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a60156a50 0x7f8a6015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600cf850 con 0x7f8a600cef00 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601610a0 con 0x7f8a60156a50 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a66800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8a600cf9d0 con 0x7f8a6006ddf0 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54216/0 (socket says 172.21.15.37:54216) 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a5ea00640 1 -- 172.21.15.37:0/21675627 learned_addr learned my addr 172.21.15.37:0/21675627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.361+0000 7f8a5f400640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 0x7f8a601f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a60156a50 0x7f8a6015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 -- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 msgr2=0x7f8a601f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 0x7f8a601f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 -- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a60156a50 msgr2=0x7f8a6015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a60156a50 0x7f8a6015b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 -- 172.21.15.37:0/21675627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a600a0de0 con 0x7f8a600cef00 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00d750 tx=0x7f8a4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a57400640 1 -- 172.21.15.37:0/21675627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c017070 con 0x7f8a600cef00 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a57400640 1 -- 172.21.15.37:0/21675627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a4c009b20 con 0x7f8a600cef00 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a600b4170 con 0x7f8a600cef00 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.362+0000 7f8a57400640 1 -- 172.21.15.37:0/21675627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c010460 con 0x7f8a600cef00 2024-09-21T06:28:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.363+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a600b4730 con 0x7f8a600cef00 2024-09-21T06:28:09.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.365+0000 7f8a57400640 1 -- 172.21.15.37:0/21675627 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x7f8a600cef00 2024-09-21T06:28:09.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.366+0000 7f8a57400640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 0x7f8a4007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.366+0000 7f8a57400640 1 -- 172.21.15.37:0/21675627 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8a4c09b180 con 0x7f8a600cef00 2024-09-21T06:28:09.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.366+0000 7f8a6da36740 1 Processor -- start 2024-09-21T06:28:09.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.367+0000 7f8a5f400640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 0x7f8a4007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.367+0000 7f8a6da36740 1 -- start start 2024-09-21T06:28:09.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.367+0000 7f8a5f400640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 0x7f8a4007b0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006fd0 tx=0x7f8a50009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e19209970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19209eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e19163430 con 0x556e18e18590 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556e191535d0 con 0x556e18e103b0 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556e19142a00 con 0x556e18e39270 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54218/0 (socket says 172.21.15.37:54218) 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a5fe00640 1 -- 172.21.15.37:0/1202618576 learned_addr learned my addr 172.21.15.37:0/1202618576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19209eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.368+0000 7f8a5f400640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e19209970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 -- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 msgr2=0x556e19209970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e19209970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 -- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 msgr2=0x556e19209eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19209eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 -- 172.21.15.37:0/1202618576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1910b520 con 0x556e18e18590 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19209eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802aff0 tx=0x7f8a58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeb37e5af9ba86f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a4be00640 1 -- 172.21.15.37:0/1202618576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58047070 con 0x556e18e18590 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a4be00640 1 -- 172.21.15.37:0/1202618576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a58003cb0 con 0x556e18e18590 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.369+0000 7f8a4be00640 1 -- 172.21.15.37:0/1202618576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58033070 con 0x556e18e18590 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 -- 172.21.15.37:0/1202618576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 msgr2=0x556e191d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802aff0 tx=0x7f8a58004ab0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 -- 172.21.15.37:0/1202618576 shutdown_connections 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191d9f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19209eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 --2- 172.21.15.37:0/1202618576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e19209970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 -- 172.21.15.37:0/1202618576 >> 172.21.15.37:0/1202618576 conn(0x556e18e2d3c0 msgr2=0x556e191df970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 -- 172.21.15.37:0/1202618576 shutdown_connections 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.370+0000 7f8a6da36740 1 -- 172.21.15.37:0/1202618576 wait complete. 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.371+0000 7f8a6da36740 1 -- start start 2024-09-21T06:28:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.372+0000 7f8a6da36740 1 Processor -- start 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19236d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e192257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e190e9330 con 0x556e18e18590 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556e19146810 con 0x556e18e103b0 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556e18eb3620 con 0x556e18e39270 2024-09-21T06:28:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a6da36740 10 client.0 _ll_register_callbacks cb 0x556e190ee420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.373+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54226/0 (socket says 172.21.15.37:54226) 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 -- 172.21.15.37:0/707372717 learned_addr learned my addr 172.21.15.37:0/707372717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e192257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5f400640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19236d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 -- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 msgr2=0x556e192257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e192257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 -- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 msgr2=0x556e19236d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19236d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 -- 172.21.15.37:0/707372717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e18ea4850 con 0x556e18e18590 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00d720 tx=0x7f8a4c062730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1144417a6b3cf046 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c021070 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8a4c067100 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.374+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a58003cb0 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c04e1a0 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 10 client.4494 fetch_fsmap learned FSMap version 6 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x556e18f1b6b0 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a58003cb0 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f8a4c05e570 con 0x556e18e18590 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 10 client.4494 fetch_fsmap finished waiting for FSMap version 6 2024-09-21T06:28:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 10 client.4494 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 20 client.4494 populate_metadata read hostname 'smithi037' 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e18df7e40 0x556e19205460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 4 client.4494 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.375+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 --> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e19155520 con 0x556e18df7e40 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.376+0000 7f8a5f400640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e18df7e40 0x556e19205460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.376+0000 7f8a5f400640 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e18df7e40 0x556e19205460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d14c2dde7b943d6 server_cookie=6cedb2221ede9bce in_seq=0 out_seq=0 2024-09-21T06:28:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.376+0000 7f8a48c00640 10 client.4494 ms_handle_connect on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.377+0000 7f8a48c00640 1 -- 172.21.15.37:0/707372717 <== mds.0 v2:172.21.15.150:6835/2075485628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e19155520 con 0x556e18df7e40 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.377+0000 7f8a48c00640 10 client.4494 handle_command_reply: tid=0 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.377+0000 7f8a6da36740 1 client.4494 shutdown 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 msgr2=0x556e191f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00d720 tx=0x7f8a4c062730 comp rx=0 tx=0).stop 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 shutdown_connections 2024-09-21T06:28:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e18df7e40 0x556e19205460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e191f4180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e39270 0x556e192257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 --2- 172.21.15.37:0/707372717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e103b0 0x556e19236d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a48c00640 0 client.4494 ms_handle_reset on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 >> 172.21.15.37:0/707372717 conn(0x556e18e2d3c0 msgr2=0x556e191db2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 shutdown_connections 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 1 -- 172.21.15.37:0/707372717 wait complete. 2024-09-21T06:28:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.378+0000 7f8a6da36740 20 client.4494 trim_cache size 0 max 16384 2024-09-21T06:28:09.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.397+0000 7f8a6da36740 1 Processor -- start 2024-09-21T06:28:09.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.397+0000 7f8a6da36740 1 -- start start 2024-09-21T06:28:09.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e191db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e39270 0x556e19236460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a602035f0 con 0x556e18e18590 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60202610 con 0x556e18e39270 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8a60159210 con 0x556e18e103b0 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54230/0 (socket says 172.21.15.37:54230) 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 -- 172.21.15.37:0/444806413 learned_addr learned my addr 172.21.15.37:0/444806413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 -- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 msgr2=0x556e191db890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e191db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 -- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e39270 msgr2=0x556e19236460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e39270 0x556e19236460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.399+0000 7f8a5ea00640 1 -- 172.21.15.37:0/444806413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1921f860 con 0x556e18e18590 2024-09-21T06:28:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c062bb0 tx=0x7f8a4c062860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbfc11e3941d514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a4b400640 1 -- 172.21.15.37:0/444806413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c06c500 con 0x556e18e18590 2024-09-21T06:28:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a4b400640 1 -- 172.21.15.37:0/444806413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a4c080020 con 0x556e18e18590 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a4b400640 1 -- 172.21.15.37:0/444806413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c04e070 con 0x556e18e18590 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 -- 172.21.15.37:0/444806413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 msgr2=0x556e19235f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c062bb0 tx=0x7f8a4c062860 comp rx=0 tx=0).stop 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 -- 172.21.15.37:0/444806413 shutdown_connections 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e18590 0x556e19235f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e39270 0x556e19236460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 --2- 172.21.15.37:0/444806413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e191db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.400+0000 7f8a6da36740 1 -- 172.21.15.37:0/444806413 >> 172.21.15.37:0/444806413 conn(0x556e18e2d3c0 msgr2=0x556e19220300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.401+0000 7f8a6da36740 1 -- 172.21.15.37:0/444806413 shutdown_connections 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.401+0000 7f8a6da36740 1 -- 172.21.15.37:0/444806413 wait complete. 2024-09-21T06:28:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.401+0000 7f8a6da36740 1 -- start start 2024-09-21T06:28:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.402+0000 7f8a6da36740 1 Processor -- start 2024-09-21T06:28:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e192a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 0x556e19220290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e19206480 con 0x556e18e39270 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60201ff0 con 0x556e18e18590 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556e19202fd0 con 0x556e18e103b0 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a6da36740 10 client.0 _ll_register_callbacks cb 0x7f8a4c062390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54238/0 (socket says 172.21.15.37:54238) 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.403+0000 7f8a5fe00640 1 -- 172.21.15.37:0/372352636 learned_addr learned my addr 172.21.15.37:0/372352636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 0x556e19220290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5f400640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e192a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 -- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 msgr2=0x556e192a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e192a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 -- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 msgr2=0x556e19220290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 0x556e19220290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 -- 172.21.15.37:0/372352636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e19247a10 con 0x556e18e39270 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5f400640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e192a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5ea00640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 0x556e19220290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a5fe00640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006010 tx=0x7f8a5802fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0277410fbe74e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:28:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58047030 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8a58038660 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.404+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c080020 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f8a58033040 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 10 client.4500 fetch_fsmap learned FSMap version 6 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x556e192a7be0 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a4c080020 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f8a58004b40 con 0x556e18e39270 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 10 client.4500 fetch_fsmap finished waiting for FSMap version 6 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 10 client.4500 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 20 client.4500 populate_metadata read hostname 'smithi037' 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e192a7470 0x556e1921e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 4 client.4500 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:28:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 --> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] -- command(tid 0: {"prefix": "status"}) -- 0x556e19150fd0 con 0x556e192a7470 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.405+0000 7f8a5f400640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e192a7470 0x556e1921e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a5f400640 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e192a7470 0x556e1921e4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ca0dba496f1dc8 server_cookie=e3929b724df00085 in_seq=0 out_seq=0 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a48c00640 10 client.4500 ms_handle_connect on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a48c00640 1 -- 172.21.15.37:0/372352636 <== mds.0 v2:172.21.15.150:6835/2075485628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x556e19150fd0 con 0x556e192a7470 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a48c00640 10 client.4500 handle_command_reply: tid=0 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a6da36740 1 client.4500 shutdown 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 msgr2=0x556e19235f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.406+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006010 tx=0x7f8a5802fb80 comp rx=0 tx=0).stop 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 shutdown_connections 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x556e192a7470 0x556e1921e4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e18e39270 0x556e19235f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556e18e18590 0x556e19220290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a48c00640 0 client.4500 ms_handle_reset on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 --2- 172.21.15.37:0/372352636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556e18e103b0 0x556e192a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 >> 172.21.15.37:0/372352636 conn(0x556e18e2d3c0 msgr2=0x556e19224e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 shutdown_connections 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 1 -- 172.21.15.37:0/372352636 wait complete. 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a6da36740 20 client.4500 trim_cache size 0 max 16384 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4399, 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:28:09.412 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 8.1825237459999993, 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 6, 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 27, 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 27, 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 14.602191414 2024-09-21T06:28:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 msgr2=0x7f8a4007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 0x7f8a4007b0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006fd0 tx=0x7f8a50009060 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 msgr2=0x7f8a600a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.407+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00d750 tx=0x7f8a4c00dc10 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 shutdown_connections 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a40078bc0 0x7f8a4007b0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a600cef00 0x7f8a600a08a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a60156a50 0x7f8a6015b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 --2- 172.21.15.37:0/21675627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a6006ddf0 0x7f8a601f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 >> 172.21.15.37:0/21675627 conn(0x7f8a600b1b90 msgr2=0x7f8a600cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 shutdown_connections 2024-09-21T06:28:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.408+0000 7f8a66800640 1 -- 172.21.15.37:0/21675627 wait complete. 2024-09-21T06:28:09.433 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:28:09.433 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:09.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:28:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 -- 172.21.15.37:0/914039070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c073dd0 msgr2=0x7f889c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 --2- 172.21.15.37:0/914039070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c073dd0 0x7f889c06ddf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8884005fd0 tx=0x7f888402a670 comp rx=0 tx=0).stop 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 -- 172.21.15.37:0/914039070 shutdown_connections 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 --2- 172.21.15.37:0/914039070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c073dd0 0x7f889c06ddf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 -- 172.21.15.37:0/914039070 >> 172.21.15.37:0/914039070 conn(0x7f889c0cca40 msgr2=0x7f889c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 -- 172.21.15.37:0/914039070 shutdown_connections 2024-09-21T06:28:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.657+0000 7f88a2c00640 1 -- 172.21.15.37:0/914039070 wait complete. 2024-09-21T06:28:09.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.659+0000 7f88a2c00640 1 Processor -- start 2024-09-21T06:28:09.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.659+0000 7f88a2c00640 1 -- start start 2024-09-21T06:28:09.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f889c073dd0 0x7f889c09cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f889c074cb0 0x7f889c09d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f889c06f670 con 0x7f889c06e330 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f889c06f4f0 con 0x7f889c074cb0 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a2c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f889c06f7f0 con 0x7f889c073dd0 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54256/0 (socket says 172.21.15.37:54256) 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 -- 172.21.15.37:0/3119363016 learned_addr learned my addr 172.21.15.37:0/3119363016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f88a0c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f889c074cb0 0x7f889c09d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f8893e00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f889c073dd0 0x7f889c09cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 -- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f889c073dd0 msgr2=0x7f889c09cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f889c073dd0 0x7f889c09cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 -- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f889c074cb0 msgr2=0x7f889c09d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f889c074cb0 0x7f889c09d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.661+0000 7f889be00640 1 -- 172.21.15.37:0/3119363016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c09f610 con 0x7f889c06e330 2024-09-21T06:28:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f889be00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c340 tx=0x7f888c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c017070 con 0x7f889c06e330 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f888c0035f0 con 0x7f889c06e330 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c09f8f0 con 0x7f889c06e330 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c003a80 con 0x7f889c06e330 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.662+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c09feb0 con 0x7f889c06e330 2024-09-21T06:28:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.663+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c06ddf0 con 0x7f889c06e330 2024-09-21T06:28:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.664+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f888c01a070 con 0x7f889c06e330 2024-09-21T06:28:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.665+0000 7f8898c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 0x7f887807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.665+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f888c09a320 con 0x7f889c06e330 2024-09-21T06:28:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.666+0000 7f8893e00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 0x7f887807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.666+0000 7f8893e00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 0x7f887807b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8884033000 tx=0x7f8884004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:09.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.670+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f888c061d30 con 0x7f889c06e330 2024-09-21T06:28:09.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.905+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f889c15e0a0 con 0x7f889c06e330 2024-09-21T06:28:09.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.906+0000 7f8898c00640 1 -- 172.21.15.37:0/3119363016 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f888c066760 con 0x7f889c06e330 2024-09-21T06:28:09.909 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:28:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 msgr2=0x7f887807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 0x7f887807b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8884033000 tx=0x7f8884004730 comp rx=0 tx=0).stop 2024-09-21T06:28:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 msgr2=0x7f889c1f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c340 tx=0x7f888c00c800 comp rx=0 tx=0).stop 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 shutdown_connections 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8878078bc0 0x7f887807b0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f889c074cb0 0x7f889c09d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f889c073dd0 0x7f889c09cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 --2- 172.21.15.37:0/3119363016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f889c06e330 0x7f889c1f66d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 >> 172.21.15.37:0/3119363016 conn(0x7f889c0cca40 msgr2=0x7f889c0ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 shutdown_connections 2024-09-21T06:28:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:09.909+0000 7f88a2c00640 1 -- 172.21.15.37:0/3119363016 wait complete. 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:28:09.936 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:28:09.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:09.937 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:28:09.972 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:28:09.973 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42702sec preferred_lft 42702sec 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:09.974 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:28:09.975 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:28:09.975 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:10.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:10.390 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:28:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:28:10.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:10.450 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:28:10.501 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:28:10.501 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:28:10.501 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:28:10.501 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:28:10.501 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:28:10.502 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:11.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:11.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:11.139 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:28:11.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:11.176 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-21T06:28:11.251 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:11.252 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:11.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:11.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T06:28:11.252 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:11.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:11.432 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T06:28:11.432 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:11.432 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T06:28:11.433 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:11.677 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:11.677 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:11.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:28:11.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:28:11.677 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:11.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:11.845 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T06:28:11.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:11.845 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:11.881 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:28:11.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:11.882 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:11.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-21T06:28:12.052 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:28:12.088 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:28:12.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T06:28:12.090 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T06:28:12.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T06:28:12.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-21T06:28:12.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:12.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:12.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:12.092 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-21T06:28:12.167 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T06:28:12.167 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T06:28:12.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-21T06:28:12.268 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-21T06:28:12.305 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:28:12.306 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:12.307 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:12.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:12.308 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-21T06:28:12.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:2024-09-21T06:28:12.462+0000 7f6237946480 -1 init, newargv = 0x555a0db5c600 newargc=15 2024-09-21T06:28:12.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[43200]: starting ceph client 2024-09-21T06:28:12.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[43200]: starting fuse 2024-09-21T06:28:13.485 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-21T06:28:13.549 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:28:13.587 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-21T06:28:13.587 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-21T06:28:13.587 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-21T06:28:13.587 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-21T06:28:13.587 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-21T06:28:13.588 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:13.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:13.590 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-21T06:28:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:41 2024-09-21T06:28:13.667 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-09-21T06:28:13.668 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.admin")) 2024-09-21T06:28:13.669 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:28:13.832 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.admin.43200.asok 2024-09-21T06:28:13.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:13.847 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43200.asok 2024-09-21T06:28:13.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:13.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43200.asok status 2024-09-21T06:28:14.096 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:28:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "admin", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "43200", 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4442, 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-09-21T06:28:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4442 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2107554789 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2107554789 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4442 172.21.15.37:0/2107554789", 2024-09-21T06:28:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "172.21.15.37:0/2107554789", 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 6, 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 27, 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-09-21T06:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:28:14.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:14.114 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-09-21T06:28:14.114 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:14.150 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-09-21T06:28:14.150 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:14.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:14.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:14.247 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T06:28:14.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:14.356 INFO:teuthology.orchestra.run.smithi037.stdout:# file: . 2024-09-21T06:28:14.356 INFO:teuthology.orchestra.run.smithi037.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T06:28:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:14.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:14.364 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:14.364 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:14.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 -- 172.21.15.37:0/4207851695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e8156a50 msgr2=0x7f12d0003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 --2- 172.21.15.37:0/4207851695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e8156a50 0x7f12d0003a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009b50 tx=0x7f12d0005830 comp rx=0 tx=0).stop 2024-09-21T06:28:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ed400640 1 -- 172.21.15.37:0/4207851695 reap_dead start 2024-09-21T06:28:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 -- 172.21.15.37:0/4207851695 shutdown_connections 2024-09-21T06:28:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 -- 172.21.15.37:0/4207851695 >> 172.21.15.37:0/4207851695 conn(0x7f12e80b1b90 msgr2=0x7f12e80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 -- 172.21.15.37:0/4207851695 shutdown_connections 2024-09-21T06:28:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.667+0000 7f12ef400640 1 -- 172.21.15.37:0/4207851695 wait complete. 2024-09-21T06:28:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.669+0000 7f12ef400640 1 Processor -- start 2024-09-21T06:28:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.669+0000 7f12ef400640 1 -- start start 2024-09-21T06:28:14.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 0x7f12e81f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 0x7f12e81f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8074620 con 0x7f12e8156a50 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8161060 con 0x7f12e80c5d30 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12ef400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f12e80747a0 con 0x7f12e80c6140 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42046/0 (socket says 172.21.15.37:42046) 2024-09-21T06:28:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.671+0000 7f12eca00640 1 -- 172.21.15.37:0/988174120 learned_addr learned my addr 172.21.15.37:0/988174120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12e7e00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 0x7f12e81f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12ed400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 0x7f12e81f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 -- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 msgr2=0x7f12e81f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 0x7f12e81f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 -- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 msgr2=0x7f12e81f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 0x7f12e81f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 -- 172.21.15.37:0/988174120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80b5810 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12e7e00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 0x7f12e81f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12ed400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 0x7f12e81f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12eca00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12dc036040 tx=0x7f12dc033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc038040 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12dc041070 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc033cd0 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e80b5530 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.672+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81f6620 con 0x7f12e8156a50 2024-09-21T06:28:14.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.674+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e806ddf0 con 0x7f12e8156a50 2024-09-21T06:28:14.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.675+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12dc046460 con 0x7f12e8156a50 2024-09-21T06:28:14.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.676+0000 7f12e4c00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 0x7f12c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.676+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f12dc0bbbd0 con 0x7f12e8156a50 2024-09-21T06:28:14.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.676+0000 7f12e7e00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 0x7f12c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.677+0000 7f12e7e00640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 0x7f12c807b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12d4005eb0 tx=0x7f12d4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:14.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.680+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12dc083c30 con 0x7f12e8156a50 2024-09-21T06:28:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.890+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12e80696c0 con 0x7f12e8156a50 2024-09-21T06:28:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.890+0000 7f12e4c00640 1 -- 172.21.15.37:0/988174120 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f12dc083530 con 0x7f12e8156a50 2024-09-21T06:28:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.892+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 msgr2=0x7f12c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.892+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 0x7f12c807b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12d4005eb0 tx=0x7f12d4005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.892+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 msgr2=0x7f12e80cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.892+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12dc036040 tx=0x7f12dc033670 comp rx=0 tx=0).stop 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.892+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 shutdown_connections 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12c8078bc0 0x7f12c807b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12e80c6140 0x7f12e81f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e80c5d30 0x7f12e81f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 --2- 172.21.15.37:0/988174120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e8156a50 0x7f12e80cc240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 >> 172.21.15.37:0/988174120 conn(0x7f12e80b1b90 msgr2=0x7f12e8072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 shutdown_connections 2024-09-21T06:28:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:14.893+0000 7f12ef400640 1 -- 172.21.15.37:0/988174120 wait complete. 2024-09-21T06:28:14.917 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:14.917 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:14.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 -- 172.21.15.37:0/2123205137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c06ddf0 msgr2=0x7f336c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 --2- 172.21.15.37:0/2123205137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c06ddf0 0x7f336c06e200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3358009830 tx=0x7f335802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 -- 172.21.15.37:0/2123205137 shutdown_connections 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 --2- 172.21.15.37:0/2123205137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c06ddf0 0x7f336c06e200 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 -- 172.21.15.37:0/2123205137 >> 172.21.15.37:0/2123205137 conn(0x7f336c0b1b90 msgr2=0x7f336c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 -- 172.21.15.37:0/2123205137 shutdown_connections 2024-09-21T06:28:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.219+0000 7f3372200640 1 -- 172.21.15.37:0/2123205137 wait complete. 2024-09-21T06:28:15.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.222+0000 7f3372200640 1 Processor -- start 2024-09-21T06:28:15.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.222+0000 7f3372200640 1 -- start start 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 0x7f336c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f336c156a50 0x7f336c1f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f336c160fd0 con 0x7f336c0cef00 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f336c160e50 con 0x7f336c06ddf0 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.223+0000 7f3372200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f336c0cfa20 con 0x7f336c156a50 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42068/0 (socket says 172.21.15.37:42068) 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 -- 172.21.15.37:0/2304210107 learned_addr learned my addr 172.21.15.37:0/2304210107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336b400640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 0x7f336c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336be00640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f336c156a50 0x7f336c1f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 -- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f336c156a50 msgr2=0x7f336c1f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f336c156a50 0x7f336c1f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 -- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 msgr2=0x7f336c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 0x7f336c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 -- 172.21.15.37:0/2304210107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c1f9880 con 0x7f336c0cef00 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336b400640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 0x7f336c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.224+0000 7f336aa00640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3358009950 tx=0x7f3358033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.225+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358031b60 con 0x7f336c0cef00 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.225+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3358031df0 con 0x7f336c0cef00 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.225+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358005280 con 0x7f336c0cef00 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.225+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c0b5af0 con 0x7f336c0cef00 2024-09-21T06:28:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.225+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c0a04c0 con 0x7f336c0cef00 2024-09-21T06:28:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.226+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c0b58e0 con 0x7f336c0cef00 2024-09-21T06:28:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.227+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3358005420 con 0x7f336c0cef00 2024-09-21T06:28:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.228+0000 7f3363400640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 0x7f334c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.228+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f33580bc300 con 0x7f336c0cef00 2024-09-21T06:28:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.228+0000 7f336b400640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 0x7f334c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.229+0000 7f336b400640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 0x7f334c07b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f335c000f60 tx=0x7f335c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.232+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f335803f030 con 0x7f336c0cef00 2024-09-21T06:28:15.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.442+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f336c06e200 con 0x7f336c0cef00 2024-09-21T06:28:15.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.442+0000 7f3363400640 1 -- 172.21.15.37:0/2304210107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f3358031770 con 0x7f336c0cef00 2024-09-21T06:28:15.444 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.444+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 msgr2=0x7f334c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.444+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 0x7f334c07b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f335c000f60 tx=0x7f335c009040 comp rx=0 tx=0).stop 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 msgr2=0x7f336c1f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3358009950 tx=0x7f3358033c10 comp rx=0 tx=0).stop 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 shutdown_connections 2024-09-21T06:28:15.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f334c078b90 0x7f334c07b0a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c0cef00 0x7f336c1f8b00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f336c156a50 0x7f336c1f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 --2- 172.21.15.37:0/2304210107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c06ddf0 0x7f336c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 >> 172.21.15.37:0/2304210107 conn(0x7f336c0b1b90 msgr2=0x7f336c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 shutdown_connections 2024-09-21T06:28:15.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.445+0000 7f3372200640 1 -- 172.21.15.37:0/2304210107 wait complete. 2024-09-21T06:28:15.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-09-21T06:28:15.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:15.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:28:15.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[43200]: fuse finished with error 0 and tester_r 0 2024-09-21T06:28:15.528 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:28:15.580 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:28:15.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:15.580 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T06:28:15.623 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 -- 172.21.15.37:0/2044319931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015b9f0 msgr2=0x7f9f000cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 --2- 172.21.15.37:0/2044319931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015b9f0 0x7f9f000cf930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009860 tx=0x7f9ef002a5a0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 -- 172.21.15.37:0/2044319931 shutdown_connections 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 --2- 172.21.15.37:0/2044319931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015b9f0 0x7f9f000cf930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 -- 172.21.15.37:0/2044319931 >> 172.21.15.37:0/2044319931 conn(0x7f9f000cca40 msgr2=0x7f9f000cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.886+0000 7f9f06400640 1 -- 172.21.15.37:0/2044319931 shutdown_connections 2024-09-21T06:28:15.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.887+0000 7f9f06400640 1 -- 172.21.15.37:0/2044319931 wait complete. 2024-09-21T06:28:15.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.889+0000 7f9f06400640 1 Processor -- start 2024-09-21T06:28:15.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.889+0000 7f9f06400640 1 -- start start 2024-09-21T06:28:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 0x7f9f001f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 0x7f9f0009cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015f5a0 0x7f9f0009d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0015f9b0 con 0x7f9f0015b9f0 2024-09-21T06:28:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0006e710 con 0x7f9f000cefa0 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.890+0000 7f9f06400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0015fb30 con 0x7f9f0015f5a0 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 0x7f9f0009cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9eff400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 0x7f9f001f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9eff400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 0x7f9f001f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:54832/0 (socket says 172.21.15.37:54832) 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9eff400640 1 -- 172.21.15.37:0/1860079748 learned_addr learned my addr 172.21.15.37:0/1860079748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9effe00640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015f5a0 0x7f9f0009d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 -- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015f5a0 msgr2=0x7f9f0009d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015f5a0 0x7f9f0009d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 -- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 msgr2=0x7f9f001f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 0x7f9f001f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 -- 172.21.15.37:0/1860079748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0015ccc0 con 0x7f9f0015b9f0 2024-09-21T06:28:15.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.891+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 0x7f9f0009cb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c3c0 tx=0x7f9ee800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.892+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8011790 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.892+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ee8017070 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.892+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8011cc0 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.892+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0015cf60 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.892+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0015d520 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.893+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f000cfc60 con 0x7f9f0015b9f0 2024-09-21T06:28:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.894+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ee801c460 con 0x7f9f0015b9f0 2024-09-21T06:28:15.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.895+0000 7f9ef7400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 0x7f9edc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.895+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9ee809a9d0 con 0x7f9f0015b9f0 2024-09-21T06:28:15.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.896+0000 7f9eff400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 0x7f9edc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.896+0000 7f9eff400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 0x7f9edc07b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009860 tx=0x7f9ef0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:15.899+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f000cfc60 con 0x7f9f0015b9f0 2024-09-21T06:28:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.099+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f000cf930 con 0x7f9f0015b9f0 2024-09-21T06:28:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.100+0000 7f9ef7400640 1 -- 172.21.15.37:0/1860079748 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f9ee8021020 con 0x7f9f0015b9f0 2024-09-21T06:28:16.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:16.103 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 msgr2=0x7f9edc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 0x7f9edc07b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009860 tx=0x7f9ef0004730 comp rx=0 tx=0).stop 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 msgr2=0x7f9f0009cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 0x7f9f0009cb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c3c0 tx=0x7f9ee800c880 comp rx=0 tx=0).stop 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 shutdown_connections 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9edc078bc0 0x7f9edc07b0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f0015b9f0 0x7f9f0009cb60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f0015f5a0 0x7f9f0009d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 --2- 172.21.15.37:0/1860079748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f000cefa0 0x7f9f001f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 >> 172.21.15.37:0/1860079748 conn(0x7f9f000cca40 msgr2=0x7f9f0006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 shutdown_connections 2024-09-21T06:28:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.103+0000 7f9f06400640 1 -- 172.21.15.37:0/1860079748 wait complete. 2024-09-21T06:28:16.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:16.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.355+0000 7fa758800640 1 -- 172.21.15.37:0/4155789937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa738005ea0 msgr2=0x7fa738006350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.355+0000 7fa758800640 1 --2- 172.21.15.37:0/4155789937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa738005ea0 0x7fa738006350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7500cc8f0 tx=0x7fa738006940 comp rx=0 tx=0).stop 2024-09-21T06:28:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.355+0000 7fa756800640 1 -- 172.21.15.37:0/4155789937 reap_dead start 2024-09-21T06:28:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.355+0000 7fa758800640 1 -- 172.21.15.37:0/4155789937 shutdown_connections 2024-09-21T06:28:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.355+0000 7fa758800640 1 -- 172.21.15.37:0/4155789937 >> 172.21.15.37:0/4155789937 conn(0x7fa7500cca40 msgr2=0x7fa7500cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.356+0000 7fa758800640 1 -- 172.21.15.37:0/4155789937 shutdown_connections 2024-09-21T06:28:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.356+0000 7fa758800640 1 -- 172.21.15.37:0/4155789937 wait complete. 2024-09-21T06:28:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.358+0000 7fa758800640 1 Processor -- start 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.358+0000 7fa758800640 1 -- start start 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 0x7fa750151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7501519a0 0x7fa7501f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa750071000 con 0x7fa75015cce0 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa750070e80 con 0x7fa750151050 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa758800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa75006ddf0 con 0x7fa7501519a0 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42098/0 (socket says 172.21.15.37:42098) 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 -- 172.21.15.37:0/69846008 learned_addr learned my addr 172.21.15.37:0/69846008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755400640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 0x7fa750151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa756800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7501519a0 0x7fa7501f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 -- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7501519a0 msgr2=0x7fa7501f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7501519a0 0x7fa7501f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 -- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 msgr2=0x7fa750151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 0x7fa750151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755e00640 1 -- 172.21.15.37:0/69846008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7500c4aa0 con 0x7fa75015cce0 2024-09-21T06:28:16.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.360+0000 7fa755400640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 0x7fa750151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa755e00640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa7440066e0 tx=0x7fa744003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400f780 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa744016020 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400fbc0 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75015d0f0 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.361+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75015d6a0 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.362+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75006f710 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.363+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa744011430 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.364+0000 7fa74a000640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 0x7fa73407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.364+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa74409ba50 con 0x7fa75015cce0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.365+0000 7fa755400640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 0x7fa73407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.366+0000 7fa755400640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 0x7fa73407b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa75015e330 tx=0x7fa740005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.369+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa744020030 con 0x7fa75015cce0 2024-09-21T06:28:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.566+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7500b5f40 con 0x7fa75015cce0 2024-09-21T06:28:16.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.567+0000 7fa74a000640 1 -- 172.21.15.37:0/69846008 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7fa744063ac0 con 0x7fa75015cce0 2024-09-21T06:28:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:16.570 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 msgr2=0x7fa73407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 0x7fa73407b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa75015e330 tx=0x7fa740005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 msgr2=0x7fa75015b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa7440066e0 tx=0x7fa744003aa0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 shutdown_connections 2024-09-21T06:28:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa734078b90 0x7fa73407b0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7501519a0 0x7fa7501f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa750151050 0x7fa750151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 --2- 172.21.15.37:0/69846008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa75015cce0 0x7fa75015b2a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 >> 172.21.15.37:0/69846008 conn(0x7fa7500cca40 msgr2=0x7fa7500cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 shutdown_connections 2024-09-21T06:28:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.570+0000 7fa758800640 1 -- 172.21.15.37:0/69846008 wait complete. 2024-09-21T06:28:16.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:28:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 -- 172.21.15.37:0/2348161424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8152450 msgr2=0x7f81c806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 --2- 172.21.15.37:0/2348161424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8152450 0x7f81c806e720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f81c80ce890 tx=0x7f81b002a410 comp rx=0 tx=0).stop 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 -- 172.21.15.37:0/2348161424 shutdown_connections 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 --2- 172.21.15.37:0/2348161424 >> v2:172.21.15.150:3300/0 conn(0x7f81c8071160 0x7f81c8071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 --2- 172.21.15.37:0/2348161424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8152450 0x7f81c806e720 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 -- 172.21.15.37:0/2348161424 >> 172.21.15.37:0/2348161424 conn(0x7f81c80d1020 msgr2=0x7f81c80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 -- 172.21.15.37:0/2348161424 shutdown_connections 2024-09-21T06:28:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.823+0000 7f81cda00640 1 -- 172.21.15.37:0/2348161424 wait complete. 2024-09-21T06:28:16.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.825+0000 7f81cda00640 1 Processor -- start 2024-09-21T06:28:16.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.825+0000 7f81cda00640 1 -- start start 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 0x7f81c8140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81c8152450 0x7f81c81f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81c80b40d0 con 0x7f81c8072320 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f81c80b3f50 con 0x7f81c8071160 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cda00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f81c80b4250 con 0x7f81c8152450 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42106/0 (socket says 172.21.15.37:42106) 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81bf400640 1 -- 172.21.15.37:0/3798884659 learned_addr learned my addr 172.21.15.37:0/3798884659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81c7e00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 0x7f81c8140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.827+0000 7f81cd000640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81c8152450 0x7f81c81f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 -- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81c8152450 msgr2=0x7f81c81f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81c8152450 0x7f81c81f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 -- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 msgr2=0x7f81c8140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 0x7f81c8140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 -- 172.21.15.37:0/3798884659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c81f75e0 con 0x7f81c8072320 2024-09-21T06:28:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81c7e00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 0x7f81c8140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81bf400640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f81b002aa00 tx=0x7f81b0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0031d60 con 0x7f81c8072320 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b0033e10 con 0x7f81c8072320 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0005260 con 0x7f81c8072320 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81f78c0 con 0x7f81c8072320 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.828+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c814fab0 con 0x7f81c8072320 2024-09-21T06:28:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.830+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c80d0e10 con 0x7f81c8072320 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.831+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81b0005400 con 0x7f81c8072320 2024-09-21T06:28:16.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.831+0000 7f81c4c00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 0x7f81a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.832+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f81b00bc300 con 0x7f81c8072320 2024-09-21T06:28:16.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.832+0000 7f81c7e00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 0x7f81a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.833+0000 7f81c7e00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 0x7f81a407b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81b80046b0 tx=0x7f81b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:16.836+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81c80d0e10 con 0x7f81c8072320 2024-09-21T06:28:17.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.078+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f81c80cdd80 con 0x7f81c8072320 2024-09-21T06:28:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.959+0000 7f81c4c00640 1 -- 172.21.15.37:0/3798884659 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f81c80cdd80 con 0x7f81c8072320 2024-09-21T06:28:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 msgr2=0x7f81a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 0x7f81a407b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81b80046b0 tx=0x7f81b8009040 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 msgr2=0x7f81c81f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f81b002aa00 tx=0x7f81b0033c20 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 shutdown_connections 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f81a4078b90 0x7f81a407b0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c8072320 0x7f81c81f6a10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81c8152450 0x7f81c81f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 --2- 172.21.15.37:0/3798884659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81c8071160 0x7f81c8140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 >> 172.21.15.37:0/3798884659 conn(0x7f81c80d1020 msgr2=0x7f81c80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 shutdown_connections 2024-09-21T06:28:17.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:17.963+0000 7f81cda00640 1 -- 172.21.15.37:0/3798884659 wait complete. 2024-09-21T06:28:17.992 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-21T06:28:17.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:28:18.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 -- 172.21.15.37:0/3158236813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c162890 msgr2=0x7fe26c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 --2- 172.21.15.37:0/3158236813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c162890 0x7fe26c154f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe258005fd0 tx=0x7fe25802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 -- 172.21.15.37:0/3158236813 shutdown_connections 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 --2- 172.21.15.37:0/3158236813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c162890 0x7fe26c154f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 --2- 172.21.15.37:0/3158236813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c0d2760 0x7fe26c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 -- 172.21.15.37:0/3158236813 >> 172.21.15.37:0/3158236813 conn(0x7fe26c0d4360 msgr2=0x7fe26c0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 -- 172.21.15.37:0/3158236813 shutdown_connections 2024-09-21T06:28:18.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.226+0000 7fe272e00640 1 -- 172.21.15.37:0/3158236813 wait complete. 2024-09-21T06:28:18.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.228+0000 7fe272e00640 1 Processor -- start 2024-09-21T06:28:18.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.229+0000 7fe272e00640 1 -- start start 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 0x7fe26c1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c162890 0x7fe26c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c15f290 con 0x7fe26c155460 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0cfee0 con 0x7fe26c0d2760 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe272e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c15f410 con 0x7fe26c162890 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.230+0000 7fe271000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42126/0 (socket says 172.21.15.37:42126) 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 -- 172.21.15.37:0/212658185 learned_addr learned my addr 172.21.15.37:0/212658185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe272400640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c162890 0x7fe26c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:18.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271a00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 0x7fe26c1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:18.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 -- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c162890 msgr2=0x7fe26c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c162890 0x7fe26c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 -- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 msgr2=0x7fe26c1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 0x7fe26c1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271000640 1 -- 172.21.15.37:0/212658185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1f6240 con 0x7fe26c155460 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.231+0000 7fe271a00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 0x7fe26c1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe271000640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe258006010 tx=0x7fe2580035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25803e030 con 0x7fe26c155460 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe258003ae0 con 0x7fe26c155460 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258035040 con 0x7fe26c155460 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c0c8e10 con 0x7fe26c155460 2024-09-21T06:28:18.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.232+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c0c93d0 con 0x7fe26c155460 2024-09-21T06:28:18.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.233+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c15f930 con 0x7fe26c155460 2024-09-21T06:28:18.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.234+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe258041030 con 0x7fe26c155460 2024-09-21T06:28:18.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.236+0000 7fe269600640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 0x7fe24c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:18.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.236+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe2580bb610 con 0x7fe26c155460 2024-09-21T06:28:18.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.236+0000 7fe271a00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 0x7fe24c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:18.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.237+0000 7fe271a00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 0x7fe24c07b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fa0 tx=0x7fe260005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.241+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe258083020 con 0x7fe26c155460 2024-09-21T06:28:18.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.475+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe26c1f5fd0 con 0x7fe26c155460 2024-09-21T06:28:18.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.966+0000 7fe269600640 1 -- 172.21.15.37:0/212658185 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fe258087a50 con 0x7fe26c155460 2024-09-21T06:28:18.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.968+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 msgr2=0x7fe24c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.968+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 0x7fe24c07b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fa0 tx=0x7fe260005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.968+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 msgr2=0x7fe26c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.968+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe258006010 tx=0x7fe2580035f0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 shutdown_connections 2024-09-21T06:28:18.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe24c078b90 0x7fe24c07b0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe26c155460 0x7fe26c0c9bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe26c162890 0x7fe26c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 --2- 172.21.15.37:0/212658185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe26c0d2760 0x7fe26c1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 >> 172.21.15.37:0/212658185 conn(0x7fe26c0d4360 msgr2=0x7fe26c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 shutdown_connections 2024-09-21T06:28:18.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:18.969+0000 7fe272e00640 1 -- 172.21.15.37:0/212658185 wait complete. 2024-09-21T06:28:18.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:28:18.995 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:28:18.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.264+0000 7fe969e00640 1 -- 172.21.15.37:0/4233983118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe948005150 msgr2=0x7fe94800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.264+0000 7fe969e00640 1 --2- 172.21.15.37:0/4233983118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe948005150 0x7fe94800b9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009f00 tx=0x7fe948005830 comp rx=0 tx=0).stop 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.264+0000 7fe963400640 1 -- 172.21.15.37:0/4233983118 reap_dead start 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.264+0000 7fe969e00640 1 -- 172.21.15.37:0/4233983118 shutdown_connections 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.264+0000 7fe969e00640 1 -- 172.21.15.37:0/4233983118 >> 172.21.15.37:0/4233983118 conn(0x7fe9640d1140 msgr2=0x7fe9640d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.265+0000 7fe969e00640 1 -- 172.21.15.37:0/4233983118 shutdown_connections 2024-09-21T06:28:19.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.265+0000 7fe969e00640 1 -- 172.21.15.37:0/4233983118 wait complete. 2024-09-21T06:28:19.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.267+0000 7fe969e00640 1 Processor -- start 2024-09-21T06:28:19.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.267+0000 7fe969e00640 1 -- start start 2024-09-21T06:28:19.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.268+0000 7fe969e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe969e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe964157650 0x7fe9640b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe969e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9641585e0 0x7fe9641589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe969e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe96406ff00 con 0x7fe9641f8620 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe969e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9640cf3c0 con 0x7fe964157650 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe969e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe964070080 con 0x7fe9641585e0 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42138/0 (socket says 172.21.15.37:42138) 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 -- 172.21.15.37:0/47889110 learned_addr learned my addr 172.21.15.37:0/47889110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe963400640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9641585e0 0x7fe9641589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe95be00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe964157650 0x7fe9640b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 -- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9641585e0 msgr2=0x7fe9641589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9641585e0 0x7fe9641589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 -- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe964157650 msgr2=0x7fe9640b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe964157650 0x7fe9640b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.269+0000 7fe962a00640 1 -- 172.21.15.37:0/47889110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe964158f30 con 0x7fe9641f8620 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe962a00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe954003670 tx=0x7fe954003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954015070 con 0x7fe9641f8620 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe954011900 con 0x7fe9641f8620 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954011d50 con 0x7fe9641f8620 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641f4fe0 con 0x7fe9641f8620 2024-09-21T06:28:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.270+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641f8e00 con 0x7fe9641f8620 2024-09-21T06:28:19.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.271+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96406f6c0 con 0x7fe9641f8620 2024-09-21T06:28:19.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.273+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe95400f430 con 0x7fe9641f8620 2024-09-21T06:28:19.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.274+0000 7fe95b400640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 0x7fe94007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.274+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe95409be10 con 0x7fe9641f8620 2024-09-21T06:28:19.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.274+0000 7fe95be00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 0x7fe94007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.275+0000 7fe95be00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 0x7fe94007b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe96409ee50 tx=0x7fe94c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.278+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe964158f30 con 0x7fe9641f8620 2024-09-21T06:28:19.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.513+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe964074f20 con 0x7fe9641f8620 2024-09-21T06:28:19.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.513+0000 7fe95b400640 1 -- 172.21.15.37:0/47889110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7fe954063fb0 con 0x7fe9641f8620 2024-09-21T06:28:19.516 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:19.516 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:19.516 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:19.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 msgr2=0x7fe94007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 0x7fe94007b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe96409ee50 tx=0x7fe94c005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 msgr2=0x7fe9640c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe954003670 tx=0x7fe954003aa0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 shutdown_connections 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe940078b90 0x7fe94007b0a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9641585e0 0x7fe9641589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe964157650 0x7fe9640b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 --2- 172.21.15.37:0/47889110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9641f8620 0x7fe9640c6c40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 >> 172.21.15.37:0/47889110 conn(0x7fe9640d1140 msgr2=0x7fe9640cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 shutdown_connections 2024-09-21T06:28:19.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.517+0000 7fe969e00640 1 -- 172.21.15.37:0/47889110 wait complete. 2024-09-21T06:28:19.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.779+0000 7f56d8800640 1 -- 172.21.15.37:0/998761987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 msgr2=0x7f56d0073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.779+0000 7f56d8800640 1 --2- 172.21.15.37:0/998761987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d0073dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009830 tx=0x7f56c402a5c0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.779+0000 7f56d8800640 1 -- 172.21.15.37:0/998761987 shutdown_connections 2024-09-21T06:28:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.779+0000 7f56d8800640 1 --2- 172.21.15.37:0/998761987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d0073dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.779+0000 7f56d8800640 1 -- 172.21.15.37:0/998761987 >> 172.21.15.37:0/998761987 conn(0x7f56d00cca40 msgr2=0x7f56d00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.780+0000 7f56d8800640 1 -- 172.21.15.37:0/998761987 shutdown_connections 2024-09-21T06:28:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.780+0000 7f56d8800640 1 -- 172.21.15.37:0/998761987 wait complete. 2024-09-21T06:28:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.782+0000 7f56d8800640 1 Processor -- start 2024-09-21T06:28:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.782+0000 7f56d8800640 1 -- start start 2024-09-21T06:28:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d01e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56d00a0340 0x7f56d00a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56d006f670 con 0x7f56d00cbc10 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56d006f4f0 con 0x7f56d0074cb0 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.783+0000 7f56d8800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f56d006f7f0 con 0x7f56d00a0340 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42144/0 (socket says 172.21.15.37:42144) 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d01e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 -- 172.21.15.37:0/3064960324 learned_addr learned my addr 172.21.15.37:0/3064960324 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d6800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56d00a0340 0x7f56d00a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 -- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56d00a0340 msgr2=0x7f56d00a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56d00a0340 0x7f56d00a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 -- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 msgr2=0x7f56d01e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d01e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.784+0000 7f56d5400640 1 -- 172.21.15.37:0/3064960324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d014d9f0 con 0x7f56d00cbc10 2024-09-21T06:28:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56d5400640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c390 tx=0x7f56c000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0011790 con 0x7f56d00cbc10 2024-09-21T06:28:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56c0017070 con 0x7f56d00cbc10 2024-09-21T06:28:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d00a0c90 con 0x7f56d00cbc10 2024-09-21T06:28:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0011d00 con 0x7f56d00cbc10 2024-09-21T06:28:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.785+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01f8d60 con 0x7f56d00cbc10 2024-09-21T06:28:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.786+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0073dd0 con 0x7f56d00cbc10 2024-09-21T06:28:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.789+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56c001c460 con 0x7f56d00cbc10 2024-09-21T06:28:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.791+0000 7f56ca000640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 0x7f56b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.791+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f56c009ad20 con 0x7f56d00cbc10 2024-09-21T06:28:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.792+0000 7f56d5e00640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 0x7f56b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.793+0000 7f56d5e00640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 0x7f56b407b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56c4034000 tx=0x7f56c4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:19.795+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56d00a0c90 con 0x7f56d00cbc10 2024-09-21T06:28:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.000+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56d006e0f0 con 0x7f56d00cbc10 2024-09-21T06:28:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.001+0000 7f56ca000640 1 -- 172.21.15.37:0/3064960324 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f56d014d9f0 con 0x7f56d00cbc10 2024-09-21T06:28:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:20.004 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 msgr2=0x7f56b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 0x7f56b407b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56c4034000 tx=0x7f56c4004730 comp rx=0 tx=0).stop 2024-09-21T06:28:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 msgr2=0x7f56d014d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c390 tx=0x7f56c000c850 comp rx=0 tx=0).stop 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 shutdown_connections 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56b4078b90 0x7f56b407b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56d00cbc10 0x7f56d014d4b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56d00a0340 0x7f56d00a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 --2- 172.21.15.37:0/3064960324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56d0074cb0 0x7f56d01e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.003+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 >> 172.21.15.37:0/3064960324 conn(0x7f56d00cca40 msgr2=0x7f56d015eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.004+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 shutdown_connections 2024-09-21T06:28:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.004+0000 7f56d8800640 1 -- 172.21.15.37:0/3064960324 wait complete. 2024-09-21T06:28:20.031 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:28:20.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:28:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.258+0000 7fbdf3600640 1 -- 172.21.15.37:0/2454732711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec074cb0 msgr2=0x7fbdec073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.258+0000 7fbdf3600640 1 --2- 172.21.15.37:0/2454732711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec074cb0 0x7fbdec073dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbdec09e110 tx=0x7fbde002a5f0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 -- 172.21.15.37:0/2454732711 shutdown_connections 2024-09-21T06:28:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 --2- 172.21.15.37:0/2454732711 >> v2:172.21.15.150:3300/0 conn(0x7fbdec0cbc10 0x7fbdec0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 --2- 172.21.15.37:0/2454732711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec074cb0 0x7fbdec073dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 -- 172.21.15.37:0/2454732711 >> 172.21.15.37:0/2454732711 conn(0x7fbdec0cca40 msgr2=0x7fbdec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 -- 172.21.15.37:0/2454732711 shutdown_connections 2024-09-21T06:28:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.259+0000 7fbdf3600640 1 -- 172.21.15.37:0/2454732711 wait complete. 2024-09-21T06:28:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.261+0000 7fbdf3600640 1 Processor -- start 2024-09-21T06:28:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.261+0000 7fbdf3600640 1 -- start start 2024-09-21T06:28:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 0x7fbdec1f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdec0cbc10 0x7fbdec1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec06f360 con 0x7fbdec06ddf0 2024-09-21T06:28:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec06f1e0 con 0x7fbdec074cb0 2024-09-21T06:28:20.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf3600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec06f4e0 con 0x7fbdec0cbc10 2024-09-21T06:28:20.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42164/0 (socket says 172.21.15.37:42164) 2024-09-21T06:28:20.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.263+0000 7fbdf0c00640 1 -- 172.21.15.37:0/996884366 learned_addr learned my addr 172.21.15.37:0/996884366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:20.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdebe00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 0x7fbdec1f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf1600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdec0cbc10 0x7fbdec1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 -- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdec0cbc10 msgr2=0x7fbdec1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdec0cbc10 0x7fbdec1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 -- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 msgr2=0x7fbdec1f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 0x7fbdec1f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 -- 172.21.15.37:0/996884366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1f6630 con 0x7fbdec06ddf0 2024-09-21T06:28:20.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdebe00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 0x7fbdec1f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:20.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbdf0c00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034040 tx=0x7fbde0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde003e030 con 0x7fbdec06ddf0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbde0003d10 con 0x7fbdec06ddf0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.264+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0035040 con 0x7fbdec06ddf0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.265+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1f6910 con 0x7fbdec06ddf0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.265+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec0ca0e0 con 0x7fbdec06ddf0 2024-09-21T06:28:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.266+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec0ca370 con 0x7fbdec06ddf0 2024-09-21T06:28:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.267+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbde0041030 con 0x7fbdec06ddf0 2024-09-21T06:28:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.269+0000 7fbde8c00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 0x7fbdcc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.269+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbde00bb7a0 con 0x7fbdec06ddf0 2024-09-21T06:28:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.269+0000 7fbdebe00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 0x7fbdcc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.270+0000 7fbdebe00640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 0x7fbdcc07b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbdec06e670 tx=0x7fbdd8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.273+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbdec0ca370 con 0x7fbdec06ddf0 2024-09-21T06:28:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.468+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdec0cc020 con 0x7fbdcc078bc0 2024-09-21T06:28:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.476+0000 7fbde8c00640 1 -- 172.21.15.37:0/996884366 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332555 (secure 0 0 0) 0x7fbdec0cc020 con 0x7fbdcc078bc0 2024-09-21T06:28:20.483 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.481+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 msgr2=0x7fbdcc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.481+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 0x7fbdcc07b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbdec06e670 tx=0x7fbdd8005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.481+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 msgr2=0x7fbdec0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.481+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034040 tx=0x7fbde0003820 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.481+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 shutdown_connections 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdcc078bc0 0x7fbdcc07b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdec0cbc10 0x7fbdec1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdec074cb0 0x7fbdec1f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 --2- 172.21.15.37:0/996884366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdec06ddf0 0x7fbdec0cf740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 >> 172.21.15.37:0/996884366 conn(0x7fbdec0cca40 msgr2=0x7fbdec15a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 shutdown_connections 2024-09-21T06:28:20.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.482+0000 7fbdf3600640 1 -- 172.21.15.37:0/996884366 wait complete. 2024-09-21T06:28:20.507 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-21T06:28:19.091151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219672,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755048,"statfs":{"total":773094113280,"available":772869169152,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003736"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275298+0000","last_change":"2024-09-21T06:28:00.220046+0000","last_active":"2024-09-21T06:28:06.275298+0000","last_peered":"2024-09-21T06:28:06.275298+0000","last_clean":"2024-09-21T06:28:06.275298+0000","last_became_active":"2024-09-21T06:28:00.219127+0000","last_became_peered":"2024-09-21T06:28:00.219127+0000","last_unstale":"2024-09-21T06:28:06.275298+0000","last_undegraded":"2024-09-21T06:28:06.275298+0000","last_fullsized":"2024-09-21T06:28:06.275298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:49.256177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918731+0000","last_change":"2024-09-21T06:28:01.226620+0000","last_active":"2024-09-21T06:28:06.918731+0000","last_peered":"2024-09-21T06:28:06.918731+0000","last_clean":"2024-09-21T06:28:06.918731+0000","last_became_active":"2024-09-21T06:28:01.224002+0000","last_became_peered":"2024-09-21T06:28:01.224002+0000","last_unstale":"2024-09-21T06:28:06.918731+0000","last_undegraded":"2024-09-21T06:28:06.918731+0000","last_fullsized":"2024-09-21T06:28:06.918731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.430586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276059+0000","last_change":"2024-09-21T06:28:05.289106+0000","last_active":"2024-09-21T06:28:06.276059+0000","last_peered":"2024-09-21T06:28:06.276059+0000","last_clean":"2024-09-21T06:28:06.276059+0000","last_became_active":"2024-09-21T06:28:05.288635+0000","last_became_peered":"2024-09-21T06:28:05.288635+0000","last_unstale":"2024-09-21T06:28:06.276059+0000","last_undegraded":"2024-09-21T06:28:06.276059+0000","last_fullsized":"2024-09-21T06:28:06.276059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:30.614325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918864+0000","last_change":"2024-09-21T06:28:00.216173+0000","last_active":"2024-09-21T06:28:06.918864+0000","last_peered":"2024-09-21T06:28:06.918864+0000","last_clean":"2024-09-21T06:28:06.918864+0000","last_became_active":"2024-09-21T06:28:00.215794+0000","last_became_peered":"2024-09-21T06:28:00.215794+0000","last_unstale":"2024-09-21T06:28:06.918864+0000","last_undegraded":"2024-09-21T06:28:06.918864+0000","last_fullsized":"2024-09-21T06:28:06.918864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:18.915364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918573+0000","last_change":"2024-09-21T06:28:01.238244+0000","last_active":"2024-09-21T06:28:06.918573+0000","last_peered":"2024-09-21T06:28:06.918573+0000","last_clean":"2024-09-21T06:28:06.918573+0000","last_became_active":"2024-09-21T06:28:01.237538+0000","last_became_peered":"2024-09-21T06:28:01.237538+0000","last_unstale":"2024-09-21T06:28:06.918573+0000","last_undegraded":"2024-09-21T06:28:06.918573+0000","last_fullsized":"2024-09-21T06:28:06.918573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:11.476210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271237+0000","last_change":"2024-09-21T06:28:05.267181+0000","last_active":"2024-09-21T06:28:06.271237+0000","last_peered":"2024-09-21T06:28:06.271237+0000","last_clean":"2024-09-21T06:28:06.271237+0000","last_became_active":"2024-09-21T06:28:05.266697+0000","last_became_peered":"2024-09-21T06:28:05.266697+0000","last_unstale":"2024-09-21T06:28:06.271237+0000","last_undegraded":"2024-09-21T06:28:06.271237+0000","last_fullsized":"2024-09-21T06:28:06.271237+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.144842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918246+0000","last_change":"2024-09-21T06:28:00.230877+0000","last_active":"2024-09-21T06:28:06.918246+0000","last_peered":"2024-09-21T06:28:06.918246+0000","last_clean":"2024-09-21T06:28:06.918246+0000","last_became_active":"2024-09-21T06:28:00.230578+0000","last_became_peered":"2024-09-21T06:28:00.230578+0000","last_unstale":"2024-09-21T06:28:06.918246+0000","last_undegraded":"2024-09-21T06:28:06.918246+0000","last_fullsized":"2024-09-21T06:28:06.918246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919137+0000","last_change":"2024-09-21T06:28:01.255720+0000","last_active":"2024-09-21T06:28:06.919137+0000","last_peered":"2024-09-21T06:28:06.919137+0000","last_clean":"2024-09-21T06:28:06.919137+0000","last_became_active":"2024-09-21T06:28:01.254968+0000","last_became_peered":"2024-09-21T06:28:01.254968+0000","last_unstale":"2024-09-21T06:28:06.919137+0000","last_undegraded":"2024-09-21T06:28:06.919137+0000","last_fullsized":"2024-09-21T06:28:06.919137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271315+0000","last_change":"2024-09-21T06:28:05.277963+0000","last_active":"2024-09-21T06:28:06.271315+0000","last_peered":"2024-09-21T06:28:06.271315+0000","last_clean":"2024-09-21T06:28:06.271315+0000","last_became_active":"2024-09-21T06:28:05.277006+0000","last_became_peered":"2024-09-21T06:28:05.277006+0000","last_unstale":"2024-09-21T06:28:06.271315+0000","last_undegraded":"2024-09-21T06:28:06.271315+0000","last_fullsized":"2024-09-21T06:28:06.271315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:35.624735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918079+0000","last_change":"2024-09-21T06:28:00.230976+0000","last_active":"2024-09-21T06:28:06.918079+0000","last_peered":"2024-09-21T06:28:06.918079+0000","last_clean":"2024-09-21T06:28:06.918079+0000","last_became_active":"2024-09-21T06:28:00.230660+0000","last_became_peered":"2024-09-21T06:28:00.230660+0000","last_unstale":"2024-09-21T06:28:06.918079+0000","last_undegraded":"2024-09-21T06:28:06.918079+0000","last_fullsized":"2024-09-21T06:28:06.918079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:59.230778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567463+0000","last_change":"2024-09-21T06:28:01.234299+0000","last_active":"2024-09-21T06:28:06.567463+0000","last_peered":"2024-09-21T06:28:06.567463+0000","last_clean":"2024-09-21T06:28:06.567463+0000","last_became_active":"2024-09-21T06:28:01.233998+0000","last_became_peered":"2024-09-21T06:28:01.233998+0000","last_unstale":"2024-09-21T06:28:06.567463+0000","last_undegraded":"2024-09-21T06:28:06.567463+0000","last_fullsized":"2024-09-21T06:28:06.567463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:08.335780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271460+0000","last_change":"2024-09-21T06:28:05.271851+0000","last_active":"2024-09-21T06:28:06.271460+0000","last_peered":"2024-09-21T06:28:06.271460+0000","last_clean":"2024-09-21T06:28:06.271460+0000","last_became_active":"2024-09-21T06:28:05.265434+0000","last_became_peered":"2024-09-21T06:28:05.265434+0000","last_unstale":"2024-09-21T06:28:06.271460+0000","last_undegraded":"2024-09-21T06:28:06.271460+0000","last_fullsized":"2024-09-21T06:28:06.271460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:39.030315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566989+0000","last_change":"2024-09-21T06:28:01.226065+0000","last_active":"2024-09-21T06:28:06.566989+0000","last_peered":"2024-09-21T06:28:06.566989+0000","last_clean":"2024-09-21T06:28:06.566989+0000","last_became_active":"2024-09-21T06:28:01.225761+0000","last_became_peered":"2024-09-21T06:28:01.225761+0000","last_unstale":"2024-09-21T06:28:06.566989+0000","last_undegraded":"2024-09-21T06:28:06.566989+0000","last_fullsized":"2024-09-21T06:28:06.566989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:02.936599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567107+0000","last_change":"2024-09-21T06:28:00.220356+0000","last_active":"2024-09-21T06:28:06.567107+0000","last_peered":"2024-09-21T06:28:06.567107+0000","last_clean":"2024-09-21T06:28:06.567107+0000","last_became_active":"2024-09-21T06:28:00.219737+0000","last_became_peered":"2024-09-21T06:28:00.219737+0000","last_unstale":"2024-09-21T06:28:06.567107+0000","last_undegraded":"2024-09-21T06:28:06.567107+0000","last_fullsized":"2024-09-21T06:28:06.567107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:58.072689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567253+0000","last_change":"2024-09-21T06:28:05.264666+0000","last_active":"2024-09-21T06:28:06.567253+0000","last_peered":"2024-09-21T06:28:06.567253+0000","last_clean":"2024-09-21T06:28:06.567253+0000","last_became_active":"2024-09-21T06:28:05.264145+0000","last_became_peered":"2024-09-21T06:28:05.264145+0000","last_unstale":"2024-09-21T06:28:06.567253+0000","last_undegraded":"2024-09-21T06:28:06.567253+0000","last_fullsized":"2024-09-21T06:28:06.567253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.384335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276438+0000","last_change":"2024-09-21T06:28:01.238641+0000","last_active":"2024-09-21T06:28:06.276438+0000","last_peered":"2024-09-21T06:28:06.276438+0000","last_clean":"2024-09-21T06:28:06.276438+0000","last_became_active":"2024-09-21T06:28:01.237104+0000","last_became_peered":"2024-09-21T06:28:01.237104+0000","last_unstale":"2024-09-21T06:28:06.276438+0000","last_undegraded":"2024-09-21T06:28:06.276438+0000","last_fullsized":"2024-09-21T06:28:06.276438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919263+0000","last_change":"2024-09-21T06:28:00.222233+0000","last_active":"2024-09-21T06:28:06.919263+0000","last_peered":"2024-09-21T06:28:06.919263+0000","last_clean":"2024-09-21T06:28:06.919263+0000","last_became_active":"2024-09-21T06:28:00.221686+0000","last_became_peered":"2024-09-21T06:28:00.221686+0000","last_unstale":"2024-09-21T06:28:06.919263+0000","last_undegraded":"2024-09-21T06:28:06.919263+0000","last_fullsized":"2024-09-21T06:28:06.919263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:40.731260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917906+0000","last_change":"2024-09-21T06:28:05.275795+0000","last_active":"2024-09-21T06:28:06.917906+0000","last_peered":"2024-09-21T06:28:06.917906+0000","last_clean":"2024-09-21T06:28:06.917906+0000","last_became_active":"2024-09-21T06:28:05.275276+0000","last_became_peered":"2024-09-21T06:28:05.275276+0000","last_unstale":"2024-09-21T06:28:06.917906+0000","last_undegraded":"2024-09-21T06:28:06.917906+0000","last_fullsized":"2024-09-21T06:28:06.917906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:55.953841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273709+0000","last_change":"2024-09-21T06:28:01.231657+0000","last_active":"2024-09-21T06:28:06.273709+0000","last_peered":"2024-09-21T06:28:06.273709+0000","last_clean":"2024-09-21T06:28:06.273709+0000","last_became_active":"2024-09-21T06:28:01.223226+0000","last_became_peered":"2024-09-21T06:28:01.223226+0000","last_unstale":"2024-09-21T06:28:06.273709+0000","last_undegraded":"2024-09-21T06:28:06.273709+0000","last_fullsized":"2024-09-21T06:28:06.273709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274062+0000","last_change":"2024-09-21T06:28:00.227556+0000","last_active":"2024-09-21T06:28:06.274062+0000","last_peered":"2024-09-21T06:28:06.274062+0000","last_clean":"2024-09-21T06:28:06.274062+0000","last_became_active":"2024-09-21T06:28:00.227125+0000","last_became_peered":"2024-09-21T06:28:00.227125+0000","last_unstale":"2024-09-21T06:28:06.274062+0000","last_undegraded":"2024-09-21T06:28:06.274062+0000","last_fullsized":"2024-09-21T06:28:06.274062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:03.936656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277113+0000","last_change":"2024-09-21T06:28:05.283986+0000","last_active":"2024-09-21T06:28:06.277113+0000","last_peered":"2024-09-21T06:28:06.277113+0000","last_clean":"2024-09-21T06:28:06.277113+0000","last_became_active":"2024-09-21T06:28:05.283382+0000","last_became_peered":"2024-09-21T06:28:05.283382+0000","last_unstale":"2024-09-21T06:28:06.277113+0000","last_undegraded":"2024-09-21T06:28:06.277113+0000","last_fullsized":"2024-09-21T06:28:06.277113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.496971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566814+0000","last_change":"2024-09-21T06:28:01.223370+0000","last_active":"2024-09-21T06:28:06.566814+0000","last_peered":"2024-09-21T06:28:06.566814+0000","last_clean":"2024-09-21T06:28:06.566814+0000","last_became_active":"2024-09-21T06:28:01.223018+0000","last_became_peered":"2024-09-21T06:28:01.223018+0000","last_unstale":"2024-09-21T06:28:06.566814+0000","last_undegraded":"2024-09-21T06:28:06.566814+0000","last_fullsized":"2024-09-21T06:28:06.566814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:57.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917258+0000","last_change":"2024-09-21T06:28:00.225692+0000","last_active":"2024-09-21T06:28:06.917258+0000","last_peered":"2024-09-21T06:28:06.917258+0000","last_clean":"2024-09-21T06:28:06.917258+0000","last_became_active":"2024-09-21T06:28:00.224466+0000","last_became_peered":"2024-09-21T06:28:00.224466+0000","last_unstale":"2024-09-21T06:28:06.917258+0000","last_undegraded":"2024-09-21T06:28:06.917258+0000","last_fullsized":"2024-09-21T06:28:06.917258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.485812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271902+0000","last_change":"2024-09-21T06:28:05.282444+0000","last_active":"2024-09-21T06:28:06.271902+0000","last_peered":"2024-09-21T06:28:06.271902+0000","last_clean":"2024-09-21T06:28:06.271902+0000","last_became_active":"2024-09-21T06:28:05.282050+0000","last_became_peered":"2024-09-21T06:28:05.282050+0000","last_unstale":"2024-09-21T06:28:06.271902+0000","last_undegraded":"2024-09-21T06:28:06.271902+0000","last_fullsized":"2024-09-21T06:28:06.271902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274767+0000","last_change":"2024-09-21T06:28:01.225464+0000","last_active":"2024-09-21T06:28:06.274767+0000","last_peered":"2024-09-21T06:28:06.274767+0000","last_clean":"2024-09-21T06:28:06.274767+0000","last_became_active":"2024-09-21T06:28:01.223116+0000","last_became_peered":"2024-09-21T06:28:01.223116+0000","last_unstale":"2024-09-21T06:28:06.274767+0000","last_undegraded":"2024-09-21T06:28:06.274767+0000","last_fullsized":"2024-09-21T06:28:06.274767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567905+0000","last_change":"2024-09-21T06:28:00.230433+0000","last_active":"2024-09-21T06:28:06.567905+0000","last_peered":"2024-09-21T06:28:06.567905+0000","last_clean":"2024-09-21T06:28:06.567905+0000","last_became_active":"2024-09-21T06:28:00.228324+0000","last_became_peered":"2024-09-21T06:28:00.228324+0000","last_unstale":"2024-09-21T06:28:06.567905+0000","last_undegraded":"2024-09-21T06:28:06.567905+0000","last_fullsized":"2024-09-21T06:28:06.567905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:40.653033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271246+0000","last_change":"2024-09-21T06:28:05.280019+0000","last_active":"2024-09-21T06:28:06.271246+0000","last_peered":"2024-09-21T06:28:06.271246+0000","last_clean":"2024-09-21T06:28:06.271246+0000","last_became_active":"2024-09-21T06:28:05.279566+0000","last_became_peered":"2024-09-21T06:28:05.279566+0000","last_unstale":"2024-09-21T06:28:06.271246+0000","last_undegraded":"2024-09-21T06:28:06.271246+0000","last_fullsized":"2024-09-21T06:28:06.271246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:46.736598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275157+0000","last_change":"2024-09-21T06:28:01.227702+0000","last_active":"2024-09-21T06:28:06.275157+0000","last_peered":"2024-09-21T06:28:06.275157+0000","last_clean":"2024-09-21T06:28:06.275157+0000","last_became_active":"2024-09-21T06:28:01.227351+0000","last_became_peered":"2024-09-21T06:28:01.227351+0000","last_unstale":"2024-09-21T06:28:06.275157+0000","last_undegraded":"2024-09-21T06:28:06.275157+0000","last_fullsized":"2024-09-21T06:28:06.275157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275087+0000","last_change":"2024-09-21T06:28:00.227291+0000","last_active":"2024-09-21T06:28:06.275087+0000","last_peered":"2024-09-21T06:28:06.275087+0000","last_clean":"2024-09-21T06:28:06.275087+0000","last_became_active":"2024-09-21T06:28:00.226969+0000","last_became_peered":"2024-09-21T06:28:00.226969+0000","last_unstale":"2024-09-21T06:28:06.275087+0000","last_undegraded":"2024-09-21T06:28:06.275087+0000","last_fullsized":"2024-09-21T06:28:06.275087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:31.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273969+0000","last_change":"2024-09-21T06:28:05.277580+0000","last_active":"2024-09-21T06:28:06.273969+0000","last_peered":"2024-09-21T06:28:06.273969+0000","last_clean":"2024-09-21T06:28:06.273969+0000","last_became_active":"2024-09-21T06:28:05.277134+0000","last_became_peered":"2024-09-21T06:28:05.277134+0000","last_unstale":"2024-09-21T06:28:06.273969+0000","last_undegraded":"2024-09-21T06:28:06.273969+0000","last_fullsized":"2024-09-21T06:28:06.273969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919851+0000","last_change":"2024-09-21T06:28:01.235542+0000","last_active":"2024-09-21T06:28:06.919851+0000","last_peered":"2024-09-21T06:28:06.919851+0000","last_clean":"2024-09-21T06:28:06.919851+0000","last_became_active":"2024-09-21T06:28:01.235235+0000","last_became_peered":"2024-09-21T06:28:01.235235+0000","last_unstale":"2024-09-21T06:28:06.919851+0000","last_undegraded":"2024-09-21T06:28:06.919851+0000","last_fullsized":"2024-09-21T06:28:06.919851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567780+0000","last_change":"2024-09-21T06:28:00.219554+0000","last_active":"2024-09-21T06:28:06.567780+0000","last_peered":"2024-09-21T06:28:06.567780+0000","last_clean":"2024-09-21T06:28:06.567780+0000","last_became_active":"2024-09-21T06:28:00.219180+0000","last_became_peered":"2024-09-21T06:28:00.219180+0000","last_unstale":"2024-09-21T06:28:06.567780+0000","last_undegraded":"2024-09-21T06:28:06.567780+0000","last_fullsized":"2024-09-21T06:28:06.567780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.267700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919430+0000","last_change":"2024-09-21T06:28:05.268962+0000","last_active":"2024-09-21T06:28:06.919430+0000","last_peered":"2024-09-21T06:28:06.919430+0000","last_clean":"2024-09-21T06:28:06.919430+0000","last_became_active":"2024-09-21T06:28:05.268415+0000","last_became_peered":"2024-09-21T06:28:05.268415+0000","last_unstale":"2024-09-21T06:28:06.919430+0000","last_undegraded":"2024-09-21T06:28:06.919430+0000","last_fullsized":"2024-09-21T06:28:06.919430+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.107198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271569+0000","last_change":"2024-09-21T06:28:01.230048+0000","last_active":"2024-09-21T06:28:06.271569+0000","last_peered":"2024-09-21T06:28:06.271569+0000","last_clean":"2024-09-21T06:28:06.271569+0000","last_became_active":"2024-09-21T06:28:01.229747+0000","last_became_peered":"2024-09-21T06:28:01.229747+0000","last_unstale":"2024-09-21T06:28:06.271569+0000","last_undegraded":"2024-09-21T06:28:06.271569+0000","last_fullsized":"2024-09-21T06:28:06.271569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918630+0000","last_change":"2024-09-21T06:28:00.221155+0000","last_active":"2024-09-21T06:28:06.918630+0000","last_peered":"2024-09-21T06:28:06.918630+0000","last_clean":"2024-09-21T06:28:06.918630+0000","last_became_active":"2024-09-21T06:28:00.220728+0000","last_became_peered":"2024-09-21T06:28:00.220728+0000","last_unstale":"2024-09-21T06:28:06.918630+0000","last_undegraded":"2024-09-21T06:28:06.918630+0000","last_fullsized":"2024-09-21T06:28:06.918630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.642076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919099+0000","last_change":"2024-09-21T06:28:05.276248+0000","last_active":"2024-09-21T06:28:06.919099+0000","last_peered":"2024-09-21T06:28:06.919099+0000","last_clean":"2024-09-21T06:28:06.919099+0000","last_became_active":"2024-09-21T06:28:05.271420+0000","last_became_peered":"2024-09-21T06:28:05.271420+0000","last_unstale":"2024-09-21T06:28:06.919099+0000","last_undegraded":"2024-09-21T06:28:06.919099+0000","last_fullsized":"2024-09-21T06:28:06.919099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:03.357365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917672+0000","last_change":"2024-09-21T06:28:01.240349+0000","last_active":"2024-09-21T06:28:06.917672+0000","last_peered":"2024-09-21T06:28:06.917672+0000","last_clean":"2024-09-21T06:28:06.917672+0000","last_became_active":"2024-09-21T06:28:01.239325+0000","last_became_peered":"2024-09-21T06:28:01.239325+0000","last_unstale":"2024-09-21T06:28:06.917672+0000","last_undegraded":"2024-09-21T06:28:06.917672+0000","last_fullsized":"2024-09-21T06:28:06.917672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567402+0000","last_change":"2024-09-21T06:28:00.225175+0000","last_active":"2024-09-21T06:28:06.567402+0000","last_peered":"2024-09-21T06:28:06.567402+0000","last_clean":"2024-09-21T06:28:06.567402+0000","last_became_active":"2024-09-21T06:28:00.224858+0000","last_became_peered":"2024-09-21T06:28:00.224858+0000","last_unstale":"2024-09-21T06:28:06.567402+0000","last_undegraded":"2024-09-21T06:28:06.567402+0000","last_fullsized":"2024-09-21T06:28:06.567402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:41.165675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919693+0000","last_change":"2024-09-21T06:28:05.283982+0000","last_active":"2024-09-21T06:28:06.919693+0000","last_peered":"2024-09-21T06:28:06.919693+0000","last_clean":"2024-09-21T06:28:06.919693+0000","last_became_active":"2024-09-21T06:28:05.283527+0000","last_became_peered":"2024-09-21T06:28:05.283527+0000","last_unstale":"2024-09-21T06:28:06.919693+0000","last_undegraded":"2024-09-21T06:28:06.919693+0000","last_fullsized":"2024-09-21T06:28:06.919693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:17.117103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917636+0000","last_change":"2024-09-21T06:28:01.224161+0000","last_active":"2024-09-21T06:28:06.917636+0000","last_peered":"2024-09-21T06:28:06.917636+0000","last_clean":"2024-09-21T06:28:06.917636+0000","last_became_active":"2024-09-21T06:28:01.223787+0000","last_became_peered":"2024-09-21T06:28:01.223787+0000","last_unstale":"2024-09-21T06:28:06.917636+0000","last_undegraded":"2024-09-21T06:28:06.917636+0000","last_fullsized":"2024-09-21T06:28:06.917636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:03.351441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276110+0000","last_change":"2024-09-21T06:28:00.218460+0000","last_active":"2024-09-21T06:28:06.276110+0000","last_peered":"2024-09-21T06:28:06.276110+0000","last_clean":"2024-09-21T06:28:06.276110+0000","last_became_active":"2024-09-21T06:28:00.218046+0000","last_became_peered":"2024-09-21T06:28:00.218046+0000","last_unstale":"2024-09-21T06:28:06.276110+0000","last_undegraded":"2024-09-21T06:28:06.276110+0000","last_fullsized":"2024-09-21T06:28:06.276110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919620+0000","last_change":"2024-09-21T06:28:05.274088+0000","last_active":"2024-09-21T06:28:06.919620+0000","last_peered":"2024-09-21T06:28:06.919620+0000","last_clean":"2024-09-21T06:28:06.919620+0000","last_became_active":"2024-09-21T06:28:05.273184+0000","last_became_peered":"2024-09-21T06:28:05.273184+0000","last_unstale":"2024-09-21T06:28:06.919620+0000","last_undegraded":"2024-09-21T06:28:06.919620+0000","last_fullsized":"2024-09-21T06:28:06.919620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:17.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918180+0000","last_change":"2024-09-21T06:28:01.230531+0000","last_active":"2024-09-21T06:28:06.918180+0000","last_peered":"2024-09-21T06:28:06.918180+0000","last_clean":"2024-09-21T06:28:06.918180+0000","last_became_active":"2024-09-21T06:28:01.230236+0000","last_became_peered":"2024-09-21T06:28:01.230236+0000","last_unstale":"2024-09-21T06:28:06.918180+0000","last_undegraded":"2024-09-21T06:28:06.918180+0000","last_fullsized":"2024-09-21T06:28:06.918180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.172198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567087+0000","last_change":"2024-09-21T06:28:00.233247+0000","last_active":"2024-09-21T06:28:06.567087+0000","last_peered":"2024-09-21T06:28:06.567087+0000","last_clean":"2024-09-21T06:28:06.567087+0000","last_became_active":"2024-09-21T06:28:00.232905+0000","last_became_peered":"2024-09-21T06:28:00.232905+0000","last_unstale":"2024-09-21T06:28:06.567087+0000","last_undegraded":"2024-09-21T06:28:06.567087+0000","last_fullsized":"2024-09-21T06:28:06.567087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918161+0000","last_change":"2024-09-21T06:28:05.266249+0000","last_active":"2024-09-21T06:28:06.918161+0000","last_peered":"2024-09-21T06:28:06.918161+0000","last_clean":"2024-09-21T06:28:06.918161+0000","last_became_active":"2024-09-21T06:28:05.265779+0000","last_became_peered":"2024-09-21T06:28:05.265779+0000","last_unstale":"2024-09-21T06:28:06.918161+0000","last_undegraded":"2024-09-21T06:28:06.918161+0000","last_fullsized":"2024-09-21T06:28:06.918161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:58.459609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274361+0000","last_change":"2024-09-21T06:28:01.236228+0000","last_active":"2024-09-21T06:28:06.274361+0000","last_peered":"2024-09-21T06:28:06.274361+0000","last_clean":"2024-09-21T06:28:06.274361+0000","last_became_active":"2024-09-21T06:28:01.229874+0000","last_became_peered":"2024-09-21T06:28:01.229874+0000","last_unstale":"2024-09-21T06:28:06.274361+0000","last_undegraded":"2024-09-21T06:28:06.274361+0000","last_fullsized":"2024-09-21T06:28:06.274361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:08.119892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567235+0000","last_change":"2024-09-21T06:28:00.219928+0000","last_active":"2024-09-21T06:28:06.567235+0000","last_peered":"2024-09-21T06:28:06.567235+0000","last_clean":"2024-09-21T06:28:06.567235+0000","last_became_active":"2024-09-21T06:28:00.219561+0000","last_became_peered":"2024-09-21T06:28:00.219561+0000","last_unstale":"2024-09-21T06:28:06.567235+0000","last_undegraded":"2024-09-21T06:28:06.567235+0000","last_fullsized":"2024-09-21T06:28:06.567235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:22.801415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918345+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:28:06.918345+0000","last_peered":"2024-09-21T06:28:06.918345+0000","last_clean":"2024-09-21T06:28:06.918345+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:28:06.918345+0000","last_undegraded":"2024-09-21T06:28:06.918345+0000","last_fullsized":"2024-09-21T06:28:06.918345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567826+0000","last_change":"2024-09-21T06:28:05.274952+0000","last_active":"2024-09-21T06:28:06.567826+0000","last_peered":"2024-09-21T06:28:06.567826+0000","last_clean":"2024-09-21T06:28:06.567826+0000","last_became_active":"2024-09-21T06:28:05.274514+0000","last_became_peered":"2024-09-21T06:28:05.274514+0000","last_unstale":"2024-09-21T06:28:06.567826+0000","last_undegraded":"2024-09-21T06:28:06.567826+0000","last_fullsized":"2024-09-21T06:28:06.567826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:04.636189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567574+0000","last_change":"2024-09-21T06:28:01.224594+0000","last_active":"2024-09-21T06:28:06.567574+0000","last_peered":"2024-09-21T06:28:06.567574+0000","last_clean":"2024-09-21T06:28:06.567574+0000","last_became_active":"2024-09-21T06:28:01.224267+0000","last_became_peered":"2024-09-21T06:28:01.224267+0000","last_unstale":"2024-09-21T06:28:06.567574+0000","last_undegraded":"2024-09-21T06:28:06.567574+0000","last_fullsized":"2024-09-21T06:28:06.567574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:57.968897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919963+0000","last_change":"2024-09-21T06:28:00.226006+0000","last_active":"2024-09-21T06:28:06.919963+0000","last_peered":"2024-09-21T06:28:06.919963+0000","last_clean":"2024-09-21T06:28:06.919963+0000","last_became_active":"2024-09-21T06:28:00.225678+0000","last_became_peered":"2024-09-21T06:28:00.225678+0000","last_unstale":"2024-09-21T06:28:06.919963+0000","last_undegraded":"2024-09-21T06:28:06.919963+0000","last_fullsized":"2024-09-21T06:28:06.919963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:54.460846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271274+0000","last_change":"2024-09-21T06:28:05.280689+0000","last_active":"2024-09-21T06:28:06.271274+0000","last_peered":"2024-09-21T06:28:06.271274+0000","last_clean":"2024-09-21T06:28:06.271274+0000","last_became_active":"2024-09-21T06:28:05.272594+0000","last_became_peered":"2024-09-21T06:28:05.272594+0000","last_unstale":"2024-09-21T06:28:06.271274+0000","last_undegraded":"2024-09-21T06:28:06.271274+0000","last_fullsized":"2024-09-21T06:28:06.271274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:55.258366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917995+0000","last_change":"2024-09-21T06:28:01.230915+0000","last_active":"2024-09-21T06:28:06.917995+0000","last_peered":"2024-09-21T06:28:06.917995+0000","last_clean":"2024-09-21T06:28:06.917995+0000","last_became_active":"2024-09-21T06:28:01.230702+0000","last_became_peered":"2024-09-21T06:28:01.230702+0000","last_unstale":"2024-09-21T06:28:06.917995+0000","last_undegraded":"2024-09-21T06:28:06.917995+0000","last_fullsized":"2024-09-21T06:28:06.917995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:34.107851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918893+0000","last_change":"2024-09-21T06:28:00.214857+0000","last_active":"2024-09-21T06:28:06.918893+0000","last_peered":"2024-09-21T06:28:06.918893+0000","last_clean":"2024-09-21T06:28:06.918893+0000","last_became_active":"2024-09-21T06:28:00.214582+0000","last_became_peered":"2024-09-21T06:28:00.214582+0000","last_unstale":"2024-09-21T06:28:06.918893+0000","last_undegraded":"2024-09-21T06:28:06.918893+0000","last_fullsized":"2024-09-21T06:28:06.918893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:49.842050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918122+0000","last_change":"2024-09-21T06:28:05.273006+0000","last_active":"2024-09-21T06:28:06.918122+0000","last_peered":"2024-09-21T06:28:06.918122+0000","last_clean":"2024-09-21T06:28:06.918122+0000","last_became_active":"2024-09-21T06:28:05.272504+0000","last_became_peered":"2024-09-21T06:28:05.272504+0000","last_unstale":"2024-09-21T06:28:06.918122+0000","last_undegraded":"2024-09-21T06:28:06.918122+0000","last_fullsized":"2024-09-21T06:28:06.918122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:52.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.247357+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.242943+0000","last_became_peered":"2024-09-21T06:28:01.242943+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274916+0000","last_change":"2024-09-21T06:28:00.219919+0000","last_active":"2024-09-21T06:28:06.274916+0000","last_peered":"2024-09-21T06:28:06.274916+0000","last_clean":"2024-09-21T06:28:06.274916+0000","last_became_active":"2024-09-21T06:28:00.219544+0000","last_became_peered":"2024-09-21T06:28:00.219544+0000","last_unstale":"2024-09-21T06:28:06.274916+0000","last_undegraded":"2024-09-21T06:28:06.274916+0000","last_fullsized":"2024-09-21T06:28:06.274916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:21.858722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567045+0000","last_change":"2024-09-21T06:28:05.258119+0000","last_active":"2024-09-21T06:28:06.567045+0000","last_peered":"2024-09-21T06:28:06.567045+0000","last_clean":"2024-09-21T06:28:06.567045+0000","last_became_active":"2024-09-21T06:28:05.257394+0000","last_became_peered":"2024-09-21T06:28:05.257394+0000","last_unstale":"2024-09-21T06:28:06.567045+0000","last_undegraded":"2024-09-21T06:28:06.567045+0000","last_fullsized":"2024-09-21T06:28:06.567045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:13.794574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271877+0000","last_change":"2024-09-21T06:28:01.239234+0000","last_active":"2024-09-21T06:28:06.271877+0000","last_peered":"2024-09-21T06:28:06.271877+0000","last_clean":"2024-09-21T06:28:06.271877+0000","last_became_active":"2024-09-21T06:28:01.238978+0000","last_became_peered":"2024-09-21T06:28:01.238978+0000","last_unstale":"2024-09-21T06:28:06.271877+0000","last_undegraded":"2024-09-21T06:28:06.271877+0000","last_fullsized":"2024-09-21T06:28:06.271877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918626+0000","last_change":"2024-09-21T06:28:00.224968+0000","last_active":"2024-09-21T06:28:06.918626+0000","last_peered":"2024-09-21T06:28:06.918626+0000","last_clean":"2024-09-21T06:28:06.918626+0000","last_became_active":"2024-09-21T06:28:00.224751+0000","last_became_peered":"2024-09-21T06:28:00.224751+0000","last_unstale":"2024-09-21T06:28:06.918626+0000","last_undegraded":"2024-09-21T06:28:06.918626+0000","last_fullsized":"2024-09-21T06:28:06.918626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919013+0000","last_change":"2024-09-21T06:28:05.268891+0000","last_active":"2024-09-21T06:28:06.919013+0000","last_peered":"2024-09-21T06:28:06.919013+0000","last_clean":"2024-09-21T06:28:06.919013+0000","last_became_active":"2024-09-21T06:28:05.268010+0000","last_became_peered":"2024-09-21T06:28:05.268010+0000","last_unstale":"2024-09-21T06:28:06.919013+0000","last_undegraded":"2024-09-21T06:28:06.919013+0000","last_fullsized":"2024-09-21T06:28:06.919013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:04.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566758+0000","last_change":"2024-09-21T06:28:01.225615+0000","last_active":"2024-09-21T06:28:06.566758+0000","last_peered":"2024-09-21T06:28:06.566758+0000","last_clean":"2024-09-21T06:28:06.566758+0000","last_became_active":"2024-09-21T06:28:01.225277+0000","last_became_peered":"2024-09-21T06:28:01.225277+0000","last_unstale":"2024-09-21T06:28:06.566758+0000","last_undegraded":"2024-09-21T06:28:06.566758+0000","last_fullsized":"2024-09-21T06:28:06.566758+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.531813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273831+0000","last_change":"2024-09-21T06:28:00.215632+0000","last_active":"2024-09-21T06:28:06.273831+0000","last_peered":"2024-09-21T06:28:06.273831+0000","last_clean":"2024-09-21T06:28:06.273831+0000","last_became_active":"2024-09-21T06:28:00.215285+0000","last_became_peered":"2024-09-21T06:28:00.215285+0000","last_unstale":"2024-09-21T06:28:06.273831+0000","last_undegraded":"2024-09-21T06:28:06.273831+0000","last_fullsized":"2024-09-21T06:28:06.273831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918621+0000","last_change":"2024-09-21T06:28:05.274989+0000","last_active":"2024-09-21T06:28:06.918621+0000","last_peered":"2024-09-21T06:28:06.918621+0000","last_clean":"2024-09-21T06:28:06.918621+0000","last_became_active":"2024-09-21T06:28:05.274517+0000","last_became_peered":"2024-09-21T06:28:05.274517+0000","last_unstale":"2024-09-21T06:28:06.918621+0000","last_undegraded":"2024-09-21T06:28:06.918621+0000","last_fullsized":"2024-09-21T06:28:06.918621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:09.260032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918268+0000","last_change":"2024-09-21T06:28:01.221685+0000","last_active":"2024-09-21T06:28:06.918268+0000","last_peered":"2024-09-21T06:28:06.918268+0000","last_clean":"2024-09-21T06:28:06.918268+0000","last_became_active":"2024-09-21T06:28:01.221360+0000","last_became_peered":"2024-09-21T06:28:01.221360+0000","last_unstale":"2024-09-21T06:28:06.918268+0000","last_undegraded":"2024-09-21T06:28:06.918268+0000","last_fullsized":"2024-09-21T06:28:06.918268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917445+0000","last_change":"2024-09-21T06:28:00.235459+0000","last_active":"2024-09-21T06:28:06.917445+0000","last_peered":"2024-09-21T06:28:06.917445+0000","last_clean":"2024-09-21T06:28:06.917445+0000","last_became_active":"2024-09-21T06:28:00.235195+0000","last_became_peered":"2024-09-21T06:28:00.235195+0000","last_unstale":"2024-09-21T06:28:06.917445+0000","last_undegraded":"2024-09-21T06:28:06.917445+0000","last_fullsized":"2024-09-21T06:28:06.917445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918627+0000","last_change":"2024-09-21T06:28:05.271996+0000","last_active":"2024-09-21T06:28:06.918627+0000","last_peered":"2024-09-21T06:28:06.918627+0000","last_clean":"2024-09-21T06:28:06.918627+0000","last_became_active":"2024-09-21T06:28:05.270956+0000","last_became_peered":"2024-09-21T06:28:05.270956+0000","last_unstale":"2024-09-21T06:28:06.918627+0000","last_undegraded":"2024-09-21T06:28:06.918627+0000","last_fullsized":"2024-09-21T06:28:06.918627+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:21.181116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566881+0000","last_change":"2024-09-21T06:28:01.237451+0000","last_active":"2024-09-21T06:28:06.566881+0000","last_peered":"2024-09-21T06:28:06.566881+0000","last_clean":"2024-09-21T06:28:06.566881+0000","last_became_active":"2024-09-21T06:28:01.237064+0000","last_became_peered":"2024-09-21T06:28:01.237064+0000","last_unstale":"2024-09-21T06:28:06.566881+0000","last_undegraded":"2024-09-21T06:28:06.566881+0000","last_fullsized":"2024-09-21T06:28:06.566881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567273+0000","last_change":"2024-09-21T06:28:00.224624+0000","last_active":"2024-09-21T06:28:06.567273+0000","last_peered":"2024-09-21T06:28:06.567273+0000","last_clean":"2024-09-21T06:28:06.567273+0000","last_became_active":"2024-09-21T06:28:00.224259+0000","last_became_peered":"2024-09-21T06:28:00.224259+0000","last_unstale":"2024-09-21T06:28:06.567273+0000","last_undegraded":"2024-09-21T06:28:06.567273+0000","last_fullsized":"2024-09-21T06:28:06.567273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919872+0000","last_change":"2024-09-21T06:28:05.283392+0000","last_active":"2024-09-21T06:28:06.919872+0000","last_peered":"2024-09-21T06:28:06.919872+0000","last_clean":"2024-09-21T06:28:06.919872+0000","last_became_active":"2024-09-21T06:28:05.271401+0000","last_became_peered":"2024-09-21T06:28:05.271401+0000","last_unstale":"2024-09-21T06:28:06.919872+0000","last_undegraded":"2024-09-21T06:28:06.919872+0000","last_fullsized":"2024-09-21T06:28:06.919872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:21.192646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919152+0000","last_change":"2024-09-21T06:28:01.224123+0000","last_active":"2024-09-21T06:28:06.919152+0000","last_peered":"2024-09-21T06:28:06.919152+0000","last_clean":"2024-09-21T06:28:06.919152+0000","last_became_active":"2024-09-21T06:28:01.223773+0000","last_became_peered":"2024-09-21T06:28:01.223773+0000","last_unstale":"2024-09-21T06:28:06.919152+0000","last_undegraded":"2024-09-21T06:28:06.919152+0000","last_fullsized":"2024-09-21T06:28:06.919152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:59.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276563+0000","last_change":"2024-09-21T06:28:00.232406+0000","last_active":"2024-09-21T06:28:06.276563+0000","last_peered":"2024-09-21T06:28:06.276563+0000","last_clean":"2024-09-21T06:28:06.276563+0000","last_became_active":"2024-09-21T06:28:00.232143+0000","last_became_peered":"2024-09-21T06:28:00.232143+0000","last_unstale":"2024-09-21T06:28:06.276563+0000","last_undegraded":"2024-09-21T06:28:06.276563+0000","last_fullsized":"2024-09-21T06:28:06.276563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:33.343595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567590+0000","last_change":"2024-09-21T06:28:05.274346+0000","last_active":"2024-09-21T06:28:06.567590+0000","last_peered":"2024-09-21T06:28:06.567590+0000","last_clean":"2024-09-21T06:28:06.567590+0000","last_became_active":"2024-09-21T06:28:05.273714+0000","last_became_peered":"2024-09-21T06:28:05.273714+0000","last_unstale":"2024-09-21T06:28:06.567590+0000","last_undegraded":"2024-09-21T06:28:06.567590+0000","last_fullsized":"2024-09-21T06:28:06.567590+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:09.113671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918127+0000","last_change":"2024-09-21T06:28:01.238637+0000","last_active":"2024-09-21T06:28:06.918127+0000","last_peered":"2024-09-21T06:28:06.918127+0000","last_clean":"2024-09-21T06:28:06.918127+0000","last_became_active":"2024-09-21T06:28:01.236539+0000","last_became_peered":"2024-09-21T06:28:01.236539+0000","last_unstale":"2024-09-21T06:28:06.918127+0000","last_undegraded":"2024-09-21T06:28:06.918127+0000","last_fullsized":"2024-09-21T06:28:06.918127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.692846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:14.350747+0000","last_change":"2024-09-21T06:28:00.217119+0000","last_active":"2024-09-21T06:28:14.350747+0000","last_peered":"2024-09-21T06:28:14.350747+0000","last_clean":"2024-09-21T06:28:14.350747+0000","last_became_active":"2024-09-21T06:28:00.216806+0000","last_became_peered":"2024-09-21T06:28:00.216806+0000","last_unstale":"2024-09-21T06:28:14.350747+0000","last_undegraded":"2024-09-21T06:28:14.350747+0000","last_fullsized":"2024-09-21T06:28:14.350747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:17.032531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274402+0000","last_change":"2024-09-21T06:28:05.273619+0000","last_active":"2024-09-21T06:28:06.274402+0000","last_peered":"2024-09-21T06:28:06.274402+0000","last_clean":"2024-09-21T06:28:06.274402+0000","last_became_active":"2024-09-21T06:28:05.273193+0000","last_became_peered":"2024-09-21T06:28:05.273193+0000","last_unstale":"2024-09-21T06:28:06.274402+0000","last_undegraded":"2024-09-21T06:28:06.274402+0000","last_fullsized":"2024-09-21T06:28:06.274402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:31.630020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567019+0000","last_change":"2024-09-21T06:28:01.233057+0000","last_active":"2024-09-21T06:28:06.567019+0000","last_peered":"2024-09-21T06:28:06.567019+0000","last_clean":"2024-09-21T06:28:06.567019+0000","last_became_active":"2024-09-21T06:28:01.232602+0000","last_became_peered":"2024-09-21T06:28:01.232602+0000","last_unstale":"2024-09-21T06:28:06.567019+0000","last_undegraded":"2024-09-21T06:28:06.567019+0000","last_fullsized":"2024-09-21T06:28:06.567019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:34.157716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276884+0000","last_change":"2024-09-21T06:28:00.218229+0000","last_active":"2024-09-21T06:28:06.276884+0000","last_peered":"2024-09-21T06:28:06.276884+0000","last_clean":"2024-09-21T06:28:06.276884+0000","last_became_active":"2024-09-21T06:28:00.217635+0000","last_became_peered":"2024-09-21T06:28:00.217635+0000","last_unstale":"2024-09-21T06:28:06.276884+0000","last_undegraded":"2024-09-21T06:28:06.276884+0000","last_fullsized":"2024-09-21T06:28:06.276884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271664+0000","last_change":"2024-09-21T06:28:05.277550+0000","last_active":"2024-09-21T06:28:06.271664+0000","last_peered":"2024-09-21T06:28:06.271664+0000","last_clean":"2024-09-21T06:28:06.271664+0000","last_became_active":"2024-09-21T06:28:05.276077+0000","last_became_peered":"2024-09-21T06:28:05.276077+0000","last_unstale":"2024-09-21T06:28:06.271664+0000","last_undegraded":"2024-09-21T06:28:06.271664+0000","last_fullsized":"2024-09-21T06:28:06.271664+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.300870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918857+0000","last_change":"2024-09-21T06:28:01.232773+0000","last_active":"2024-09-21T06:28:06.918857+0000","last_peered":"2024-09-21T06:28:06.918857+0000","last_clean":"2024-09-21T06:28:06.918857+0000","last_became_active":"2024-09-21T06:28:01.232081+0000","last_became_peered":"2024-09-21T06:28:01.232081+0000","last_unstale":"2024-09-21T06:28:06.918857+0000","last_undegraded":"2024-09-21T06:28:06.918857+0000","last_fullsized":"2024-09-21T06:28:06.918857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:54.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917778+0000","last_change":"2024-09-21T06:28:00.234206+0000","last_active":"2024-09-21T06:28:06.917778+0000","last_peered":"2024-09-21T06:28:06.917778+0000","last_clean":"2024-09-21T06:28:06.917778+0000","last_became_active":"2024-09-21T06:28:00.233696+0000","last_became_peered":"2024-09-21T06:28:00.233696+0000","last_unstale":"2024-09-21T06:28:06.917778+0000","last_undegraded":"2024-09-21T06:28:06.917778+0000","last_fullsized":"2024-09-21T06:28:06.917778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275902+0000","last_change":"2024-09-21T06:28:05.283000+0000","last_active":"2024-09-21T06:28:06.275902+0000","last_peered":"2024-09-21T06:28:06.275902+0000","last_clean":"2024-09-21T06:28:06.275902+0000","last_became_active":"2024-09-21T06:28:05.282638+0000","last_became_peered":"2024-09-21T06:28:05.282638+0000","last_unstale":"2024-09-21T06:28:06.275902+0000","last_undegraded":"2024-09-21T06:28:06.275902+0000","last_fullsized":"2024-09-21T06:28:06.275902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271814+0000","last_change":"2024-09-21T06:28:01.221987+0000","last_active":"2024-09-21T06:28:06.271814+0000","last_peered":"2024-09-21T06:28:06.271814+0000","last_clean":"2024-09-21T06:28:06.271814+0000","last_became_active":"2024-09-21T06:28:01.221678+0000","last_became_peered":"2024-09-21T06:28:01.221678+0000","last_unstale":"2024-09-21T06:28:06.271814+0000","last_undegraded":"2024-09-21T06:28:06.271814+0000","last_fullsized":"2024-09-21T06:28:06.271814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:20.730550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919718+0000","last_change":"2024-09-21T06:28:00.202531+0000","last_active":"2024-09-21T06:28:06.919718+0000","last_peered":"2024-09-21T06:28:06.919718+0000","last_clean":"2024-09-21T06:28:06.919718+0000","last_became_active":"2024-09-21T06:28:00.202126+0000","last_became_peered":"2024-09-21T06:28:00.202126+0000","last_unstale":"2024-09-21T06:28:06.919718+0000","last_undegraded":"2024-09-21T06:28:06.919718+0000","last_fullsized":"2024-09-21T06:28:06.919718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274058+0000","last_change":"2024-09-21T06:28:05.278157+0000","last_active":"2024-09-21T06:28:06.274058+0000","last_peered":"2024-09-21T06:28:06.274058+0000","last_clean":"2024-09-21T06:28:06.274058+0000","last_became_active":"2024-09-21T06:28:05.277532+0000","last_became_peered":"2024-09-21T06:28:05.277532+0000","last_unstale":"2024-09-21T06:28:06.274058+0000","last_undegraded":"2024-09-21T06:28:06.274058+0000","last_fullsized":"2024-09-21T06:28:06.274058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.737605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274991+0000","last_change":"2024-09-21T06:28:05.278200+0000","last_active":"2024-09-21T06:28:06.274991+0000","last_peered":"2024-09-21T06:28:06.274991+0000","last_clean":"2024-09-21T06:28:06.274991+0000","last_became_active":"2024-09-21T06:28:05.277575+0000","last_became_peered":"2024-09-21T06:28:05.277575+0000","last_unstale":"2024-09-21T06:28:06.274991+0000","last_undegraded":"2024-09-21T06:28:06.274991+0000","last_fullsized":"2024-09-21T06:28:06.274991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:05.311819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273908+0000","last_change":"2024-09-21T06:28:01.228981+0000","last_active":"2024-09-21T06:28:06.273908+0000","last_peered":"2024-09-21T06:28:06.273908+0000","last_clean":"2024-09-21T06:28:06.273908+0000","last_became_active":"2024-09-21T06:28:01.222592+0000","last_became_peered":"2024-09-21T06:28:01.222592+0000","last_unstale":"2024-09-21T06:28:06.273908+0000","last_undegraded":"2024-09-21T06:28:06.273908+0000","last_fullsized":"2024-09-21T06:28:06.273908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566966+0000","last_change":"2024-09-21T06:28:00.214521+0000","last_active":"2024-09-21T06:28:06.566966+0000","last_peered":"2024-09-21T06:28:06.566966+0000","last_clean":"2024-09-21T06:28:06.566966+0000","last_became_active":"2024-09-21T06:28:00.212408+0000","last_became_peered":"2024-09-21T06:28:00.212408+0000","last_unstale":"2024-09-21T06:28:06.566966+0000","last_undegraded":"2024-09-21T06:28:06.566966+0000","last_fullsized":"2024-09-21T06:28:06.566966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567614+0000","last_change":"2024-09-21T06:28:05.265492+0000","last_active":"2024-09-21T06:28:06.567614+0000","last_peered":"2024-09-21T06:28:06.567614+0000","last_clean":"2024-09-21T06:28:06.567614+0000","last_became_active":"2024-09-21T06:28:05.264988+0000","last_became_peered":"2024-09-21T06:28:05.264988+0000","last_unstale":"2024-09-21T06:28:06.567614+0000","last_undegraded":"2024-09-21T06:28:06.567614+0000","last_fullsized":"2024-09-21T06:28:06.567614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:02.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917735+0000","last_change":"2024-09-21T06:28:01.228392+0000","last_active":"2024-09-21T06:28:06.917735+0000","last_peered":"2024-09-21T06:28:06.917735+0000","last_clean":"2024-09-21T06:28:06.917735+0000","last_became_active":"2024-09-21T06:28:01.228163+0000","last_became_peered":"2024-09-21T06:28:01.228163+0000","last_unstale":"2024-09-21T06:28:06.917735+0000","last_undegraded":"2024-09-21T06:28:06.917735+0000","last_fullsized":"2024-09-21T06:28:06.917735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:24.791034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918738+0000","last_change":"2024-09-21T06:28:00.228862+0000","last_active":"2024-09-21T06:28:06.918738+0000","last_peered":"2024-09-21T06:28:06.918738+0000","last_clean":"2024-09-21T06:28:06.918738+0000","last_became_active":"2024-09-21T06:28:00.228490+0000","last_became_peered":"2024-09-21T06:28:00.228490+0000","last_unstale":"2024-09-21T06:28:06.918738+0000","last_undegraded":"2024-09-21T06:28:06.918738+0000","last_fullsized":"2024-09-21T06:28:06.918738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:09:00.810925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567138+0000","last_change":"2024-09-21T06:28:00.234315+0000","last_active":"2024-09-21T06:28:06.567138+0000","last_peered":"2024-09-21T06:28:06.567138+0000","last_clean":"2024-09-21T06:28:06.567138+0000","last_became_active":"2024-09-21T06:28:00.233567+0000","last_became_peered":"2024-09-21T06:28:00.233567+0000","last_unstale":"2024-09-21T06:28:06.567138+0000","last_undegraded":"2024-09-21T06:28:06.567138+0000","last_fullsized":"2024-09-21T06:28:06.567138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274635+0000","last_change":"2024-09-21T06:28:01.236968+0000","last_active":"2024-09-21T06:28:06.274635+0000","last_peered":"2024-09-21T06:28:06.274635+0000","last_clean":"2024-09-21T06:28:06.274635+0000","last_became_active":"2024-09-21T06:28:01.236577+0000","last_became_peered":"2024-09-21T06:28:01.236577+0000","last_unstale":"2024-09-21T06:28:06.274635+0000","last_undegraded":"2024-09-21T06:28:06.274635+0000","last_fullsized":"2024-09-21T06:28:06.274635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:01.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274854+0000","last_change":"2024-09-21T06:28:05.271567+0000","last_active":"2024-09-21T06:28:06.274854+0000","last_peered":"2024-09-21T06:28:06.274854+0000","last_clean":"2024-09-21T06:28:06.274854+0000","last_became_active":"2024-09-21T06:28:05.271160+0000","last_became_peered":"2024-09-21T06:28:05.271160+0000","last_unstale":"2024-09-21T06:28:06.274854+0000","last_undegraded":"2024-09-21T06:28:06.274854+0000","last_fullsized":"2024-09-21T06:28:06.274854+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:02.495413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918355+0000","last_change":"2024-09-21T06:28:00.215612+0000","last_active":"2024-09-21T06:28:06.918355+0000","last_peered":"2024-09-21T06:28:06.918355+0000","last_clean":"2024-09-21T06:28:06.918355+0000","last_became_active":"2024-09-21T06:28:00.214929+0000","last_became_peered":"2024-09-21T06:28:00.214929+0000","last_unstale":"2024-09-21T06:28:06.918355+0000","last_undegraded":"2024-09-21T06:28:06.918355+0000","last_fullsized":"2024-09-21T06:28:06.918355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:41.140041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918598+0000","last_change":"2024-09-21T06:28:01.234866+0000","last_active":"2024-09-21T06:28:06.918598+0000","last_peered":"2024-09-21T06:28:06.918598+0000","last_clean":"2024-09-21T06:28:06.918598+0000","last_became_active":"2024-09-21T06:28:01.234568+0000","last_became_peered":"2024-09-21T06:28:01.234568+0000","last_unstale":"2024-09-21T06:28:06.918598+0000","last_undegraded":"2024-09-21T06:28:06.918598+0000","last_fullsized":"2024-09-21T06:28:06.918598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:00.807517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276818+0000","last_change":"2024-09-21T06:28:05.287452+0000","last_active":"2024-09-21T06:28:06.276818+0000","last_peered":"2024-09-21T06:28:06.276818+0000","last_clean":"2024-09-21T06:28:06.276818+0000","last_became_active":"2024-09-21T06:28:05.286963+0000","last_became_peered":"2024-09-21T06:28:05.286963+0000","last_unstale":"2024-09-21T06:28:06.276818+0000","last_undegraded":"2024-09-21T06:28:06.276818+0000","last_fullsized":"2024-09-21T06:28:06.276818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:03.321463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271084+0000","last_change":"2024-09-21T06:28:01.232705+0000","last_active":"2024-09-21T06:28:06.271084+0000","last_peered":"2024-09-21T06:28:06.271084+0000","last_clean":"2024-09-21T06:28:06.271084+0000","last_became_active":"2024-09-21T06:28:01.232305+0000","last_became_peered":"2024-09-21T06:28:01.232305+0000","last_unstale":"2024-09-21T06:28:06.271084+0000","last_undegraded":"2024-09-21T06:28:06.271084+0000","last_fullsized":"2024-09-21T06:28:06.271084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275922+0000","last_change":"2024-09-21T06:28:01.235529+0000","last_active":"2024-09-21T06:28:06.275922+0000","last_peered":"2024-09-21T06:28:06.275922+0000","last_clean":"2024-09-21T06:28:06.275922+0000","last_became_active":"2024-09-21T06:28:01.235170+0000","last_became_peered":"2024-09-21T06:28:01.235170+0000","last_unstale":"2024-09-21T06:28:06.275922+0000","last_undegraded":"2024-09-21T06:28:06.275922+0000","last_fullsized":"2024-09-21T06:28:06.275922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566964+0000","last_change":"2024-09-21T06:28:01.242450+0000","last_active":"2024-09-21T06:28:06.566964+0000","last_peered":"2024-09-21T06:28:06.566964+0000","last_clean":"2024-09-21T06:28:06.566964+0000","last_became_active":"2024-09-21T06:28:01.242078+0000","last_became_peered":"2024-09-21T06:28:01.242078+0000","last_unstale":"2024-09-21T06:28:06.566964+0000","last_undegraded":"2024-09-21T06:28:06.566964+0000","last_fullsized":"2024-09-21T06:28:06.566964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:58.450653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918263+0000","last_change":"2024-09-21T06:28:01.230631+0000","last_active":"2024-09-21T06:28:06.918263+0000","last_peered":"2024-09-21T06:28:06.918263+0000","last_clean":"2024-09-21T06:28:06.918263+0000","last_became_active":"2024-09-21T06:28:01.223844+0000","last_became_peered":"2024-09-21T06:28:01.223844+0000","last_unstale":"2024-09-21T06:28:06.918263+0000","last_undegraded":"2024-09-21T06:28:06.918263+0000","last_fullsized":"2024-09-21T06:28:06.918263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:28.649005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919766+0000","last_change":"2024-09-21T06:28:01.248984+0000","last_active":"2024-09-21T06:28:06.919766+0000","last_peered":"2024-09-21T06:28:06.919766+0000","last_clean":"2024-09-21T06:28:06.919766+0000","last_became_active":"2024-09-21T06:28:01.248639+0000","last_became_peered":"2024-09-21T06:28:01.248639+0000","last_unstale":"2024-09-21T06:28:06.919766+0000","last_undegraded":"2024-09-21T06:28:06.919766+0000","last_fullsized":"2024-09-21T06:28:06.919766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.659556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919689+0000","last_change":"2024-09-21T06:28:01.226847+0000","last_active":"2024-09-21T06:28:06.919689+0000","last_peered":"2024-09-21T06:28:06.919689+0000","last_clean":"2024-09-21T06:28:06.919689+0000","last_became_active":"2024-09-21T06:28:01.226536+0000","last_became_peered":"2024-09-21T06:28:01.226536+0000","last_unstale":"2024-09-21T06:28:06.919689+0000","last_undegraded":"2024-09-21T06:28:06.919689+0000","last_fullsized":"2024-09-21T06:28:06.919689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271732+0000","last_change":"2024-09-21T06:28:01.223972+0000","last_active":"2024-09-21T06:28:06.271732+0000","last_peered":"2024-09-21T06:28:06.271732+0000","last_clean":"2024-09-21T06:28:06.271732+0000","last_became_active":"2024-09-21T06:28:01.223601+0000","last_became_peered":"2024-09-21T06:28:01.223601+0000","last_unstale":"2024-09-21T06:28:06.271732+0000","last_undegraded":"2024-09-21T06:28:06.271732+0000","last_fullsized":"2024-09-21T06:28:06.271732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918404+0000","last_change":"2024-09-21T06:28:01.233118+0000","last_active":"2024-09-21T06:28:06.918404+0000","last_peered":"2024-09-21T06:28:06.918404+0000","last_clean":"2024-09-21T06:28:06.918404+0000","last_became_active":"2024-09-21T06:28:01.232482+0000","last_became_peered":"2024-09-21T06:28:01.232482+0000","last_unstale":"2024-09-21T06:28:06.918404+0000","last_undegraded":"2024-09-21T06:28:06.918404+0000","last_fullsized":"2024-09-21T06:28:06.918404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:39.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566851+0000","last_change":"2024-09-21T06:28:01.243008+0000","last_active":"2024-09-21T06:28:06.566851+0000","last_peered":"2024-09-21T06:28:06.566851+0000","last_clean":"2024-09-21T06:28:06.566851+0000","last_became_active":"2024-09-21T06:28:01.242713+0000","last_became_peered":"2024-09-21T06:28:01.242713+0000","last_unstale":"2024-09-21T06:28:06.566851+0000","last_undegraded":"2024-09-21T06:28:06.566851+0000","last_fullsized":"2024-09-21T06:28:06.566851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:17.147886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917756+0000","last_change":"2024-09-21T06:28:01.216205+0000","last_active":"2024-09-21T06:28:06.917756+0000","last_peered":"2024-09-21T06:28:06.917756+0000","last_clean":"2024-09-21T06:28:06.917756+0000","last_became_active":"2024-09-21T06:28:01.215869+0000","last_became_peered":"2024-09-21T06:28:01.215869+0000","last_unstale":"2024-09-21T06:28:06.917756+0000","last_undegraded":"2024-09-21T06:28:06.917756+0000","last_fullsized":"2024-09-21T06:28:06.917756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.725175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.246071+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.245739+0000","last_became_peered":"2024-09-21T06:28:01.245739+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274517+0000","last_change":"2024-09-21T06:28:01.232435+0000","last_active":"2024-09-21T06:28:06.274517+0000","last_peered":"2024-09-21T06:28:06.274517+0000","last_clean":"2024-09-21T06:28:06.274517+0000","last_became_active":"2024-09-21T06:28:01.229739+0000","last_became_peered":"2024-09-21T06:28:01.229739+0000","last_unstale":"2024-09-21T06:28:06.274517+0000","last_undegraded":"2024-09-21T06:28:06.274517+0000","last_fullsized":"2024-09-21T06:28:06.274517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:59.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917974+0000","last_change":"2024-09-21T06:28:01.242252+0000","last_active":"2024-09-21T06:28:06.917974+0000","last_peered":"2024-09-21T06:28:06.917974+0000","last_clean":"2024-09-21T06:28:06.917974+0000","last_became_active":"2024-09-21T06:28:01.241926+0000","last_became_peered":"2024-09-21T06:28:01.241926+0000","last_unstale":"2024-09-21T06:28:06.917974+0000","last_undegraded":"2024-09-21T06:28:06.917974+0000","last_fullsized":"2024-09-21T06:28:06.917974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:31.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567677+0000","last_change":"2024-09-21T06:28:01.232426+0000","last_active":"2024-09-21T06:28:06.567677+0000","last_peered":"2024-09-21T06:28:06.567677+0000","last_clean":"2024-09-21T06:28:06.567677+0000","last_became_active":"2024-09-21T06:28:01.232067+0000","last_became_peered":"2024-09-21T06:28:01.232067+0000","last_unstale":"2024-09-21T06:28:06.567677+0000","last_undegraded":"2024-09-21T06:28:06.567677+0000","last_fullsized":"2024-09-21T06:28:06.567677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:28.274577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277142+0000","last_change":"2024-09-21T06:28:01.247978+0000","last_active":"2024-09-21T06:28:06.277142+0000","last_peered":"2024-09-21T06:28:06.277142+0000","last_clean":"2024-09-21T06:28:06.277142+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.277142+0000","last_undegraded":"2024-09-21T06:28:06.277142+0000","last_fullsized":"2024-09-21T06:28:06.277142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567819+0000","last_change":"2024-09-21T06:28:01.236566+0000","last_active":"2024-09-21T06:28:06.567819+0000","last_peered":"2024-09-21T06:28:06.567819+0000","last_clean":"2024-09-21T06:28:06.567819+0000","last_became_active":"2024-09-21T06:28:01.236209+0000","last_became_peered":"2024-09-21T06:28:01.236209+0000","last_unstale":"2024-09-21T06:28:06.567819+0000","last_undegraded":"2024-09-21T06:28:06.567819+0000","last_fullsized":"2024-09-21T06:28:06.567819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:55.410838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566732+0000","last_change":"2024-09-21T06:28:01.227863+0000","last_active":"2024-09-21T06:28:06.566732+0000","last_peered":"2024-09-21T06:28:06.566732+0000","last_clean":"2024-09-21T06:28:06.566732+0000","last_became_active":"2024-09-21T06:28:01.227553+0000","last_became_peered":"2024-09-21T06:28:01.227553+0000","last_unstale":"2024-09-21T06:28:06.566732+0000","last_undegraded":"2024-09-21T06:28:06.566732+0000","last_fullsized":"2024-09-21T06:28:06.566732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:51.034026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917669+0000","last_change":"2024-09-21T06:28:01.242950+0000","last_active":"2024-09-21T06:28:06.917669+0000","last_peered":"2024-09-21T06:28:06.917669+0000","last_clean":"2024-09-21T06:28:06.917669+0000","last_became_active":"2024-09-21T06:28:01.242715+0000","last_became_peered":"2024-09-21T06:28:01.242715+0000","last_unstale":"2024-09-21T06:28:06.917669+0000","last_undegraded":"2024-09-21T06:28:06.917669+0000","last_fullsized":"2024-09-21T06:28:06.917669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:12.827134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919061+0000","last_change":"2024-09-21T06:28:01.231009+0000","last_active":"2024-09-21T06:28:06.919061+0000","last_peered":"2024-09-21T06:28:06.919061+0000","last_clean":"2024-09-21T06:28:06.919061+0000","last_became_active":"2024-09-21T06:28:01.230706+0000","last_became_peered":"2024-09-21T06:28:01.230706+0000","last_unstale":"2024-09-21T06:28:06.919061+0000","last_undegraded":"2024-09-21T06:28:06.919061+0000","last_fullsized":"2024-09-21T06:28:06.919061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:38.274224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919349+0000","last_change":"2024-09-21T06:28:01.235786+0000","last_active":"2024-09-21T06:28:06.919349+0000","last_peered":"2024-09-21T06:28:06.919349+0000","last_clean":"2024-09-21T06:28:06.919349+0000","last_became_active":"2024-09-21T06:28:01.235551+0000","last_became_peered":"2024-09-21T06:28:01.235551+0000","last_unstale":"2024-09-21T06:28:06.919349+0000","last_undegraded":"2024-09-21T06:28:06.919349+0000","last_fullsized":"2024-09-21T06:28:06.919349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:00.237301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917934+0000","last_change":"2024-09-21T06:28:01.242543+0000","last_active":"2024-09-21T06:28:06.917934+0000","last_peered":"2024-09-21T06:28:06.917934+0000","last_clean":"2024-09-21T06:28:06.917934+0000","last_became_active":"2024-09-21T06:28:01.242225+0000","last_became_peered":"2024-09-21T06:28:01.242225+0000","last_unstale":"2024-09-21T06:28:06.917934+0000","last_undegraded":"2024-09-21T06:28:06.917934+0000","last_fullsized":"2024-09-21T06:28:06.917934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:39.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918078+0000","last_change":"2024-09-21T06:28:01.237214+0000","last_active":"2024-09-21T06:28:06.918078+0000","last_peered":"2024-09-21T06:28:06.918078+0000","last_clean":"2024-09-21T06:28:06.918078+0000","last_became_active":"2024-09-21T06:28:01.235938+0000","last_became_peered":"2024-09-21T06:28:01.235938+0000","last_unstale":"2024-09-21T06:28:06.918078+0000","last_undegraded":"2024-09-21T06:28:06.918078+0000","last_fullsized":"2024-09-21T06:28:06.918078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.564737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917663+0000","last_change":"2024-09-21T06:28:01.251436+0000","last_active":"2024-09-21T06:28:06.917663+0000","last_peered":"2024-09-21T06:28:06.917663+0000","last_clean":"2024-09-21T06:28:06.917663+0000","last_became_active":"2024-09-21T06:28:01.250562+0000","last_became_peered":"2024-09-21T06:28:01.250562+0000","last_unstale":"2024-09-21T06:28:06.917663+0000","last_undegraded":"2024-09-21T06:28:06.917663+0000","last_fullsized":"2024-09-21T06:28:06.917663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:58.692504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919694+0000","last_change":"2024-09-21T06:28:01.226763+0000","last_active":"2024-09-21T06:28:06.919694+0000","last_peered":"2024-09-21T06:28:06.919694+0000","last_clean":"2024-09-21T06:28:06.919694+0000","last_became_active":"2024-09-21T06:28:01.226438+0000","last_became_peered":"2024-09-21T06:28:01.226438+0000","last_unstale":"2024-09-21T06:28:06.919694+0000","last_undegraded":"2024-09-21T06:28:06.919694+0000","last_fullsized":"2024-09-21T06:28:06.919694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917570+0000","last_change":"2024-09-21T06:28:01.242596+0000","last_active":"2024-09-21T06:28:06.917570+0000","last_peered":"2024-09-21T06:28:06.917570+0000","last_clean":"2024-09-21T06:28:06.917570+0000","last_became_active":"2024-09-21T06:28:01.242205+0000","last_became_peered":"2024-09-21T06:28:01.242205+0000","last_unstale":"2024-09-21T06:28:06.917570+0000","last_undegraded":"2024-09-21T06:28:06.917570+0000","last_fullsized":"2024-09-21T06:28:06.917570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:44.145821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917382+0000","last_change":"2024-09-21T06:28:01.249034+0000","last_active":"2024-09-21T06:28:06.917382+0000","last_peered":"2024-09-21T06:28:06.917382+0000","last_clean":"2024-09-21T06:28:06.917382+0000","last_became_active":"2024-09-21T06:28:01.248727+0000","last_became_peered":"2024-09-21T06:28:01.248727+0000","last_unstale":"2024-09-21T06:28:06.917382+0000","last_undegraded":"2024-09-21T06:28:06.917382+0000","last_fullsized":"2024-09-21T06:28:06.917382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271467+0000","last_change":"2024-09-21T06:28:01.215764+0000","last_active":"2024-09-21T06:28:06.271467+0000","last_peered":"2024-09-21T06:28:06.271467+0000","last_clean":"2024-09-21T06:28:06.271467+0000","last_became_active":"2024-09-21T06:28:01.215412+0000","last_became_peered":"2024-09-21T06:28:01.215412+0000","last_unstale":"2024-09-21T06:28:06.271467+0000","last_undegraded":"2024-09-21T06:28:06.271467+0000","last_fullsized":"2024-09-21T06:28:06.271467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917795+0000","last_change":"2024-09-21T06:28:01.253480+0000","last_active":"2024-09-21T06:28:06.917795+0000","last_peered":"2024-09-21T06:28:06.917795+0000","last_clean":"2024-09-21T06:28:06.917795+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.917795+0000","last_undegraded":"2024-09-21T06:28:06.917795+0000","last_fullsized":"2024-09-21T06:28:06.917795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:51.213607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271436+0000","last_change":"2024-09-21T06:28:01.231505+0000","last_active":"2024-09-21T06:28:06.271436+0000","last_peered":"2024-09-21T06:28:06.271436+0000","last_clean":"2024-09-21T06:28:06.271436+0000","last_became_active":"2024-09-21T06:28:01.223703+0000","last_became_peered":"2024-09-21T06:28:01.223703+0000","last_unstale":"2024-09-21T06:28:06.271436+0000","last_undegraded":"2024-09-21T06:28:06.271436+0000","last_fullsized":"2024-09-21T06:28:06.271436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918618+0000","last_change":"2024-09-21T06:28:01.236472+0000","last_active":"2024-09-21T06:28:06.918618+0000","last_peered":"2024-09-21T06:28:06.918618+0000","last_clean":"2024-09-21T06:28:06.918618+0000","last_became_active":"2024-09-21T06:28:01.221503+0000","last_became_peered":"2024-09-21T06:28:01.221503+0000","last_unstale":"2024-09-21T06:28:06.918618+0000","last_undegraded":"2024-09-21T06:28:06.918618+0000","last_fullsized":"2024-09-21T06:28:06.918618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:35.021658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918538+0000","last_change":"2024-09-21T06:28:01.227458+0000","last_active":"2024-09-21T06:28:06.918538+0000","last_peered":"2024-09-21T06:28:06.918538+0000","last_clean":"2024-09-21T06:28:06.918538+0000","last_became_active":"2024-09-21T06:28:01.227145+0000","last_became_peered":"2024-09-21T06:28:01.227145+0000","last_unstale":"2024-09-21T06:28:06.918538+0000","last_undegraded":"2024-09-21T06:28:06.918538+0000","last_fullsized":"2024-09-21T06:28:06.918538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:24.909945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918495+0000","last_change":"2024-09-21T06:28:01.233932+0000","last_active":"2024-09-21T06:28:06.918495+0000","last_peered":"2024-09-21T06:28:06.918495+0000","last_clean":"2024-09-21T06:28:06.918495+0000","last_became_active":"2024-09-21T06:28:01.233617+0000","last_became_peered":"2024-09-21T06:28:01.233617+0000","last_unstale":"2024-09-21T06:28:06.918495+0000","last_undegraded":"2024-09-21T06:28:06.918495+0000","last_fullsized":"2024-09-21T06:28:06.918495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:27.585260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:28:20.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.773+0000 7f5279000640 1 -- 172.21.15.37:0/1738656623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 msgr2=0x7f5274162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.773+0000 7f5279000640 1 --2- 172.21.15.37:0/1738656623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f5274162690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5260005fd0 tx=0x7f526002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 -- 172.21.15.37:0/1738656623 shutdown_connections 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 --2- 172.21.15.37:0/1738656623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f5274162690 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 --2- 172.21.15.37:0/1738656623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5274162f00 0x7f52740d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 -- 172.21.15.37:0/1738656623 >> 172.21.15.37:0/1738656623 conn(0x7f52740b6b20 msgr2=0x7f52740d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 -- 172.21.15.37:0/1738656623 shutdown_connections 2024-09-21T06:28:20.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.774+0000 7f5279000640 1 -- 172.21.15.37:0/1738656623 wait complete. 2024-09-21T06:28:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.776+0000 7f5279000640 1 Processor -- start 2024-09-21T06:28:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.776+0000 7f5279000640 1 -- start start 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.777+0000 7f5279000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52740b2e40 0x7f5274161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.777+0000 7f5279000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.777+0000 7f5279000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52741466c0 0x7f5274146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5279000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f527415e630 con 0x7f5274162280 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5279000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f527415e4b0 con 0x7f52740b2e40 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5279000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f527415e7b0 con 0x7f52741466c0 2024-09-21T06:28:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42176/0 (socket says 172.21.15.37:42176) 2024-09-21T06:28:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 -- 172.21.15.37:0/1114986754 learned_addr learned my addr 172.21.15.37:0/1114986754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271e00640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52740b2e40 0x7f5274161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5272800640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52741466c0 0x7f5274146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 -- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52741466c0 msgr2=0x7f5274146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52741466c0 0x7f5274146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 -- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52740b2e40 msgr2=0x7f5274161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52740b2e40 0x7f5274161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.778+0000 7f5271400640 1 -- 172.21.15.37:0/1114986754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5274155a30 con 0x7f5274162280 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f5271400640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5260000c00 tx=0x7f52600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526003e030 con 0x7f5274162280 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5260003ae0 con 0x7f5274162280 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260035040 con 0x7f5274162280 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52740cb620 con 0x7f5274162280 2024-09-21T06:28:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.779+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5274160c50 con 0x7f5274162280 2024-09-21T06:28:20.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.780+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5274162f00 con 0x7f5274162280 2024-09-21T06:28:20.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.781+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5260041030 con 0x7f5274162280 2024-09-21T06:28:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.782+0000 7f526a000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 0x7f525407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.783+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f52600bb5d0 con 0x7f5274162280 2024-09-21T06:28:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.783+0000 7f5271e00640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 0x7f525407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.784+0000 7f5271e00640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 0x7f525407b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5264009bb0 tx=0x7f5264005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.787+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5260082fe0 con 0x7f5274162280 2024-09-21T06:28:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.980+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5274162730 con 0x7f5254078b90 2024-09-21T06:28:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.987+0000 7f526a000640 1 -- 172.21.15.37:0/1114986754 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332555 (secure 0 0 0) 0x7f5274162730 con 0x7f5254078b90 2024-09-21T06:28:20.993 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:20.995 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 msgr2=0x7f525407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 0x7f525407b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5264009bb0 tx=0x7f5264005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 msgr2=0x7f52740ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5260000c00 tx=0x7f52600035f0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 shutdown_connections 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5254078b90 0x7f525407b0a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5274162280 0x7f52740ca900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52741466c0 0x7f5274146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 --2- 172.21.15.37:0/1114986754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52740b2e40 0x7f5274161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 >> 172.21.15.37:0/1114986754 conn(0x7f52740b6b20 msgr2=0x7f527415f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 shutdown_connections 2024-09-21T06:28:20.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:20.992+0000 7f5279000640 1 -- 172.21.15.37:0/1114986754 wait complete. 2024-09-21T06:28:21.015 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-21T06:28:19.091151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219672,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755048,"statfs":{"total":773094113280,"available":772869169152,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003736"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275298+0000","last_change":"2024-09-21T06:28:00.220046+0000","last_active":"2024-09-21T06:28:06.275298+0000","last_peered":"2024-09-21T06:28:06.275298+0000","last_clean":"2024-09-21T06:28:06.275298+0000","last_became_active":"2024-09-21T06:28:00.219127+0000","last_became_peered":"2024-09-21T06:28:00.219127+0000","last_unstale":"2024-09-21T06:28:06.275298+0000","last_undegraded":"2024-09-21T06:28:06.275298+0000","last_fullsized":"2024-09-21T06:28:06.275298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:49.256177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918731+0000","last_change":"2024-09-21T06:28:01.226620+0000","last_active":"2024-09-21T06:28:06.918731+0000","last_peered":"2024-09-21T06:28:06.918731+0000","last_clean":"2024-09-21T06:28:06.918731+0000","last_became_active":"2024-09-21T06:28:01.224002+0000","last_became_peered":"2024-09-21T06:28:01.224002+0000","last_unstale":"2024-09-21T06:28:06.918731+0000","last_undegraded":"2024-09-21T06:28:06.918731+0000","last_fullsized":"2024-09-21T06:28:06.918731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.430586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276059+0000","last_change":"2024-09-21T06:28:05.289106+0000","last_active":"2024-09-21T06:28:06.276059+0000","last_peered":"2024-09-21T06:28:06.276059+0000","last_clean":"2024-09-21T06:28:06.276059+0000","last_became_active":"2024-09-21T06:28:05.288635+0000","last_became_peered":"2024-09-21T06:28:05.288635+0000","last_unstale":"2024-09-21T06:28:06.276059+0000","last_undegraded":"2024-09-21T06:28:06.276059+0000","last_fullsized":"2024-09-21T06:28:06.276059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:30.614325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918864+0000","last_change":"2024-09-21T06:28:00.216173+0000","last_active":"2024-09-21T06:28:06.918864+0000","last_peered":"2024-09-21T06:28:06.918864+0000","last_clean":"2024-09-21T06:28:06.918864+0000","last_became_active":"2024-09-21T06:28:00.215794+0000","last_became_peered":"2024-09-21T06:28:00.215794+0000","last_unstale":"2024-09-21T06:28:06.918864+0000","last_undegraded":"2024-09-21T06:28:06.918864+0000","last_fullsized":"2024-09-21T06:28:06.918864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:18.915364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918573+0000","last_change":"2024-09-21T06:28:01.238244+0000","last_active":"2024-09-21T06:28:06.918573+0000","last_peered":"2024-09-21T06:28:06.918573+0000","last_clean":"2024-09-21T06:28:06.918573+0000","last_became_active":"2024-09-21T06:28:01.237538+0000","last_became_peered":"2024-09-21T06:28:01.237538+0000","last_unstale":"2024-09-21T06:28:06.918573+0000","last_undegraded":"2024-09-21T06:28:06.918573+0000","last_fullsized":"2024-09-21T06:28:06.918573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:11.476210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271237+0000","last_change":"2024-09-21T06:28:05.267181+0000","last_active":"2024-09-21T06:28:06.271237+0000","last_peered":"2024-09-21T06:28:06.271237+0000","last_clean":"2024-09-21T06:28:06.271237+0000","last_became_active":"2024-09-21T06:28:05.266697+0000","last_became_peered":"2024-09-21T06:28:05.266697+0000","last_unstale":"2024-09-21T06:28:06.271237+0000","last_undegraded":"2024-09-21T06:28:06.271237+0000","last_fullsized":"2024-09-21T06:28:06.271237+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.144842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918246+0000","last_change":"2024-09-21T06:28:00.230877+0000","last_active":"2024-09-21T06:28:06.918246+0000","last_peered":"2024-09-21T06:28:06.918246+0000","last_clean":"2024-09-21T06:28:06.918246+0000","last_became_active":"2024-09-21T06:28:00.230578+0000","last_became_peered":"2024-09-21T06:28:00.230578+0000","last_unstale":"2024-09-21T06:28:06.918246+0000","last_undegraded":"2024-09-21T06:28:06.918246+0000","last_fullsized":"2024-09-21T06:28:06.918246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919137+0000","last_change":"2024-09-21T06:28:01.255720+0000","last_active":"2024-09-21T06:28:06.919137+0000","last_peered":"2024-09-21T06:28:06.919137+0000","last_clean":"2024-09-21T06:28:06.919137+0000","last_became_active":"2024-09-21T06:28:01.254968+0000","last_became_peered":"2024-09-21T06:28:01.254968+0000","last_unstale":"2024-09-21T06:28:06.919137+0000","last_undegraded":"2024-09-21T06:28:06.919137+0000","last_fullsized":"2024-09-21T06:28:06.919137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271315+0000","last_change":"2024-09-21T06:28:05.277963+0000","last_active":"2024-09-21T06:28:06.271315+0000","last_peered":"2024-09-21T06:28:06.271315+0000","last_clean":"2024-09-21T06:28:06.271315+0000","last_became_active":"2024-09-21T06:28:05.277006+0000","last_became_peered":"2024-09-21T06:28:05.277006+0000","last_unstale":"2024-09-21T06:28:06.271315+0000","last_undegraded":"2024-09-21T06:28:06.271315+0000","last_fullsized":"2024-09-21T06:28:06.271315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:35.624735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918079+0000","last_change":"2024-09-21T06:28:00.230976+0000","last_active":"2024-09-21T06:28:06.918079+0000","last_peered":"2024-09-21T06:28:06.918079+0000","last_clean":"2024-09-21T06:28:06.918079+0000","last_became_active":"2024-09-21T06:28:00.230660+0000","last_became_peered":"2024-09-21T06:28:00.230660+0000","last_unstale":"2024-09-21T06:28:06.918079+0000","last_undegraded":"2024-09-21T06:28:06.918079+0000","last_fullsized":"2024-09-21T06:28:06.918079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:59.230778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567463+0000","last_change":"2024-09-21T06:28:01.234299+0000","last_active":"2024-09-21T06:28:06.567463+0000","last_peered":"2024-09-21T06:28:06.567463+0000","last_clean":"2024-09-21T06:28:06.567463+0000","last_became_active":"2024-09-21T06:28:01.233998+0000","last_became_peered":"2024-09-21T06:28:01.233998+0000","last_unstale":"2024-09-21T06:28:06.567463+0000","last_undegraded":"2024-09-21T06:28:06.567463+0000","last_fullsized":"2024-09-21T06:28:06.567463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:08.335780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271460+0000","last_change":"2024-09-21T06:28:05.271851+0000","last_active":"2024-09-21T06:28:06.271460+0000","last_peered":"2024-09-21T06:28:06.271460+0000","last_clean":"2024-09-21T06:28:06.271460+0000","last_became_active":"2024-09-21T06:28:05.265434+0000","last_became_peered":"2024-09-21T06:28:05.265434+0000","last_unstale":"2024-09-21T06:28:06.271460+0000","last_undegraded":"2024-09-21T06:28:06.271460+0000","last_fullsized":"2024-09-21T06:28:06.271460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:39.030315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566989+0000","last_change":"2024-09-21T06:28:01.226065+0000","last_active":"2024-09-21T06:28:06.566989+0000","last_peered":"2024-09-21T06:28:06.566989+0000","last_clean":"2024-09-21T06:28:06.566989+0000","last_became_active":"2024-09-21T06:28:01.225761+0000","last_became_peered":"2024-09-21T06:28:01.225761+0000","last_unstale":"2024-09-21T06:28:06.566989+0000","last_undegraded":"2024-09-21T06:28:06.566989+0000","last_fullsized":"2024-09-21T06:28:06.566989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:02.936599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567107+0000","last_change":"2024-09-21T06:28:00.220356+0000","last_active":"2024-09-21T06:28:06.567107+0000","last_peered":"2024-09-21T06:28:06.567107+0000","last_clean":"2024-09-21T06:28:06.567107+0000","last_became_active":"2024-09-21T06:28:00.219737+0000","last_became_peered":"2024-09-21T06:28:00.219737+0000","last_unstale":"2024-09-21T06:28:06.567107+0000","last_undegraded":"2024-09-21T06:28:06.567107+0000","last_fullsized":"2024-09-21T06:28:06.567107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:58.072689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567253+0000","last_change":"2024-09-21T06:28:05.264666+0000","last_active":"2024-09-21T06:28:06.567253+0000","last_peered":"2024-09-21T06:28:06.567253+0000","last_clean":"2024-09-21T06:28:06.567253+0000","last_became_active":"2024-09-21T06:28:05.264145+0000","last_became_peered":"2024-09-21T06:28:05.264145+0000","last_unstale":"2024-09-21T06:28:06.567253+0000","last_undegraded":"2024-09-21T06:28:06.567253+0000","last_fullsized":"2024-09-21T06:28:06.567253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.384335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276438+0000","last_change":"2024-09-21T06:28:01.238641+0000","last_active":"2024-09-21T06:28:06.276438+0000","last_peered":"2024-09-21T06:28:06.276438+0000","last_clean":"2024-09-21T06:28:06.276438+0000","last_became_active":"2024-09-21T06:28:01.237104+0000","last_became_peered":"2024-09-21T06:28:01.237104+0000","last_unstale":"2024-09-21T06:28:06.276438+0000","last_undegraded":"2024-09-21T06:28:06.276438+0000","last_fullsized":"2024-09-21T06:28:06.276438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919263+0000","last_change":"2024-09-21T06:28:00.222233+0000","last_active":"2024-09-21T06:28:06.919263+0000","last_peered":"2024-09-21T06:28:06.919263+0000","last_clean":"2024-09-21T06:28:06.919263+0000","last_became_active":"2024-09-21T06:28:00.221686+0000","last_became_peered":"2024-09-21T06:28:00.221686+0000","last_unstale":"2024-09-21T06:28:06.919263+0000","last_undegraded":"2024-09-21T06:28:06.919263+0000","last_fullsized":"2024-09-21T06:28:06.919263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:40.731260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917906+0000","last_change":"2024-09-21T06:28:05.275795+0000","last_active":"2024-09-21T06:28:06.917906+0000","last_peered":"2024-09-21T06:28:06.917906+0000","last_clean":"2024-09-21T06:28:06.917906+0000","last_became_active":"2024-09-21T06:28:05.275276+0000","last_became_peered":"2024-09-21T06:28:05.275276+0000","last_unstale":"2024-09-21T06:28:06.917906+0000","last_undegraded":"2024-09-21T06:28:06.917906+0000","last_fullsized":"2024-09-21T06:28:06.917906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:55.953841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273709+0000","last_change":"2024-09-21T06:28:01.231657+0000","last_active":"2024-09-21T06:28:06.273709+0000","last_peered":"2024-09-21T06:28:06.273709+0000","last_clean":"2024-09-21T06:28:06.273709+0000","last_became_active":"2024-09-21T06:28:01.223226+0000","last_became_peered":"2024-09-21T06:28:01.223226+0000","last_unstale":"2024-09-21T06:28:06.273709+0000","last_undegraded":"2024-09-21T06:28:06.273709+0000","last_fullsized":"2024-09-21T06:28:06.273709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274062+0000","last_change":"2024-09-21T06:28:00.227556+0000","last_active":"2024-09-21T06:28:06.274062+0000","last_peered":"2024-09-21T06:28:06.274062+0000","last_clean":"2024-09-21T06:28:06.274062+0000","last_became_active":"2024-09-21T06:28:00.227125+0000","last_became_peered":"2024-09-21T06:28:00.227125+0000","last_unstale":"2024-09-21T06:28:06.274062+0000","last_undegraded":"2024-09-21T06:28:06.274062+0000","last_fullsized":"2024-09-21T06:28:06.274062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:03.936656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277113+0000","last_change":"2024-09-21T06:28:05.283986+0000","last_active":"2024-09-21T06:28:06.277113+0000","last_peered":"2024-09-21T06:28:06.277113+0000","last_clean":"2024-09-21T06:28:06.277113+0000","last_became_active":"2024-09-21T06:28:05.283382+0000","last_became_peered":"2024-09-21T06:28:05.283382+0000","last_unstale":"2024-09-21T06:28:06.277113+0000","last_undegraded":"2024-09-21T06:28:06.277113+0000","last_fullsized":"2024-09-21T06:28:06.277113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.496971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566814+0000","last_change":"2024-09-21T06:28:01.223370+0000","last_active":"2024-09-21T06:28:06.566814+0000","last_peered":"2024-09-21T06:28:06.566814+0000","last_clean":"2024-09-21T06:28:06.566814+0000","last_became_active":"2024-09-21T06:28:01.223018+0000","last_became_peered":"2024-09-21T06:28:01.223018+0000","last_unstale":"2024-09-21T06:28:06.566814+0000","last_undegraded":"2024-09-21T06:28:06.566814+0000","last_fullsized":"2024-09-21T06:28:06.566814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:57.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917258+0000","last_change":"2024-09-21T06:28:00.225692+0000","last_active":"2024-09-21T06:28:06.917258+0000","last_peered":"2024-09-21T06:28:06.917258+0000","last_clean":"2024-09-21T06:28:06.917258+0000","last_became_active":"2024-09-21T06:28:00.224466+0000","last_became_peered":"2024-09-21T06:28:00.224466+0000","last_unstale":"2024-09-21T06:28:06.917258+0000","last_undegraded":"2024-09-21T06:28:06.917258+0000","last_fullsized":"2024-09-21T06:28:06.917258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.485812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271902+0000","last_change":"2024-09-21T06:28:05.282444+0000","last_active":"2024-09-21T06:28:06.271902+0000","last_peered":"2024-09-21T06:28:06.271902+0000","last_clean":"2024-09-21T06:28:06.271902+0000","last_became_active":"2024-09-21T06:28:05.282050+0000","last_became_peered":"2024-09-21T06:28:05.282050+0000","last_unstale":"2024-09-21T06:28:06.271902+0000","last_undegraded":"2024-09-21T06:28:06.271902+0000","last_fullsized":"2024-09-21T06:28:06.271902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274767+0000","last_change":"2024-09-21T06:28:01.225464+0000","last_active":"2024-09-21T06:28:06.274767+0000","last_peered":"2024-09-21T06:28:06.274767+0000","last_clean":"2024-09-21T06:28:06.274767+0000","last_became_active":"2024-09-21T06:28:01.223116+0000","last_became_peered":"2024-09-21T06:28:01.223116+0000","last_unstale":"2024-09-21T06:28:06.274767+0000","last_undegraded":"2024-09-21T06:28:06.274767+0000","last_fullsized":"2024-09-21T06:28:06.274767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567905+0000","last_change":"2024-09-21T06:28:00.230433+0000","last_active":"2024-09-21T06:28:06.567905+0000","last_peered":"2024-09-21T06:28:06.567905+0000","last_clean":"2024-09-21T06:28:06.567905+0000","last_became_active":"2024-09-21T06:28:00.228324+0000","last_became_peered":"2024-09-21T06:28:00.228324+0000","last_unstale":"2024-09-21T06:28:06.567905+0000","last_undegraded":"2024-09-21T06:28:06.567905+0000","last_fullsized":"2024-09-21T06:28:06.567905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:40.653033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271246+0000","last_change":"2024-09-21T06:28:05.280019+0000","last_active":"2024-09-21T06:28:06.271246+0000","last_peered":"2024-09-21T06:28:06.271246+0000","last_clean":"2024-09-21T06:28:06.271246+0000","last_became_active":"2024-09-21T06:28:05.279566+0000","last_became_peered":"2024-09-21T06:28:05.279566+0000","last_unstale":"2024-09-21T06:28:06.271246+0000","last_undegraded":"2024-09-21T06:28:06.271246+0000","last_fullsized":"2024-09-21T06:28:06.271246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:46.736598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275157+0000","last_change":"2024-09-21T06:28:01.227702+0000","last_active":"2024-09-21T06:28:06.275157+0000","last_peered":"2024-09-21T06:28:06.275157+0000","last_clean":"2024-09-21T06:28:06.275157+0000","last_became_active":"2024-09-21T06:28:01.227351+0000","last_became_peered":"2024-09-21T06:28:01.227351+0000","last_unstale":"2024-09-21T06:28:06.275157+0000","last_undegraded":"2024-09-21T06:28:06.275157+0000","last_fullsized":"2024-09-21T06:28:06.275157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275087+0000","last_change":"2024-09-21T06:28:00.227291+0000","last_active":"2024-09-21T06:28:06.275087+0000","last_peered":"2024-09-21T06:28:06.275087+0000","last_clean":"2024-09-21T06:28:06.275087+0000","last_became_active":"2024-09-21T06:28:00.226969+0000","last_became_peered":"2024-09-21T06:28:00.226969+0000","last_unstale":"2024-09-21T06:28:06.275087+0000","last_undegraded":"2024-09-21T06:28:06.275087+0000","last_fullsized":"2024-09-21T06:28:06.275087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:31.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273969+0000","last_change":"2024-09-21T06:28:05.277580+0000","last_active":"2024-09-21T06:28:06.273969+0000","last_peered":"2024-09-21T06:28:06.273969+0000","last_clean":"2024-09-21T06:28:06.273969+0000","last_became_active":"2024-09-21T06:28:05.277134+0000","last_became_peered":"2024-09-21T06:28:05.277134+0000","last_unstale":"2024-09-21T06:28:06.273969+0000","last_undegraded":"2024-09-21T06:28:06.273969+0000","last_fullsized":"2024-09-21T06:28:06.273969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919851+0000","last_change":"2024-09-21T06:28:01.235542+0000","last_active":"2024-09-21T06:28:06.919851+0000","last_peered":"2024-09-21T06:28:06.919851+0000","last_clean":"2024-09-21T06:28:06.919851+0000","last_became_active":"2024-09-21T06:28:01.235235+0000","last_became_peered":"2024-09-21T06:28:01.235235+0000","last_unstale":"2024-09-21T06:28:06.919851+0000","last_undegraded":"2024-09-21T06:28:06.919851+0000","last_fullsized":"2024-09-21T06:28:06.919851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567780+0000","last_change":"2024-09-21T06:28:00.219554+0000","last_active":"2024-09-21T06:28:06.567780+0000","last_peered":"2024-09-21T06:28:06.567780+0000","last_clean":"2024-09-21T06:28:06.567780+0000","last_became_active":"2024-09-21T06:28:00.219180+0000","last_became_peered":"2024-09-21T06:28:00.219180+0000","last_unstale":"2024-09-21T06:28:06.567780+0000","last_undegraded":"2024-09-21T06:28:06.567780+0000","last_fullsized":"2024-09-21T06:28:06.567780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.267700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919430+0000","last_change":"2024-09-21T06:28:05.268962+0000","last_active":"2024-09-21T06:28:06.919430+0000","last_peered":"2024-09-21T06:28:06.919430+0000","last_clean":"2024-09-21T06:28:06.919430+0000","last_became_active":"2024-09-21T06:28:05.268415+0000","last_became_peered":"2024-09-21T06:28:05.268415+0000","last_unstale":"2024-09-21T06:28:06.919430+0000","last_undegraded":"2024-09-21T06:28:06.919430+0000","last_fullsized":"2024-09-21T06:28:06.919430+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.107198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271569+0000","last_change":"2024-09-21T06:28:01.230048+0000","last_active":"2024-09-21T06:28:06.271569+0000","last_peered":"2024-09-21T06:28:06.271569+0000","last_clean":"2024-09-21T06:28:06.271569+0000","last_became_active":"2024-09-21T06:28:01.229747+0000","last_became_peered":"2024-09-21T06:28:01.229747+0000","last_unstale":"2024-09-21T06:28:06.271569+0000","last_undegraded":"2024-09-21T06:28:06.271569+0000","last_fullsized":"2024-09-21T06:28:06.271569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918630+0000","last_change":"2024-09-21T06:28:00.221155+0000","last_active":"2024-09-21T06:28:06.918630+0000","last_peered":"2024-09-21T06:28:06.918630+0000","last_clean":"2024-09-21T06:28:06.918630+0000","last_became_active":"2024-09-21T06:28:00.220728+0000","last_became_peered":"2024-09-21T06:28:00.220728+0000","last_unstale":"2024-09-21T06:28:06.918630+0000","last_undegraded":"2024-09-21T06:28:06.918630+0000","last_fullsized":"2024-09-21T06:28:06.918630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.642076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919099+0000","last_change":"2024-09-21T06:28:05.276248+0000","last_active":"2024-09-21T06:28:06.919099+0000","last_peered":"2024-09-21T06:28:06.919099+0000","last_clean":"2024-09-21T06:28:06.919099+0000","last_became_active":"2024-09-21T06:28:05.271420+0000","last_became_peered":"2024-09-21T06:28:05.271420+0000","last_unstale":"2024-09-21T06:28:06.919099+0000","last_undegraded":"2024-09-21T06:28:06.919099+0000","last_fullsized":"2024-09-21T06:28:06.919099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:03.357365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917672+0000","last_change":"2024-09-21T06:28:01.240349+0000","last_active":"2024-09-21T06:28:06.917672+0000","last_peered":"2024-09-21T06:28:06.917672+0000","last_clean":"2024-09-21T06:28:06.917672+0000","last_became_active":"2024-09-21T06:28:01.239325+0000","last_became_peered":"2024-09-21T06:28:01.239325+0000","last_unstale":"2024-09-21T06:28:06.917672+0000","last_undegraded":"2024-09-21T06:28:06.917672+0000","last_fullsized":"2024-09-21T06:28:06.917672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567402+0000","last_change":"2024-09-21T06:28:00.225175+0000","last_active":"2024-09-21T06:28:06.567402+0000","last_peered":"2024-09-21T06:28:06.567402+0000","last_clean":"2024-09-21T06:28:06.567402+0000","last_became_active":"2024-09-21T06:28:00.224858+0000","last_became_peered":"2024-09-21T06:28:00.224858+0000","last_unstale":"2024-09-21T06:28:06.567402+0000","last_undegraded":"2024-09-21T06:28:06.567402+0000","last_fullsized":"2024-09-21T06:28:06.567402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:41.165675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919693+0000","last_change":"2024-09-21T06:28:05.283982+0000","last_active":"2024-09-21T06:28:06.919693+0000","last_peered":"2024-09-21T06:28:06.919693+0000","last_clean":"2024-09-21T06:28:06.919693+0000","last_became_active":"2024-09-21T06:28:05.283527+0000","last_became_peered":"2024-09-21T06:28:05.283527+0000","last_unstale":"2024-09-21T06:28:06.919693+0000","last_undegraded":"2024-09-21T06:28:06.919693+0000","last_fullsized":"2024-09-21T06:28:06.919693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:17.117103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917636+0000","last_change":"2024-09-21T06:28:01.224161+0000","last_active":"2024-09-21T06:28:06.917636+0000","last_peered":"2024-09-21T06:28:06.917636+0000","last_clean":"2024-09-21T06:28:06.917636+0000","last_became_active":"2024-09-21T06:28:01.223787+0000","last_became_peered":"2024-09-21T06:28:01.223787+0000","last_unstale":"2024-09-21T06:28:06.917636+0000","last_undegraded":"2024-09-21T06:28:06.917636+0000","last_fullsized":"2024-09-21T06:28:06.917636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:03.351441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276110+0000","last_change":"2024-09-21T06:28:00.218460+0000","last_active":"2024-09-21T06:28:06.276110+0000","last_peered":"2024-09-21T06:28:06.276110+0000","last_clean":"2024-09-21T06:28:06.276110+0000","last_became_active":"2024-09-21T06:28:00.218046+0000","last_became_peered":"2024-09-21T06:28:00.218046+0000","last_unstale":"2024-09-21T06:28:06.276110+0000","last_undegraded":"2024-09-21T06:28:06.276110+0000","last_fullsized":"2024-09-21T06:28:06.276110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919620+0000","last_change":"2024-09-21T06:28:05.274088+0000","last_active":"2024-09-21T06:28:06.919620+0000","last_peered":"2024-09-21T06:28:06.919620+0000","last_clean":"2024-09-21T06:28:06.919620+0000","last_became_active":"2024-09-21T06:28:05.273184+0000","last_became_peered":"2024-09-21T06:28:05.273184+0000","last_unstale":"2024-09-21T06:28:06.919620+0000","last_undegraded":"2024-09-21T06:28:06.919620+0000","last_fullsized":"2024-09-21T06:28:06.919620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:17.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918180+0000","last_change":"2024-09-21T06:28:01.230531+0000","last_active":"2024-09-21T06:28:06.918180+0000","last_peered":"2024-09-21T06:28:06.918180+0000","last_clean":"2024-09-21T06:28:06.918180+0000","last_became_active":"2024-09-21T06:28:01.230236+0000","last_became_peered":"2024-09-21T06:28:01.230236+0000","last_unstale":"2024-09-21T06:28:06.918180+0000","last_undegraded":"2024-09-21T06:28:06.918180+0000","last_fullsized":"2024-09-21T06:28:06.918180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.172198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567087+0000","last_change":"2024-09-21T06:28:00.233247+0000","last_active":"2024-09-21T06:28:06.567087+0000","last_peered":"2024-09-21T06:28:06.567087+0000","last_clean":"2024-09-21T06:28:06.567087+0000","last_became_active":"2024-09-21T06:28:00.232905+0000","last_became_peered":"2024-09-21T06:28:00.232905+0000","last_unstale":"2024-09-21T06:28:06.567087+0000","last_undegraded":"2024-09-21T06:28:06.567087+0000","last_fullsized":"2024-09-21T06:28:06.567087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918161+0000","last_change":"2024-09-21T06:28:05.266249+0000","last_active":"2024-09-21T06:28:06.918161+0000","last_peered":"2024-09-21T06:28:06.918161+0000","last_clean":"2024-09-21T06:28:06.918161+0000","last_became_active":"2024-09-21T06:28:05.265779+0000","last_became_peered":"2024-09-21T06:28:05.265779+0000","last_unstale":"2024-09-21T06:28:06.918161+0000","last_undegraded":"2024-09-21T06:28:06.918161+0000","last_fullsized":"2024-09-21T06:28:06.918161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:58.459609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274361+0000","last_change":"2024-09-21T06:28:01.236228+0000","last_active":"2024-09-21T06:28:06.274361+0000","last_peered":"2024-09-21T06:28:06.274361+0000","last_clean":"2024-09-21T06:28:06.274361+0000","last_became_active":"2024-09-21T06:28:01.229874+0000","last_became_peered":"2024-09-21T06:28:01.229874+0000","last_unstale":"2024-09-21T06:28:06.274361+0000","last_undegraded":"2024-09-21T06:28:06.274361+0000","last_fullsized":"2024-09-21T06:28:06.274361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:08.119892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567235+0000","last_change":"2024-09-21T06:28:00.219928+0000","last_active":"2024-09-21T06:28:06.567235+0000","last_peered":"2024-09-21T06:28:06.567235+0000","last_clean":"2024-09-21T06:28:06.567235+0000","last_became_active":"2024-09-21T06:28:00.219561+0000","last_became_peered":"2024-09-21T06:28:00.219561+0000","last_unstale":"2024-09-21T06:28:06.567235+0000","last_undegraded":"2024-09-21T06:28:06.567235+0000","last_fullsized":"2024-09-21T06:28:06.567235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:22.801415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918345+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:28:06.918345+0000","last_peered":"2024-09-21T06:28:06.918345+0000","last_clean":"2024-09-21T06:28:06.918345+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:28:06.918345+0000","last_undegraded":"2024-09-21T06:28:06.918345+0000","last_fullsized":"2024-09-21T06:28:06.918345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567826+0000","last_change":"2024-09-21T06:28:05.274952+0000","last_active":"2024-09-21T06:28:06.567826+0000","last_peered":"2024-09-21T06:28:06.567826+0000","last_clean":"2024-09-21T06:28:06.567826+0000","last_became_active":"2024-09-21T06:28:05.274514+0000","last_became_peered":"2024-09-21T06:28:05.274514+0000","last_unstale":"2024-09-21T06:28:06.567826+0000","last_undegraded":"2024-09-21T06:28:06.567826+0000","last_fullsized":"2024-09-21T06:28:06.567826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:04.636189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567574+0000","last_change":"2024-09-21T06:28:01.224594+0000","last_active":"2024-09-21T06:28:06.567574+0000","last_peered":"2024-09-21T06:28:06.567574+0000","last_clean":"2024-09-21T06:28:06.567574+0000","last_became_active":"2024-09-21T06:28:01.224267+0000","last_became_peered":"2024-09-21T06:28:01.224267+0000","last_unstale":"2024-09-21T06:28:06.567574+0000","last_undegraded":"2024-09-21T06:28:06.567574+0000","last_fullsized":"2024-09-21T06:28:06.567574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:57.968897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919963+0000","last_change":"2024-09-21T06:28:00.226006+0000","last_active":"2024-09-21T06:28:06.919963+0000","last_peered":"2024-09-21T06:28:06.919963+0000","last_clean":"2024-09-21T06:28:06.919963+0000","last_became_active":"2024-09-21T06:28:00.225678+0000","last_became_peered":"2024-09-21T06:28:00.225678+0000","last_unstale":"2024-09-21T06:28:06.919963+0000","last_undegraded":"2024-09-21T06:28:06.919963+0000","last_fullsized":"2024-09-21T06:28:06.919963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:54.460846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271274+0000","last_change":"2024-09-21T06:28:05.280689+0000","last_active":"2024-09-21T06:28:06.271274+0000","last_peered":"2024-09-21T06:28:06.271274+0000","last_clean":"2024-09-21T06:28:06.271274+0000","last_became_active":"2024-09-21T06:28:05.272594+0000","last_became_peered":"2024-09-21T06:28:05.272594+0000","last_unstale":"2024-09-21T06:28:06.271274+0000","last_undegraded":"2024-09-21T06:28:06.271274+0000","last_fullsized":"2024-09-21T06:28:06.271274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:55.258366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917995+0000","last_change":"2024-09-21T06:28:01.230915+0000","last_active":"2024-09-21T06:28:06.917995+0000","last_peered":"2024-09-21T06:28:06.917995+0000","last_clean":"2024-09-21T06:28:06.917995+0000","last_became_active":"2024-09-21T06:28:01.230702+0000","last_became_peered":"2024-09-21T06:28:01.230702+0000","last_unstale":"2024-09-21T06:28:06.917995+0000","last_undegraded":"2024-09-21T06:28:06.917995+0000","last_fullsized":"2024-09-21T06:28:06.917995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:34.107851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918893+0000","last_change":"2024-09-21T06:28:00.214857+0000","last_active":"2024-09-21T06:28:06.918893+0000","last_peered":"2024-09-21T06:28:06.918893+0000","last_clean":"2024-09-21T06:28:06.918893+0000","last_became_active":"2024-09-21T06:28:00.214582+0000","last_became_peered":"2024-09-21T06:28:00.214582+0000","last_unstale":"2024-09-21T06:28:06.918893+0000","last_undegraded":"2024-09-21T06:28:06.918893+0000","last_fullsized":"2024-09-21T06:28:06.918893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:49.842050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918122+0000","last_change":"2024-09-21T06:28:05.273006+0000","last_active":"2024-09-21T06:28:06.918122+0000","last_peered":"2024-09-21T06:28:06.918122+0000","last_clean":"2024-09-21T06:28:06.918122+0000","last_became_active":"2024-09-21T06:28:05.272504+0000","last_became_peered":"2024-09-21T06:28:05.272504+0000","last_unstale":"2024-09-21T06:28:06.918122+0000","last_undegraded":"2024-09-21T06:28:06.918122+0000","last_fullsized":"2024-09-21T06:28:06.918122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:52.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.247357+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.242943+0000","last_became_peered":"2024-09-21T06:28:01.242943+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274916+0000","last_change":"2024-09-21T06:28:00.219919+0000","last_active":"2024-09-21T06:28:06.274916+0000","last_peered":"2024-09-21T06:28:06.274916+0000","last_clean":"2024-09-21T06:28:06.274916+0000","last_became_active":"2024-09-21T06:28:00.219544+0000","last_became_peered":"2024-09-21T06:28:00.219544+0000","last_unstale":"2024-09-21T06:28:06.274916+0000","last_undegraded":"2024-09-21T06:28:06.274916+0000","last_fullsized":"2024-09-21T06:28:06.274916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:21.858722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567045+0000","last_change":"2024-09-21T06:28:05.258119+0000","last_active":"2024-09-21T06:28:06.567045+0000","last_peered":"2024-09-21T06:28:06.567045+0000","last_clean":"2024-09-21T06:28:06.567045+0000","last_became_active":"2024-09-21T06:28:05.257394+0000","last_became_peered":"2024-09-21T06:28:05.257394+0000","last_unstale":"2024-09-21T06:28:06.567045+0000","last_undegraded":"2024-09-21T06:28:06.567045+0000","last_fullsized":"2024-09-21T06:28:06.567045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:13.794574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271877+0000","last_change":"2024-09-21T06:28:01.239234+0000","last_active":"2024-09-21T06:28:06.271877+0000","last_peered":"2024-09-21T06:28:06.271877+0000","last_clean":"2024-09-21T06:28:06.271877+0000","last_became_active":"2024-09-21T06:28:01.238978+0000","last_became_peered":"2024-09-21T06:28:01.238978+0000","last_unstale":"2024-09-21T06:28:06.271877+0000","last_undegraded":"2024-09-21T06:28:06.271877+0000","last_fullsized":"2024-09-21T06:28:06.271877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918626+0000","last_change":"2024-09-21T06:28:00.224968+0000","last_active":"2024-09-21T06:28:06.918626+0000","last_peered":"2024-09-21T06:28:06.918626+0000","last_clean":"2024-09-21T06:28:06.918626+0000","last_became_active":"2024-09-21T06:28:00.224751+0000","last_became_peered":"2024-09-21T06:28:00.224751+0000","last_unstale":"2024-09-21T06:28:06.918626+0000","last_undegraded":"2024-09-21T06:28:06.918626+0000","last_fullsized":"2024-09-21T06:28:06.918626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919013+0000","last_change":"2024-09-21T06:28:05.268891+0000","last_active":"2024-09-21T06:28:06.919013+0000","last_peered":"2024-09-21T06:28:06.919013+0000","last_clean":"2024-09-21T06:28:06.919013+0000","last_became_active":"2024-09-21T06:28:05.268010+0000","last_became_peered":"2024-09-21T06:28:05.268010+0000","last_unstale":"2024-09-21T06:28:06.919013+0000","last_undegraded":"2024-09-21T06:28:06.919013+0000","last_fullsized":"2024-09-21T06:28:06.919013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:04.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566758+0000","last_change":"2024-09-21T06:28:01.225615+0000","last_active":"2024-09-21T06:28:06.566758+0000","last_peered":"2024-09-21T06:28:06.566758+0000","last_clean":"2024-09-21T06:28:06.566758+0000","last_became_active":"2024-09-21T06:28:01.225277+0000","last_became_peered":"2024-09-21T06:28:01.225277+0000","last_unstale":"2024-09-21T06:28:06.566758+0000","last_undegraded":"2024-09-21T06:28:06.566758+0000","last_fullsized":"2024-09-21T06:28:06.566758+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.531813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273831+0000","last_change":"2024-09-21T06:28:00.215632+0000","last_active":"2024-09-21T06:28:06.273831+0000","last_peered":"2024-09-21T06:28:06.273831+0000","last_clean":"2024-09-21T06:28:06.273831+0000","last_became_active":"2024-09-21T06:28:00.215285+0000","last_became_peered":"2024-09-21T06:28:00.215285+0000","last_unstale":"2024-09-21T06:28:06.273831+0000","last_undegraded":"2024-09-21T06:28:06.273831+0000","last_fullsized":"2024-09-21T06:28:06.273831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918621+0000","last_change":"2024-09-21T06:28:05.274989+0000","last_active":"2024-09-21T06:28:06.918621+0000","last_peered":"2024-09-21T06:28:06.918621+0000","last_clean":"2024-09-21T06:28:06.918621+0000","last_became_active":"2024-09-21T06:28:05.274517+0000","last_became_peered":"2024-09-21T06:28:05.274517+0000","last_unstale":"2024-09-21T06:28:06.918621+0000","last_undegraded":"2024-09-21T06:28:06.918621+0000","last_fullsized":"2024-09-21T06:28:06.918621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:09.260032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918268+0000","last_change":"2024-09-21T06:28:01.221685+0000","last_active":"2024-09-21T06:28:06.918268+0000","last_peered":"2024-09-21T06:28:06.918268+0000","last_clean":"2024-09-21T06:28:06.918268+0000","last_became_active":"2024-09-21T06:28:01.221360+0000","last_became_peered":"2024-09-21T06:28:01.221360+0000","last_unstale":"2024-09-21T06:28:06.918268+0000","last_undegraded":"2024-09-21T06:28:06.918268+0000","last_fullsized":"2024-09-21T06:28:06.918268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917445+0000","last_change":"2024-09-21T06:28:00.235459+0000","last_active":"2024-09-21T06:28:06.917445+0000","last_peered":"2024-09-21T06:28:06.917445+0000","last_clean":"2024-09-21T06:28:06.917445+0000","last_became_active":"2024-09-21T06:28:00.235195+0000","last_became_peered":"2024-09-21T06:28:00.235195+0000","last_unstale":"2024-09-21T06:28:06.917445+0000","last_undegraded":"2024-09-21T06:28:06.917445+0000","last_fullsized":"2024-09-21T06:28:06.917445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918627+0000","last_change":"2024-09-21T06:28:05.271996+0000","last_active":"2024-09-21T06:28:06.918627+0000","last_peered":"2024-09-21T06:28:06.918627+0000","last_clean":"2024-09-21T06:28:06.918627+0000","last_became_active":"2024-09-21T06:28:05.270956+0000","last_became_peered":"2024-09-21T06:28:05.270956+0000","last_unstale":"2024-09-21T06:28:06.918627+0000","last_undegraded":"2024-09-21T06:28:06.918627+0000","last_fullsized":"2024-09-21T06:28:06.918627+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:21.181116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566881+0000","last_change":"2024-09-21T06:28:01.237451+0000","last_active":"2024-09-21T06:28:06.566881+0000","last_peered":"2024-09-21T06:28:06.566881+0000","last_clean":"2024-09-21T06:28:06.566881+0000","last_became_active":"2024-09-21T06:28:01.237064+0000","last_became_peered":"2024-09-21T06:28:01.237064+0000","last_unstale":"2024-09-21T06:28:06.566881+0000","last_undegraded":"2024-09-21T06:28:06.566881+0000","last_fullsized":"2024-09-21T06:28:06.566881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567273+0000","last_change":"2024-09-21T06:28:00.224624+0000","last_active":"2024-09-21T06:28:06.567273+0000","last_peered":"2024-09-21T06:28:06.567273+0000","last_clean":"2024-09-21T06:28:06.567273+0000","last_became_active":"2024-09-21T06:28:00.224259+0000","last_became_peered":"2024-09-21T06:28:00.224259+0000","last_unstale":"2024-09-21T06:28:06.567273+0000","last_undegraded":"2024-09-21T06:28:06.567273+0000","last_fullsized":"2024-09-21T06:28:06.567273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919872+0000","last_change":"2024-09-21T06:28:05.283392+0000","last_active":"2024-09-21T06:28:06.919872+0000","last_peered":"2024-09-21T06:28:06.919872+0000","last_clean":"2024-09-21T06:28:06.919872+0000","last_became_active":"2024-09-21T06:28:05.271401+0000","last_became_peered":"2024-09-21T06:28:05.271401+0000","last_unstale":"2024-09-21T06:28:06.919872+0000","last_undegraded":"2024-09-21T06:28:06.919872+0000","last_fullsized":"2024-09-21T06:28:06.919872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:21.192646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919152+0000","last_change":"2024-09-21T06:28:01.224123+0000","last_active":"2024-09-21T06:28:06.919152+0000","last_peered":"2024-09-21T06:28:06.919152+0000","last_clean":"2024-09-21T06:28:06.919152+0000","last_became_active":"2024-09-21T06:28:01.223773+0000","last_became_peered":"2024-09-21T06:28:01.223773+0000","last_unstale":"2024-09-21T06:28:06.919152+0000","last_undegraded":"2024-09-21T06:28:06.919152+0000","last_fullsized":"2024-09-21T06:28:06.919152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:59.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276563+0000","last_change":"2024-09-21T06:28:00.232406+0000","last_active":"2024-09-21T06:28:06.276563+0000","last_peered":"2024-09-21T06:28:06.276563+0000","last_clean":"2024-09-21T06:28:06.276563+0000","last_became_active":"2024-09-21T06:28:00.232143+0000","last_became_peered":"2024-09-21T06:28:00.232143+0000","last_unstale":"2024-09-21T06:28:06.276563+0000","last_undegraded":"2024-09-21T06:28:06.276563+0000","last_fullsized":"2024-09-21T06:28:06.276563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:33.343595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567590+0000","last_change":"2024-09-21T06:28:05.274346+0000","last_active":"2024-09-21T06:28:06.567590+0000","last_peered":"2024-09-21T06:28:06.567590+0000","last_clean":"2024-09-21T06:28:06.567590+0000","last_became_active":"2024-09-21T06:28:05.273714+0000","last_became_peered":"2024-09-21T06:28:05.273714+0000","last_unstale":"2024-09-21T06:28:06.567590+0000","last_undegraded":"2024-09-21T06:28:06.567590+0000","last_fullsized":"2024-09-21T06:28:06.567590+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:09.113671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918127+0000","last_change":"2024-09-21T06:28:01.238637+0000","last_active":"2024-09-21T06:28:06.918127+0000","last_peered":"2024-09-21T06:28:06.918127+0000","last_clean":"2024-09-21T06:28:06.918127+0000","last_became_active":"2024-09-21T06:28:01.236539+0000","last_became_peered":"2024-09-21T06:28:01.236539+0000","last_unstale":"2024-09-21T06:28:06.918127+0000","last_undegraded":"2024-09-21T06:28:06.918127+0000","last_fullsized":"2024-09-21T06:28:06.918127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.692846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:14.350747+0000","last_change":"2024-09-21T06:28:00.217119+0000","last_active":"2024-09-21T06:28:14.350747+0000","last_peered":"2024-09-21T06:28:14.350747+0000","last_clean":"2024-09-21T06:28:14.350747+0000","last_became_active":"2024-09-21T06:28:00.216806+0000","last_became_peered":"2024-09-21T06:28:00.216806+0000","last_unstale":"2024-09-21T06:28:14.350747+0000","last_undegraded":"2024-09-21T06:28:14.350747+0000","last_fullsized":"2024-09-21T06:28:14.350747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:17.032531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274402+0000","last_change":"2024-09-21T06:28:05.273619+0000","last_active":"2024-09-21T06:28:06.274402+0000","last_peered":"2024-09-21T06:28:06.274402+0000","last_clean":"2024-09-21T06:28:06.274402+0000","last_became_active":"2024-09-21T06:28:05.273193+0000","last_became_peered":"2024-09-21T06:28:05.273193+0000","last_unstale":"2024-09-21T06:28:06.274402+0000","last_undegraded":"2024-09-21T06:28:06.274402+0000","last_fullsized":"2024-09-21T06:28:06.274402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:31.630020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567019+0000","last_change":"2024-09-21T06:28:01.233057+0000","last_active":"2024-09-21T06:28:06.567019+0000","last_peered":"2024-09-21T06:28:06.567019+0000","last_clean":"2024-09-21T06:28:06.567019+0000","last_became_active":"2024-09-21T06:28:01.232602+0000","last_became_peered":"2024-09-21T06:28:01.232602+0000","last_unstale":"2024-09-21T06:28:06.567019+0000","last_undegraded":"2024-09-21T06:28:06.567019+0000","last_fullsized":"2024-09-21T06:28:06.567019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:34.157716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276884+0000","last_change":"2024-09-21T06:28:00.218229+0000","last_active":"2024-09-21T06:28:06.276884+0000","last_peered":"2024-09-21T06:28:06.276884+0000","last_clean":"2024-09-21T06:28:06.276884+0000","last_became_active":"2024-09-21T06:28:00.217635+0000","last_became_peered":"2024-09-21T06:28:00.217635+0000","last_unstale":"2024-09-21T06:28:06.276884+0000","last_undegraded":"2024-09-21T06:28:06.276884+0000","last_fullsized":"2024-09-21T06:28:06.276884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271664+0000","last_change":"2024-09-21T06:28:05.277550+0000","last_active":"2024-09-21T06:28:06.271664+0000","last_peered":"2024-09-21T06:28:06.271664+0000","last_clean":"2024-09-21T06:28:06.271664+0000","last_became_active":"2024-09-21T06:28:05.276077+0000","last_became_peered":"2024-09-21T06:28:05.276077+0000","last_unstale":"2024-09-21T06:28:06.271664+0000","last_undegraded":"2024-09-21T06:28:06.271664+0000","last_fullsized":"2024-09-21T06:28:06.271664+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.300870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918857+0000","last_change":"2024-09-21T06:28:01.232773+0000","last_active":"2024-09-21T06:28:06.918857+0000","last_peered":"2024-09-21T06:28:06.918857+0000","last_clean":"2024-09-21T06:28:06.918857+0000","last_became_active":"2024-09-21T06:28:01.232081+0000","last_became_peered":"2024-09-21T06:28:01.232081+0000","last_unstale":"2024-09-21T06:28:06.918857+0000","last_undegraded":"2024-09-21T06:28:06.918857+0000","last_fullsized":"2024-09-21T06:28:06.918857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:54.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917778+0000","last_change":"2024-09-21T06:28:00.234206+0000","last_active":"2024-09-21T06:28:06.917778+0000","last_peered":"2024-09-21T06:28:06.917778+0000","last_clean":"2024-09-21T06:28:06.917778+0000","last_became_active":"2024-09-21T06:28:00.233696+0000","last_became_peered":"2024-09-21T06:28:00.233696+0000","last_unstale":"2024-09-21T06:28:06.917778+0000","last_undegraded":"2024-09-21T06:28:06.917778+0000","last_fullsized":"2024-09-21T06:28:06.917778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275902+0000","last_change":"2024-09-21T06:28:05.283000+0000","last_active":"2024-09-21T06:28:06.275902+0000","last_peered":"2024-09-21T06:28:06.275902+0000","last_clean":"2024-09-21T06:28:06.275902+0000","last_became_active":"2024-09-21T06:28:05.282638+0000","last_became_peered":"2024-09-21T06:28:05.282638+0000","last_unstale":"2024-09-21T06:28:06.275902+0000","last_undegraded":"2024-09-21T06:28:06.275902+0000","last_fullsized":"2024-09-21T06:28:06.275902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271814+0000","last_change":"2024-09-21T06:28:01.221987+0000","last_active":"2024-09-21T06:28:06.271814+0000","last_peered":"2024-09-21T06:28:06.271814+0000","last_clean":"2024-09-21T06:28:06.271814+0000","last_became_active":"2024-09-21T06:28:01.221678+0000","last_became_peered":"2024-09-21T06:28:01.221678+0000","last_unstale":"2024-09-21T06:28:06.271814+0000","last_undegraded":"2024-09-21T06:28:06.271814+0000","last_fullsized":"2024-09-21T06:28:06.271814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:20.730550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919718+0000","last_change":"2024-09-21T06:28:00.202531+0000","last_active":"2024-09-21T06:28:06.919718+0000","last_peered":"2024-09-21T06:28:06.919718+0000","last_clean":"2024-09-21T06:28:06.919718+0000","last_became_active":"2024-09-21T06:28:00.202126+0000","last_became_peered":"2024-09-21T06:28:00.202126+0000","last_unstale":"2024-09-21T06:28:06.919718+0000","last_undegraded":"2024-09-21T06:28:06.919718+0000","last_fullsized":"2024-09-21T06:28:06.919718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274058+0000","last_change":"2024-09-21T06:28:05.278157+0000","last_active":"2024-09-21T06:28:06.274058+0000","last_peered":"2024-09-21T06:28:06.274058+0000","last_clean":"2024-09-21T06:28:06.274058+0000","last_became_active":"2024-09-21T06:28:05.277532+0000","last_became_peered":"2024-09-21T06:28:05.277532+0000","last_unstale":"2024-09-21T06:28:06.274058+0000","last_undegraded":"2024-09-21T06:28:06.274058+0000","last_fullsized":"2024-09-21T06:28:06.274058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.737605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274991+0000","last_change":"2024-09-21T06:28:05.278200+0000","last_active":"2024-09-21T06:28:06.274991+0000","last_peered":"2024-09-21T06:28:06.274991+0000","last_clean":"2024-09-21T06:28:06.274991+0000","last_became_active":"2024-09-21T06:28:05.277575+0000","last_became_peered":"2024-09-21T06:28:05.277575+0000","last_unstale":"2024-09-21T06:28:06.274991+0000","last_undegraded":"2024-09-21T06:28:06.274991+0000","last_fullsized":"2024-09-21T06:28:06.274991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:05.311819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273908+0000","last_change":"2024-09-21T06:28:01.228981+0000","last_active":"2024-09-21T06:28:06.273908+0000","last_peered":"2024-09-21T06:28:06.273908+0000","last_clean":"2024-09-21T06:28:06.273908+0000","last_became_active":"2024-09-21T06:28:01.222592+0000","last_became_peered":"2024-09-21T06:28:01.222592+0000","last_unstale":"2024-09-21T06:28:06.273908+0000","last_undegraded":"2024-09-21T06:28:06.273908+0000","last_fullsized":"2024-09-21T06:28:06.273908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566966+0000","last_change":"2024-09-21T06:28:00.214521+0000","last_active":"2024-09-21T06:28:06.566966+0000","last_peered":"2024-09-21T06:28:06.566966+0000","last_clean":"2024-09-21T06:28:06.566966+0000","last_became_active":"2024-09-21T06:28:00.212408+0000","last_became_peered":"2024-09-21T06:28:00.212408+0000","last_unstale":"2024-09-21T06:28:06.566966+0000","last_undegraded":"2024-09-21T06:28:06.566966+0000","last_fullsized":"2024-09-21T06:28:06.566966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567614+0000","last_change":"2024-09-21T06:28:05.265492+0000","last_active":"2024-09-21T06:28:06.567614+0000","last_peered":"2024-09-21T06:28:06.567614+0000","last_clean":"2024-09-21T06:28:06.567614+0000","last_became_active":"2024-09-21T06:28:05.264988+0000","last_became_peered":"2024-09-21T06:28:05.264988+0000","last_unstale":"2024-09-21T06:28:06.567614+0000","last_undegraded":"2024-09-21T06:28:06.567614+0000","last_fullsized":"2024-09-21T06:28:06.567614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:02.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917735+0000","last_change":"2024-09-21T06:28:01.228392+0000","last_active":"2024-09-21T06:28:06.917735+0000","last_peered":"2024-09-21T06:28:06.917735+0000","last_clean":"2024-09-21T06:28:06.917735+0000","last_became_active":"2024-09-21T06:28:01.228163+0000","last_became_peered":"2024-09-21T06:28:01.228163+0000","last_unstale":"2024-09-21T06:28:06.917735+0000","last_undegraded":"2024-09-21T06:28:06.917735+0000","last_fullsized":"2024-09-21T06:28:06.917735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:24.791034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918738+0000","last_change":"2024-09-21T06:28:00.228862+0000","last_active":"2024-09-21T06:28:06.918738+0000","last_peered":"2024-09-21T06:28:06.918738+0000","last_clean":"2024-09-21T06:28:06.918738+0000","last_became_active":"2024-09-21T06:28:00.228490+0000","last_became_peered":"2024-09-21T06:28:00.228490+0000","last_unstale":"2024-09-21T06:28:06.918738+0000","last_undegraded":"2024-09-21T06:28:06.918738+0000","last_fullsized":"2024-09-21T06:28:06.918738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:09:00.810925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567138+0000","last_change":"2024-09-21T06:28:00.234315+0000","last_active":"2024-09-21T06:28:06.567138+0000","last_peered":"2024-09-21T06:28:06.567138+0000","last_clean":"2024-09-21T06:28:06.567138+0000","last_became_active":"2024-09-21T06:28:00.233567+0000","last_became_peered":"2024-09-21T06:28:00.233567+0000","last_unstale":"2024-09-21T06:28:06.567138+0000","last_undegraded":"2024-09-21T06:28:06.567138+0000","last_fullsized":"2024-09-21T06:28:06.567138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274635+0000","last_change":"2024-09-21T06:28:01.236968+0000","last_active":"2024-09-21T06:28:06.274635+0000","last_peered":"2024-09-21T06:28:06.274635+0000","last_clean":"2024-09-21T06:28:06.274635+0000","last_became_active":"2024-09-21T06:28:01.236577+0000","last_became_peered":"2024-09-21T06:28:01.236577+0000","last_unstale":"2024-09-21T06:28:06.274635+0000","last_undegraded":"2024-09-21T06:28:06.274635+0000","last_fullsized":"2024-09-21T06:28:06.274635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:01.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274854+0000","last_change":"2024-09-21T06:28:05.271567+0000","last_active":"2024-09-21T06:28:06.274854+0000","last_peered":"2024-09-21T06:28:06.274854+0000","last_clean":"2024-09-21T06:28:06.274854+0000","last_became_active":"2024-09-21T06:28:05.271160+0000","last_became_peered":"2024-09-21T06:28:05.271160+0000","last_unstale":"2024-09-21T06:28:06.274854+0000","last_undegraded":"2024-09-21T06:28:06.274854+0000","last_fullsized":"2024-09-21T06:28:06.274854+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:02.495413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918355+0000","last_change":"2024-09-21T06:28:00.215612+0000","last_active":"2024-09-21T06:28:06.918355+0000","last_peered":"2024-09-21T06:28:06.918355+0000","last_clean":"2024-09-21T06:28:06.918355+0000","last_became_active":"2024-09-21T06:28:00.214929+0000","last_became_peered":"2024-09-21T06:28:00.214929+0000","last_unstale":"2024-09-21T06:28:06.918355+0000","last_undegraded":"2024-09-21T06:28:06.918355+0000","last_fullsized":"2024-09-21T06:28:06.918355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:41.140041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918598+0000","last_change":"2024-09-21T06:28:01.234866+0000","last_active":"2024-09-21T06:28:06.918598+0000","last_peered":"2024-09-21T06:28:06.918598+0000","last_clean":"2024-09-21T06:28:06.918598+0000","last_became_active":"2024-09-21T06:28:01.234568+0000","last_became_peered":"2024-09-21T06:28:01.234568+0000","last_unstale":"2024-09-21T06:28:06.918598+0000","last_undegraded":"2024-09-21T06:28:06.918598+0000","last_fullsized":"2024-09-21T06:28:06.918598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:00.807517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276818+0000","last_change":"2024-09-21T06:28:05.287452+0000","last_active":"2024-09-21T06:28:06.276818+0000","last_peered":"2024-09-21T06:28:06.276818+0000","last_clean":"2024-09-21T06:28:06.276818+0000","last_became_active":"2024-09-21T06:28:05.286963+0000","last_became_peered":"2024-09-21T06:28:05.286963+0000","last_unstale":"2024-09-21T06:28:06.276818+0000","last_undegraded":"2024-09-21T06:28:06.276818+0000","last_fullsized":"2024-09-21T06:28:06.276818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:03.321463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271084+0000","last_change":"2024-09-21T06:28:01.232705+0000","last_active":"2024-09-21T06:28:06.271084+0000","last_peered":"2024-09-21T06:28:06.271084+0000","last_clean":"2024-09-21T06:28:06.271084+0000","last_became_active":"2024-09-21T06:28:01.232305+0000","last_became_peered":"2024-09-21T06:28:01.232305+0000","last_unstale":"2024-09-21T06:28:06.271084+0000","last_undegraded":"2024-09-21T06:28:06.271084+0000","last_fullsized":"2024-09-21T06:28:06.271084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275922+0000","last_change":"2024-09-21T06:28:01.235529+0000","last_active":"2024-09-21T06:28:06.275922+0000","last_peered":"2024-09-21T06:28:06.275922+0000","last_clean":"2024-09-21T06:28:06.275922+0000","last_became_active":"2024-09-21T06:28:01.235170+0000","last_became_peered":"2024-09-21T06:28:01.235170+0000","last_unstale":"2024-09-21T06:28:06.275922+0000","last_undegraded":"2024-09-21T06:28:06.275922+0000","last_fullsized":"2024-09-21T06:28:06.275922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566964+0000","last_change":"2024-09-21T06:28:01.242450+0000","last_active":"2024-09-21T06:28:06.566964+0000","last_peered":"2024-09-21T06:28:06.566964+0000","last_clean":"2024-09-21T06:28:06.566964+0000","last_became_active":"2024-09-21T06:28:01.242078+0000","last_became_peered":"2024-09-21T06:28:01.242078+0000","last_unstale":"2024-09-21T06:28:06.566964+0000","last_undegraded":"2024-09-21T06:28:06.566964+0000","last_fullsized":"2024-09-21T06:28:06.566964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:58.450653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918263+0000","last_change":"2024-09-21T06:28:01.230631+0000","last_active":"2024-09-21T06:28:06.918263+0000","last_peered":"2024-09-21T06:28:06.918263+0000","last_clean":"2024-09-21T06:28:06.918263+0000","last_became_active":"2024-09-21T06:28:01.223844+0000","last_became_peered":"2024-09-21T06:28:01.223844+0000","last_unstale":"2024-09-21T06:28:06.918263+0000","last_undegraded":"2024-09-21T06:28:06.918263+0000","last_fullsized":"2024-09-21T06:28:06.918263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:28.649005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919766+0000","last_change":"2024-09-21T06:28:01.248984+0000","last_active":"2024-09-21T06:28:06.919766+0000","last_peered":"2024-09-21T06:28:06.919766+0000","last_clean":"2024-09-21T06:28:06.919766+0000","last_became_active":"2024-09-21T06:28:01.248639+0000","last_became_peered":"2024-09-21T06:28:01.248639+0000","last_unstale":"2024-09-21T06:28:06.919766+0000","last_undegraded":"2024-09-21T06:28:06.919766+0000","last_fullsized":"2024-09-21T06:28:06.919766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.659556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919689+0000","last_change":"2024-09-21T06:28:01.226847+0000","last_active":"2024-09-21T06:28:06.919689+0000","last_peered":"2024-09-21T06:28:06.919689+0000","last_clean":"2024-09-21T06:28:06.919689+0000","last_became_active":"2024-09-21T06:28:01.226536+0000","last_became_peered":"2024-09-21T06:28:01.226536+0000","last_unstale":"2024-09-21T06:28:06.919689+0000","last_undegraded":"2024-09-21T06:28:06.919689+0000","last_fullsized":"2024-09-21T06:28:06.919689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271732+0000","last_change":"2024-09-21T06:28:01.223972+0000","last_active":"2024-09-21T06:28:06.271732+0000","last_peered":"2024-09-21T06:28:06.271732+0000","last_clean":"2024-09-21T06:28:06.271732+0000","last_became_active":"2024-09-21T06:28:01.223601+0000","last_became_peered":"2024-09-21T06:28:01.223601+0000","last_unstale":"2024-09-21T06:28:06.271732+0000","last_undegraded":"2024-09-21T06:28:06.271732+0000","last_fullsized":"2024-09-21T06:28:06.271732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918404+0000","last_change":"2024-09-21T06:28:01.233118+0000","last_active":"2024-09-21T06:28:06.918404+0000","last_peered":"2024-09-21T06:28:06.918404+0000","last_clean":"2024-09-21T06:28:06.918404+0000","last_became_active":"2024-09-21T06:28:01.232482+0000","last_became_peered":"2024-09-21T06:28:01.232482+0000","last_unstale":"2024-09-21T06:28:06.918404+0000","last_undegraded":"2024-09-21T06:28:06.918404+0000","last_fullsized":"2024-09-21T06:28:06.918404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:39.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566851+0000","last_change":"2024-09-21T06:28:01.243008+0000","last_active":"2024-09-21T06:28:06.566851+0000","last_peered":"2024-09-21T06:28:06.566851+0000","last_clean":"2024-09-21T06:28:06.566851+0000","last_became_active":"2024-09-21T06:28:01.242713+0000","last_became_peered":"2024-09-21T06:28:01.242713+0000","last_unstale":"2024-09-21T06:28:06.566851+0000","last_undegraded":"2024-09-21T06:28:06.566851+0000","last_fullsized":"2024-09-21T06:28:06.566851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:17.147886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917756+0000","last_change":"2024-09-21T06:28:01.216205+0000","last_active":"2024-09-21T06:28:06.917756+0000","last_peered":"2024-09-21T06:28:06.917756+0000","last_clean":"2024-09-21T06:28:06.917756+0000","last_became_active":"2024-09-21T06:28:01.215869+0000","last_became_peered":"2024-09-21T06:28:01.215869+0000","last_unstale":"2024-09-21T06:28:06.917756+0000","last_undegraded":"2024-09-21T06:28:06.917756+0000","last_fullsized":"2024-09-21T06:28:06.917756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.725175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.246071+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.245739+0000","last_became_peered":"2024-09-21T06:28:01.245739+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274517+0000","last_change":"2024-09-21T06:28:01.232435+0000","last_active":"2024-09-21T06:28:06.274517+0000","last_peered":"2024-09-21T06:28:06.274517+0000","last_clean":"2024-09-21T06:28:06.274517+0000","last_became_active":"2024-09-21T06:28:01.229739+0000","last_became_peered":"2024-09-21T06:28:01.229739+0000","last_unstale":"2024-09-21T06:28:06.274517+0000","last_undegraded":"2024-09-21T06:28:06.274517+0000","last_fullsized":"2024-09-21T06:28:06.274517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:59.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917974+0000","last_change":"2024-09-21T06:28:01.242252+0000","last_active":"2024-09-21T06:28:06.917974+0000","last_peered":"2024-09-21T06:28:06.917974+0000","last_clean":"2024-09-21T06:28:06.917974+0000","last_became_active":"2024-09-21T06:28:01.241926+0000","last_became_peered":"2024-09-21T06:28:01.241926+0000","last_unstale":"2024-09-21T06:28:06.917974+0000","last_undegraded":"2024-09-21T06:28:06.917974+0000","last_fullsized":"2024-09-21T06:28:06.917974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:31.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567677+0000","last_change":"2024-09-21T06:28:01.232426+0000","last_active":"2024-09-21T06:28:06.567677+0000","last_peered":"2024-09-21T06:28:06.567677+0000","last_clean":"2024-09-21T06:28:06.567677+0000","last_became_active":"2024-09-21T06:28:01.232067+0000","last_became_peered":"2024-09-21T06:28:01.232067+0000","last_unstale":"2024-09-21T06:28:06.567677+0000","last_undegraded":"2024-09-21T06:28:06.567677+0000","last_fullsized":"2024-09-21T06:28:06.567677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:28.274577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277142+0000","last_change":"2024-09-21T06:28:01.247978+0000","last_active":"2024-09-21T06:28:06.277142+0000","last_peered":"2024-09-21T06:28:06.277142+0000","last_clean":"2024-09-21T06:28:06.277142+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.277142+0000","last_undegraded":"2024-09-21T06:28:06.277142+0000","last_fullsized":"2024-09-21T06:28:06.277142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567819+0000","last_change":"2024-09-21T06:28:01.236566+0000","last_active":"2024-09-21T06:28:06.567819+0000","last_peered":"2024-09-21T06:28:06.567819+0000","last_clean":"2024-09-21T06:28:06.567819+0000","last_became_active":"2024-09-21T06:28:01.236209+0000","last_became_peered":"2024-09-21T06:28:01.236209+0000","last_unstale":"2024-09-21T06:28:06.567819+0000","last_undegraded":"2024-09-21T06:28:06.567819+0000","last_fullsized":"2024-09-21T06:28:06.567819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:55.410838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566732+0000","last_change":"2024-09-21T06:28:01.227863+0000","last_active":"2024-09-21T06:28:06.566732+0000","last_peered":"2024-09-21T06:28:06.566732+0000","last_clean":"2024-09-21T06:28:06.566732+0000","last_became_active":"2024-09-21T06:28:01.227553+0000","last_became_peered":"2024-09-21T06:28:01.227553+0000","last_unstale":"2024-09-21T06:28:06.566732+0000","last_undegraded":"2024-09-21T06:28:06.566732+0000","last_fullsized":"2024-09-21T06:28:06.566732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:51.034026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917669+0000","last_change":"2024-09-21T06:28:01.242950+0000","last_active":"2024-09-21T06:28:06.917669+0000","last_peered":"2024-09-21T06:28:06.917669+0000","last_clean":"2024-09-21T06:28:06.917669+0000","last_became_active":"2024-09-21T06:28:01.242715+0000","last_became_peered":"2024-09-21T06:28:01.242715+0000","last_unstale":"2024-09-21T06:28:06.917669+0000","last_undegraded":"2024-09-21T06:28:06.917669+0000","last_fullsized":"2024-09-21T06:28:06.917669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:12.827134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919061+0000","last_change":"2024-09-21T06:28:01.231009+0000","last_active":"2024-09-21T06:28:06.919061+0000","last_peered":"2024-09-21T06:28:06.919061+0000","last_clean":"2024-09-21T06:28:06.919061+0000","last_became_active":"2024-09-21T06:28:01.230706+0000","last_became_peered":"2024-09-21T06:28:01.230706+0000","last_unstale":"2024-09-21T06:28:06.919061+0000","last_undegraded":"2024-09-21T06:28:06.919061+0000","last_fullsized":"2024-09-21T06:28:06.919061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:38.274224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919349+0000","last_change":"2024-09-21T06:28:01.235786+0000","last_active":"2024-09-21T06:28:06.919349+0000","last_peered":"2024-09-21T06:28:06.919349+0000","last_clean":"2024-09-21T06:28:06.919349+0000","last_became_active":"2024-09-21T06:28:01.235551+0000","last_became_peered":"2024-09-21T06:28:01.235551+0000","last_unstale":"2024-09-21T06:28:06.919349+0000","last_undegraded":"2024-09-21T06:28:06.919349+0000","last_fullsized":"2024-09-21T06:28:06.919349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:00.237301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917934+0000","last_change":"2024-09-21T06:28:01.242543+0000","last_active":"2024-09-21T06:28:06.917934+0000","last_peered":"2024-09-21T06:28:06.917934+0000","last_clean":"2024-09-21T06:28:06.917934+0000","last_became_active":"2024-09-21T06:28:01.242225+0000","last_became_peered":"2024-09-21T06:28:01.242225+0000","last_unstale":"2024-09-21T06:28:06.917934+0000","last_undegraded":"2024-09-21T06:28:06.917934+0000","last_fullsized":"2024-09-21T06:28:06.917934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:39.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918078+0000","last_change":"2024-09-21T06:28:01.237214+0000","last_active":"2024-09-21T06:28:06.918078+0000","last_peered":"2024-09-21T06:28:06.918078+0000","last_clean":"2024-09-21T06:28:06.918078+0000","last_became_active":"2024-09-21T06:28:01.235938+0000","last_became_peered":"2024-09-21T06:28:01.235938+0000","last_unstale":"2024-09-21T06:28:06.918078+0000","last_undegraded":"2024-09-21T06:28:06.918078+0000","last_fullsized":"2024-09-21T06:28:06.918078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.564737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917663+0000","last_change":"2024-09-21T06:28:01.251436+0000","last_active":"2024-09-21T06:28:06.917663+0000","last_peered":"2024-09-21T06:28:06.917663+0000","last_clean":"2024-09-21T06:28:06.917663+0000","last_became_active":"2024-09-21T06:28:01.250562+0000","last_became_peered":"2024-09-21T06:28:01.250562+0000","last_unstale":"2024-09-21T06:28:06.917663+0000","last_undegraded":"2024-09-21T06:28:06.917663+0000","last_fullsized":"2024-09-21T06:28:06.917663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:58.692504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919694+0000","last_change":"2024-09-21T06:28:01.226763+0000","last_active":"2024-09-21T06:28:06.919694+0000","last_peered":"2024-09-21T06:28:06.919694+0000","last_clean":"2024-09-21T06:28:06.919694+0000","last_became_active":"2024-09-21T06:28:01.226438+0000","last_became_peered":"2024-09-21T06:28:01.226438+0000","last_unstale":"2024-09-21T06:28:06.919694+0000","last_undegraded":"2024-09-21T06:28:06.919694+0000","last_fullsized":"2024-09-21T06:28:06.919694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917570+0000","last_change":"2024-09-21T06:28:01.242596+0000","last_active":"2024-09-21T06:28:06.917570+0000","last_peered":"2024-09-21T06:28:06.917570+0000","last_clean":"2024-09-21T06:28:06.917570+0000","last_became_active":"2024-09-21T06:28:01.242205+0000","last_became_peered":"2024-09-21T06:28:01.242205+0000","last_unstale":"2024-09-21T06:28:06.917570+0000","last_undegraded":"2024-09-21T06:28:06.917570+0000","last_fullsized":"2024-09-21T06:28:06.917570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:44.145821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917382+0000","last_change":"2024-09-21T06:28:01.249034+0000","last_active":"2024-09-21T06:28:06.917382+0000","last_peered":"2024-09-21T06:28:06.917382+0000","last_clean":"2024-09-21T06:28:06.917382+0000","last_became_active":"2024-09-21T06:28:01.248727+0000","last_became_peered":"2024-09-21T06:28:01.248727+0000","last_unstale":"2024-09-21T06:28:06.917382+0000","last_undegraded":"2024-09-21T06:28:06.917382+0000","last_fullsized":"2024-09-21T06:28:06.917382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271467+0000","last_change":"2024-09-21T06:28:01.215764+0000","last_active":"2024-09-21T06:28:06.271467+0000","last_peered":"2024-09-21T06:28:06.271467+0000","last_clean":"2024-09-21T06:28:06.271467+0000","last_became_active":"2024-09-21T06:28:01.215412+0000","last_became_peered":"2024-09-21T06:28:01.215412+0000","last_unstale":"2024-09-21T06:28:06.271467+0000","last_undegraded":"2024-09-21T06:28:06.271467+0000","last_fullsized":"2024-09-21T06:28:06.271467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917795+0000","last_change":"2024-09-21T06:28:01.253480+0000","last_active":"2024-09-21T06:28:06.917795+0000","last_peered":"2024-09-21T06:28:06.917795+0000","last_clean":"2024-09-21T06:28:06.917795+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.917795+0000","last_undegraded":"2024-09-21T06:28:06.917795+0000","last_fullsized":"2024-09-21T06:28:06.917795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:51.213607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271436+0000","last_change":"2024-09-21T06:28:01.231505+0000","last_active":"2024-09-21T06:28:06.271436+0000","last_peered":"2024-09-21T06:28:06.271436+0000","last_clean":"2024-09-21T06:28:06.271436+0000","last_became_active":"2024-09-21T06:28:01.223703+0000","last_became_peered":"2024-09-21T06:28:01.223703+0000","last_unstale":"2024-09-21T06:28:06.271436+0000","last_undegraded":"2024-09-21T06:28:06.271436+0000","last_fullsized":"2024-09-21T06:28:06.271436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918618+0000","last_change":"2024-09-21T06:28:01.236472+0000","last_active":"2024-09-21T06:28:06.918618+0000","last_peered":"2024-09-21T06:28:06.918618+0000","last_clean":"2024-09-21T06:28:06.918618+0000","last_became_active":"2024-09-21T06:28:01.221503+0000","last_became_peered":"2024-09-21T06:28:01.221503+0000","last_unstale":"2024-09-21T06:28:06.918618+0000","last_undegraded":"2024-09-21T06:28:06.918618+0000","last_fullsized":"2024-09-21T06:28:06.918618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:35.021658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918538+0000","last_change":"2024-09-21T06:28:01.227458+0000","last_active":"2024-09-21T06:28:06.918538+0000","last_peered":"2024-09-21T06:28:06.918538+0000","last_clean":"2024-09-21T06:28:06.918538+0000","last_became_active":"2024-09-21T06:28:01.227145+0000","last_became_peered":"2024-09-21T06:28:01.227145+0000","last_unstale":"2024-09-21T06:28:06.918538+0000","last_undegraded":"2024-09-21T06:28:06.918538+0000","last_fullsized":"2024-09-21T06:28:06.918538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:24.909945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918495+0000","last_change":"2024-09-21T06:28:01.233932+0000","last_active":"2024-09-21T06:28:06.918495+0000","last_peered":"2024-09-21T06:28:06.918495+0000","last_clean":"2024-09-21T06:28:06.918495+0000","last_became_active":"2024-09-21T06:28:01.233617+0000","last_became_peered":"2024-09-21T06:28:01.233617+0000","last_unstale":"2024-09-21T06:28:06.918495+0000","last_undegraded":"2024-09-21T06:28:06.918495+0000","last_fullsized":"2024-09-21T06:28:06.918495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:27.585260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:28:21.021 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:28:21.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T06:28:21.023 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.285+0000 7fdf0be00640 1 -- 172.21.15.37:0/1825971550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c15c8c0 msgr2=0x7fdf0c0b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.285+0000 7fdf0be00640 1 --2- 172.21.15.37:0/1825971550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c0b5570 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0c9070 tx=0x7fdef402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.285+0000 7fdf0be00640 1 -- 172.21.15.37:0/1825971550 shutdown_connections 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.285+0000 7fdf0be00640 1 --2- 172.21.15.37:0/1825971550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c0b5570 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.286+0000 7fdf0be00640 1 --2- 172.21.15.37:0/1825971550 >> v2:172.21.15.150:3300/0 conn(0x7fdf0c0cd050 0x7fdf0c0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.286+0000 7fdf0be00640 1 -- 172.21.15.37:0/1825971550 >> 172.21.15.37:0/1825971550 conn(0x7fdf0c0d2620 msgr2=0x7fdf0c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:21.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.286+0000 7fdf0be00640 1 -- 172.21.15.37:0/1825971550 shutdown_connections 2024-09-21T06:28:21.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.286+0000 7fdf0be00640 1 -- 172.21.15.37:0/1825971550 wait complete. 2024-09-21T06:28:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.288+0000 7fdf0be00640 1 Processor -- start 2024-09-21T06:28:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.288+0000 7fdf0be00640 1 -- start start 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf0c0b5ab0 0x7fdf0c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c0719a0 con 0x7fdf0c0cd050 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c071820 con 0x7fdf0c15c8c0 2024-09-21T06:28:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c0b5ec0 con 0x7fdf0c0b5ab0 2024-09-21T06:28:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42186/0 (socket says 172.21.15.37:42186) 2024-09-21T06:28:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 -- 172.21.15.37:0/48606991 learned_addr learned my addr 172.21.15.37:0/48606991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0aa00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf0c0b5ab0 0x7fdf0c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf0b400640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 -- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf0c0b5ab0 msgr2=0x7fdf0c0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf0c0b5ab0 0x7fdf0c0c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 -- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c15c8c0 msgr2=0x7fdf0c15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c15ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.290+0000 7fdf03e00640 1 -- 172.21.15.37:0/48606991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c06f1e0 con 0x7fdf0c0cd050 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf03e00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009850 tx=0x7fdef40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef403e030 con 0x7fdf0c0cd050 2024-09-21T06:28:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdef4003d90 con 0x7fdf0c0cd050 2024-09-21T06:28:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4035040 con 0x7fdf0c0cd050 2024-09-21T06:28:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c06f4c0 con 0x7fdf0c0cd050 2024-09-21T06:28:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.291+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c0c7160 con 0x7fdf0c0cd050 2024-09-21T06:28:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.292+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c075780 con 0x7fdf0c0cd050 2024-09-21T06:28:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.293+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdef4041030 con 0x7fdf0c0cd050 2024-09-21T06:28:21.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.295+0000 7fdf02a00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.295+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdef40bb760 con 0x7fdf0c0cd050 2024-09-21T06:28:21.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.295+0000 7fdf0aa00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.296+0000 7fdf0aa00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 0x7fdee807b0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009ac0 tx=0x7fdefc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:21.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.300+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdef4083850 con 0x7fdf0c0cd050 2024-09-21T06:28:21.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.501+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fdf0c0b5030 con 0x7fdf0c0cd050 2024-09-21T06:28:21.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.502+0000 7fdf02a00640 1 -- 172.21.15.37:0/48606991 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fdef4083150 con 0x7fdf0c0cd050 2024-09-21T06:28:21.504 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 msgr2=0x7fdee807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 0x7fdee807b0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009ac0 tx=0x7fdefc005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 msgr2=0x7fdf0c1f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009850 tx=0x7fdef40038a0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 shutdown_connections 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0c0cd050 0x7fdf0c1f4fd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c15c8c0 0x7fdf0c15ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 --2- 172.21.15.37:0/48606991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf0c0b5ab0 0x7fdf0c0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 >> 172.21.15.37:0/48606991 conn(0x7fdf0c0d2620 msgr2=0x7fdf0c15a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 shutdown_connections 2024-09-21T06:28:21.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.504+0000 7fdf0be00640 1 -- 172.21.15.37:0/48606991 wait complete. 2024-09-21T06:28:21.527 INFO:teuthology.orchestra.run.smithi037.stdout:[{"version":31,"timestamp":"2024-09-21T06:27:47.549188+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"2888.132894"}]}] 2024-09-21T06:28:21.527 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T06:28:21.527 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T06:28:21.528 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T06:28:21.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T06:28:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 -- 172.21.15.37:0/2197244858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e40b9e40 msgr2=0x7f43e41622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 --2- 172.21.15.37:0/2197244858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e40b9e40 0x7f43e41622b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f43d8005fd0 tx=0x7f43d802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 -- 172.21.15.37:0/2197244858 shutdown_connections 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 --2- 172.21.15.37:0/2197244858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e406d9c0 0x7f43e4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 --2- 172.21.15.37:0/2197244858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e40b9e40 0x7f43e41622b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 -- 172.21.15.37:0/2197244858 >> 172.21.15.37:0/2197244858 conn(0x7f43e40d1930 msgr2=0x7f43e40d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 -- 172.21.15.37:0/2197244858 shutdown_connections 2024-09-21T06:28:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.752+0000 7f43ea000640 1 -- 172.21.15.37:0/2197244858 wait complete. 2024-09-21T06:28:21.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.754+0000 7f43ea000640 1 Processor -- start 2024-09-21T06:28:21.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.755+0000 7f43ea000640 1 -- start start 2024-09-21T06:28:21.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e40b9e40 0x7f43e41f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e41627f0 0x7f43e41f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4155770 con 0x7f43e406d9c0 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41555f0 con 0x7f43e41627f0 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43ea000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43e41558f0 con 0x7f43e40b9e40 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43e8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43e8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42196/0 (socket says 172.21.15.37:42196) 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43e8c00640 1 -- 172.21.15.37:0/2774539267 learned_addr learned my addr 172.21.15.37:0/2774539267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43e3e00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e40b9e40 0x7f43e41f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.756+0000 7f43e9600640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e41627f0 0x7f43e41f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 -- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e40b9e40 msgr2=0x7f43e41f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e40b9e40 0x7f43e41f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 -- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e41627f0 msgr2=0x7f43e41f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e41627f0 0x7f43e41f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 -- 172.21.15.37:0/2774539267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40c99a0 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.757+0000 7f43e8c00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43d8034040 tx=0x7f43d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.758+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d803e030 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.758+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43d8003db0 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.758+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8035040 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.758+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e40c9c80 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.758+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e40ca240 con 0x7f43e406d9c0 2024-09-21T06:28:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.759+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e4154ea0 con 0x7f43e406d9c0 2024-09-21T06:28:21.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.760+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43d8041030 con 0x7f43e406d9c0 2024-09-21T06:28:21.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.761+0000 7f43e0c00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 0x7f43c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:21.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.761+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f43d80bb6e0 con 0x7f43e406d9c0 2024-09-21T06:28:21.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.761+0000 7f43e3e00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 0x7f43c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:21.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.762+0000 7f43e3e00640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 0x7f43c407b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cd9d0 tx=0x7f43d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:21.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:21.765+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f43d8083180 con 0x7f43e406d9c0 2024-09-21T06:28:22.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.015+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f43e4069380 con 0x7f43e406d9c0 2024-09-21T06:28:22.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.019+0000 7f43e0c00640 1 -- 172.21.15.37:0/2774539267 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194721 (secure 0 0 0) 0x7f43d8087bb0 con 0x7f43e406d9c0 2024-09-21T06:28:22.023 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 msgr2=0x7f43c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 0x7f43c407b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cd9d0 tx=0x7f43d0009040 comp rx=0 tx=0).stop 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 msgr2=0x7f43e414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43d8034040 tx=0x7f43d80038a0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 shutdown_connections 2024-09-21T06:28:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078bc0 0x7f43c407b0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e41627f0 0x7f43e41f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e40b9e40 0x7f43e41f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 --2- 172.21.15.37:0/2774539267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406d9c0 0x7f43e414e670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 >> 172.21.15.37:0/2774539267 conn(0x7f43e40d1930 msgr2=0x7f43e4154730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 shutdown_connections 2024-09-21T06:28:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.023+0000 7f43ea000640 1 -- 172.21.15.37:0/2774539267 wait complete. 2024-09-21T06:28:22.049 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"flags":0,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3586070453},{"type":"v1","addr":"172.21.15.150:6801","nonce":3586070453}]},"active_addr":"172.21.15.150:6801/3586070453","active_change":"2024-09-21T06:27:37.072996+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4099,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":1727308266}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":1615794347}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3830687467}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":490616290}]}]} 2024-09-21T06:28:22.054 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T06:28:22.054 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T06:28:22.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.278+0000 7f2642600640 1 -- 172.21.15.37:0/4261018085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2624004000 msgr2=0x7f2624004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.278+0000 7f2642600640 1 --2- 172.21.15.37:0/4261018085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2624004000 0x7f2624004800 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f263c162210 tx=0x7f2624005830 comp rx=0 tx=0).stop 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.278+0000 7f2641c00640 1 -- 172.21.15.37:0/4261018085 reap_dead start 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.279+0000 7f2642600640 1 -- 172.21.15.37:0/4261018085 shutdown_connections 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.279+0000 7f2642600640 1 -- 172.21.15.37:0/4261018085 >> 172.21.15.37:0/4261018085 conn(0x7f263c0d1930 msgr2=0x7f263c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.279+0000 7f2642600640 1 -- 172.21.15.37:0/4261018085 shutdown_connections 2024-09-21T06:28:22.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.279+0000 7f2642600640 1 -- 172.21.15.37:0/4261018085 wait complete. 2024-09-21T06:28:22.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.281+0000 7f2642600640 1 Processor -- start 2024-09-21T06:28:22.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.281+0000 7f2642600640 1 -- start start 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2624004000 0x7f263c1f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f263c069140 0x7f263c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f263c1622b0 con 0x7f263c0c8e60 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0d3830 con 0x7f2624004000 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.282+0000 7f2642600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f263c162430 con 0x7f263c069140 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42214/0 (socket says 172.21.15.37:42214) 2024-09-21T06:28:22.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f2641200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2624004000 0x7f263c1f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 -- 172.21.15.37:0/4092251981 learned_addr learned my addr 172.21.15.37:0/4092251981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:22.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f2641c00640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f263c069140 0x7f263c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 -- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f263c069140 msgr2=0x7f263c0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f263c069140 0x7f263c0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 -- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2624004000 msgr2=0x7f263c1f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2624004000 0x7f263c1f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.283+0000 7f263be00640 1 -- 172.21.15.37:0/4092251981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c0c9540 con 0x7f263c0c8e60 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f263be00640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f262800c560 tx=0x7f2628006850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628039070 con 0x7f263c0c8e60 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f262802b610 con 0x7f263c0c8e60 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c0c9820 con 0x7f263c0c8e60 2024-09-21T06:28:22.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262802ba60 con 0x7f263c0c8e60 2024-09-21T06:28:22.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.284+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c0c9de0 con 0x7f263c0c8e60 2024-09-21T06:28:22.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.285+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c155690 con 0x7f263c0c8e60 2024-09-21T06:28:22.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.286+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2628044020 con 0x7f263c0c8e60 2024-09-21T06:28:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.288+0000 7f2638c00640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 0x7f261c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.288+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f26280baa40 con 0x7f263c0c8e60 2024-09-21T06:28:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.289+0000 7f2641200640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 0x7f261c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.289+0000 7f2641200640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 0x7f261c07b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2630009c50 tx=0x7f2630005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.293+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f263c163170 con 0x7f263c0c8e60 2024-09-21T06:28:22.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.490+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f263c0c92d0 con 0x7f263c0c8e60 2024-09-21T06:28:22.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.491+0000 7f2638c00640 1 -- 172.21.15.37:0/4092251981 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f2628082bf0 con 0x7f263c0c8e60 2024-09-21T06:28:22.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:22.494 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 msgr2=0x7f261c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 0x7f261c07b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2630009c50 tx=0x7f2630005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 msgr2=0x7f263c0cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f262800c560 tx=0x7f2628006850 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 shutdown_connections 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f261c078b90 0x7f261c07b0a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f263c0c8e60 0x7f263c0cd0e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f263c069140 0x7f263c0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 --2- 172.21.15.37:0/4092251981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2624004000 0x7f263c1f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.493+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 >> 172.21.15.37:0/4092251981 conn(0x7f263c0d1930 msgr2=0x7f263c15fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.494+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 shutdown_connections 2024-09-21T06:28:22.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.494+0000 7f2642600640 1 -- 172.21.15.37:0/4092251981 wait complete. 2024-09-21T06:28:22.518 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T06:28:22.518 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:22.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 -- 172.21.15.37:0/114693272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8001627f0 msgr2=0x7fa80006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 --2- 172.21.15.37:0/114693272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8001627f0 0x7fa80006d2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009830 tx=0x7fa7ec02a510 comp rx=0 tx=0).stop 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 -- 172.21.15.37:0/114693272 shutdown_connections 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 --2- 172.21.15.37:0/114693272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8001627f0 0x7fa80006d2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 --2- 172.21.15.37:0/114693272 >> v2:172.21.15.150:3300/0 conn(0x7fa8000b9e40 0x7fa8001622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 -- 172.21.15.37:0/114693272 >> 172.21.15.37:0/114693272 conn(0x7fa8000d1930 msgr2=0x7fa8000d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 -- 172.21.15.37:0/114693272 shutdown_connections 2024-09-21T06:28:22.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.741+0000 7fa806c00640 1 -- 172.21.15.37:0/114693272 wait complete. 2024-09-21T06:28:22.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.743+0000 7fa806c00640 1 Processor -- start 2024-09-21T06:28:22.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.743+0000 7fa806c00640 1 -- start start 2024-09-21T06:28:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80006d7f0 0x7fa8000c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8000b9e40 0x7fa8000cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001557c0 con 0x7fa8001627f0 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa80006e7f0 con 0x7fa80006d7f0 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa800155940 con 0x7fa8000b9e40 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa804e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8000b9e40 0x7fa8000cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa805800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80006d7f0 0x7fa8000c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42228/0 (socket says 172.21.15.37:42228) 2024-09-21T06:28:22.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.745+0000 7fa806200640 1 -- 172.21.15.37:0/401814299 learned_addr learned my addr 172.21.15.37:0/401814299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:22.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 -- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8000b9e40 msgr2=0x7fa8000cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8000b9e40 0x7fa8000cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 -- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80006d7f0 msgr2=0x7fa8000c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80006d7f0 0x7fa8000c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 -- 172.21.15.37:0/401814299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8000c8fe0 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806200640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800caa0 tx=0x7fa7f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8016020 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7f800ac90 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8012680 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8000c9790 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.746+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80015ab50 con 0x7fa8001627f0 2024-09-21T06:28:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.747+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8001609d0 con 0x7fa8001627f0 2024-09-21T06:28:22.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.749+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7f8019020 con 0x7fa8001627f0 2024-09-21T06:28:22.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.750+0000 7fa7fd600640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 0x7fa7e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.750+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa7f809b670 con 0x7fa8001627f0 2024-09-21T06:28:22.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.750+0000 7fa805800640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 0x7fa7e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.752+0000 7fa805800640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 0x7fa7e007b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009b30 tx=0x7fa7f4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.755+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7f8020020 con 0x7fa8001627f0 2024-09-21T06:28:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.954+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa800151fe0 con 0x7fa8001627f0 2024-09-21T06:28:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.955+0000 7fa7fd600640 1 -- 172.21.15.37:0/401814299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7fa800163170 con 0x7fa8001627f0 2024-09-21T06:28:22.957 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:22.958 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 msgr2=0x7fa7e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 0x7fa7e007b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009b30 tx=0x7fa7f4005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 msgr2=0x7fa8000cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800caa0 tx=0x7fa7f800cf60 comp rx=0 tx=0).stop 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 shutdown_connections 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa7e0078b90 0x7fa7e007b0a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8001627f0 0x7fa8000cc820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8000b9e40 0x7fa8000cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 --2- 172.21.15.37:0/401814299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80006d7f0 0x7fa8000c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 >> 172.21.15.37:0/401814299 conn(0x7fa8000d1930 msgr2=0x7fa8001611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.957+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 shutdown_connections 2024-09-21T06:28:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:22.958+0000 7fa806c00640 1 -- 172.21.15.37:0/401814299 wait complete. 2024-09-21T06:28:22.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T06:28:22.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T06:28:22.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T06:28:22.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T06:28:22.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T06:28:22.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T06:28:22.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T06:28:22.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 -- 172.21.15.37:0/1588717960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80739f0 msgr2=0x7f82b80ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 --2- 172.21.15.37:0/1588717960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80739f0 0x7f82b80ccbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f82b8072640 tx=0x7f82b002bc00 comp rx=0 tx=0).stop 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 -- 172.21.15.37:0/1588717960 shutdown_connections 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 --2- 172.21.15.37:0/1588717960 >> v2:172.21.15.37:3300/0 conn(0x7f82b80704d0 0x7f82b80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 --2- 172.21.15.37:0/1588717960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80739f0 0x7f82b80ccbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.392+0000 7f82c0600640 1 -- 172.21.15.37:0/1588717960 >> 172.21.15.37:0/1588717960 conn(0x7f82b80d0350 msgr2=0x7f82b80d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.393+0000 7f82c0600640 1 -- 172.21.15.37:0/1588717960 shutdown_connections 2024-09-21T06:28:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.393+0000 7f82c0600640 1 -- 172.21.15.37:0/1588717960 wait complete. 2024-09-21T06:28:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.397+0000 7f82c0600640 1 Processor -- start 2024-09-21T06:28:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.397+0000 7f82c0600640 1 -- start start 2024-09-21T06:28:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.397+0000 7f3335600640 1 -- 172.21.15.37:0/631355963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 msgr2=0x7f33300cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.397+0000 7f3335600640 1 --2- 172.21.15.37:0/631355963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 0x7f33300cf7f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3320009860 tx=0x7f332002a530 comp rx=0 tx=0).stop 2024-09-21T06:28:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.398+0000 7f3335600640 1 -- 172.21.15.37:0/631355963 shutdown_connections 2024-09-21T06:28:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.398+0000 7f3335600640 1 --2- 172.21.15.37:0/631355963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 0x7f33300cf7f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.398+0000 7f3335600640 1 -- 172.21.15.37:0/631355963 >> 172.21.15.37:0/631355963 conn(0x7f33300d1960 msgr2=0x7f33300d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.398+0000 7f3335600640 1 -- 172.21.15.37:0/631355963 shutdown_connections 2024-09-21T06:28:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.398+0000 7f3335600640 1 -- 172.21.15.37:0/631355963 wait complete. 2024-09-21T06:28:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82b80704d0 0x7f82b8169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 0x7f82b81f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80cd100 0x7f82b81f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80cf3b0 con 0x7f82b80704d0 2024-09-21T06:28:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82b8073830 con 0x7f82b80cd100 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82c0600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f82b80cf530 con 0x7f82b80739f0 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.400+0000 7f82bd200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 0x7f82b81f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 0x7f82b81f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44118/0 (socket says 172.21.15.37:44118) 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 -- 172.21.15.37:0/1818760827 learned_addr learned my addr 172.21.15.37:0/1818760827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bdc00640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82b80704d0 0x7f82b8169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82be600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80cd100 0x7f82b81f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 -- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80cd100 msgr2=0x7f82b81f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b80cd100 0x7f82b81f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 -- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82b80704d0 msgr2=0x7f82b8169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82b80704d0 0x7f82b8169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bd200640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b820eb10 con 0x7f82b80739f0 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f921ac00640 1 -- 172.21.15.37:0/3356977056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 msgr2=0x7f921415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f921ac00640 1 --2- 172.21.15.37:0/3356977056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921415e5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92140b52c0 tx=0x7f920002a4d0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.401+0000 7f82bdc00640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82b80704d0 0x7f82b8169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 -- 172.21.15.37:0/3356977056 shutdown_connections 2024-09-21T06:28:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f82bd200640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 0x7f82b81f83c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c3a0 tx=0x7f82a800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 --2- 172.21.15.37:0/3356977056 >> v2:172.21.15.150:3300/0 conn(0x7f921406edf0 0x7f921406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 --2- 172.21.15.37:0/3356977056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921415e5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 -- 172.21.15.37:0/3356977056 >> 172.21.15.37:0/3356977056 conn(0x7f92140d05c0 msgr2=0x7f921406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 -- 172.21.15.37:0/3356977056 shutdown_connections 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f921ac00640 1 -- 172.21.15.37:0/3356977056 wait complete. 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8017070 con 0x7f82b80739f0 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b80a9570 con 0x7f82b80739f0 2024-09-21T06:28:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.403+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b820e830 con 0x7f82b80739f0 2024-09-21T06:28:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.403+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a80053c0 con 0x7f82b80739f0 2024-09-21T06:28:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.403+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8003850 con 0x7f82b80739f0 2024-09-21T06:28:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f3335600640 1 Processor -- start 2024-09-21T06:28:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.402+0000 7f3335600640 1 -- start start 2024-09-21T06:28:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.404+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82b80ccbc0 con 0x7f82b80739f0 2024-09-21T06:28:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 0x7f3330152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 0x7f33300c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33301621d0 0x7f333014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f333016e100 con 0x7f3330153e80 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f333016df80 con 0x7f33301621d0 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f3335600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f333016e280 con 0x7f333006d2b0 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f332f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 0x7f33300c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f332f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 0x7f33300c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42294/0 (socket says 172.21.15.37:42294) 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.405+0000 7f332f400640 1 -- 172.21.15.37:0/2858643975 learned_addr learned my addr 172.21.15.37:0/2858643975 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332fe00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 0x7f3330152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332f400640 1 -- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 msgr2=0x7f3330152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332f400640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f333006d2b0 0x7f3330152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332f400640 1 -- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33301621d0 msgr2=0x7f333014b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332f400640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33301621d0 0x7f333014b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.406+0000 7f332f400640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300c8c00 con 0x7f3330153e80 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.407+0000 7f332f400640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 0x7f33300c7ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f332800c730 tx=0x7f332800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.407+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328003b70 con 0x7f3330153e80 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.408+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3328003d10 con 0x7f3330153e80 2024-09-21T06:28:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.408+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328003b70 con 0x7f3330153e80 2024-09-21T06:28:23.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.408+0000 7f3335600640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33300ca060 con 0x7f3330153e80 2024-09-21T06:28:23.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.408+0000 7f921ac00640 1 Processor -- start 2024-09-21T06:28:23.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.409+0000 7f921ac00640 1 -- start start 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.409+0000 7f3335600640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3330150e70 con 0x7f3330153e80 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 -- 172.21.15.37:0/2687219666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 msgr2=0x7f0a18003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 --2- 172.21.15.37:0/2687219666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 0x7f0a18003f70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009b50 tx=0x7f0a18005830 comp rx=0 tx=0).stop 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 -- 172.21.15.37:0/2687219666 shutdown_connections 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 --2- 172.21.15.37:0/2687219666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 0x7f0a18003f70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 -- 172.21.15.37:0/2687219666 >> 172.21.15.37:0/2687219666 conn(0x7f0a340d41e0 msgr2=0x7f0a340d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.410+0000 7f0a33e00640 1 -- 172.21.15.37:0/2687219666 shutdown_connections 2024-09-21T06:28:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.411+0000 7f0a33e00640 1 -- 172.21.15.37:0/2687219666 wait complete. 2024-09-21T06:28:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.411+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82a801a070 con 0x7f82b80739f0 2024-09-21T06:28:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.411+0000 7f921ac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 0x7f92141f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3304001030 con 0x7f3330153e80 2024-09-21T06:28:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f921ac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9214151080 0x7f9214154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f921ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921413c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f921ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f921406f740 con 0x7f921415e1c0 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f921ac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9214070e70 con 0x7f921406edf0 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f921ac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f921406f8c0 con 0x7f9214151080 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f9219800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 0x7f92141f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f9219800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 0x7f92141f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:55100/0 (socket says 172.21.15.37:55100) 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f9219800640 1 -- 172.21.15.37:0/3674294803 learned_addr learned my addr 172.21.15.37:0/3674294803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 -- 172.21.15.37:0/2752842062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c24056da0 msgr2=0x7f0c24057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 --2- 172.21.15.37:0/2752842062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c24056da0 0x7f0c24057c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06c0e0 tx=0x7f0c2c0a62e0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f82b5600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8298078b90 0x7f829807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 -- 172.21.15.37:0/2752842062 shutdown_connections 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 --2- 172.21.15.37:0/2752842062 >> v2:172.21.15.37:3300/0 conn(0x7f0c240e2f40 0x7f0c2403c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 --2- 172.21.15.37:0/2752842062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c24056da0 0x7f0c24057c80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.412+0000 7f0c2b200640 1 -- 172.21.15.37:0/2752842062 >> 172.21.15.37:0/2752842062 conn(0x7f0c2403a5f0 msgr2=0x7f0c2403aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f9219800640 1 -- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9214151080 msgr2=0x7f9214154380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:28:23.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f9219800640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9214151080 0x7f9214154380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f0c2b200640 1 -- 172.21.15.37:0/2752842062 shutdown_connections 2024-09-21T06:28:23.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f9219800640 1 -- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 msgr2=0x7f921413c920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:28:23.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f0c2b200640 1 -- 172.21.15.37:0/2752842062 wait complete. 2024-09-21T06:28:23.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f82a809ab80 con 0x7f82b80739f0 2024-09-21T06:28:23.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f82b5600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f82980822d0 0x7f8298084760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8298084e30 con 0x7f82980822d0 2024-09-21T06:28:23.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82a809b010 con 0x7f82b80739f0 2024-09-21T06:28:23.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f82bdc00640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8298078b90 0x7f829807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f921a200640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921413c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.413+0000 7f9219800640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921413c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.414+0000 7f9219800640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92141548c0 con 0x7f921406edf0 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.414+0000 7f921a200640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921415e1c0 0x7f921413c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.414+0000 7f82be600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f82980822d0 0x7f8298084760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.414+0000 7f9219800640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 0x7f92141f5440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f9200003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f2dc1600640 1 -- 172.21.15.37:0/1446577563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da0004800 msgr2=0x7f2da0004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f2dc1600640 1 --2- 172.21.15.37:0/1446577563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da0004800 0x7f2da0004c30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c6e0 tx=0x7f2da0005830 comp rx=0 tx=0).stop 2024-09-21T06:28:23.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f0a33e00640 1 Processor -- start 2024-09-21T06:28:23.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920003e030 con 0x7f921406edf0 2024-09-21T06:28:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f0a33e00640 1 -- start start 2024-09-21T06:28:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9200003d10 con 0x7f921406edf0 2024-09-21T06:28:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.415+0000 7f82bdc00640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8298078b90 0x7f829807b0a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82b0035000 tx=0x7f82b0004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f332800a850 con 0x7f3330153e80 2024-09-21T06:28:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7ff220800640 1 -- 172.21.15.37:0/2758054875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21806e330 msgr2=0x7ff2180cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7ff220800640 1 --2- 172.21.15.37:0/2758054875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21806e330 0x7ff2180cf290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff214005fd0 tx=0x7ff21402a9f0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7f0c2b200640 1 Processor -- start 2024-09-21T06:28:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7f2dc1600640 1 -- 172.21.15.37:0/1446577563 shutdown_connections 2024-09-21T06:28:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7f2dc1600640 1 --2- 172.21.15.37:0/1446577563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da0004800 0x7f2da0004c30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.416+0000 7f2dc1600640 1 -- 172.21.15.37:0/1446577563 >> 172.21.15.37:0/1446577563 conn(0x7f2dbc0d41e0 msgr2=0x7f2dbc0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f82be600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f82980822d0 0x7f8298084760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 -- 172.21.15.37:0/2758054875 shutdown_connections 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 --2- 172.21.15.37:0/2758054875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21806e330 0x7ff2180cf290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 --2- 172.21.15.37:0/2758054875 >> v2:172.21.15.37:3300/0 conn(0x7ff21815fc50 0x7ff21806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 -- 172.21.15.37:0/2758054875 >> 172.21.15.37:0/2758054875 conn(0x7ff2180b1b90 msgr2=0x7ff2180b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f2dc1600640 1 -- 172.21.15.37:0/1446577563 shutdown_connections 2024-09-21T06:28:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f2dc1600640 1 -- 172.21.15.37:0/1446577563 wait complete. 2024-09-21T06:28:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0c2b200640 1 -- start start 2024-09-21T06:28:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200035040 con 0x7f921406edf0 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9214154b50 con 0x7f921406edf0 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921413d470 con 0x7f921406edf0 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 0x7f0a34150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a34151920 0x7f0a34151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 0x7f0a341fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a340cf6a0 con 0x7f0a34151920 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0a340cf520 con 0x7f0a18003ae0 2024-09-21T06:28:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f0a33e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0a340cf820 con 0x7f0a34152270 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7f3327e00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3310078dc0 0x7f331007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.418+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3328098aa0 con 0x7f3330153e80 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.419+0000 7f3327e00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f3310083150 0x7f3310085640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.419+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3310085d10 con 0x7f3310083150 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.419+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f332805ea20 con 0x7f3330153e80 2024-09-21T06:28:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.419+0000 7f332fe00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3310078dc0 0x7f331007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.418+0000 7f0a32a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 0x7f0a34150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.418+0000 7f0a33400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 0x7f0a341fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.418+0000 7f0a33400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 0x7f0a341fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44212/0 (socket says 172.21.15.37:44212) 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.418+0000 7f0a33400640 1 -- 172.21.15.37:0/2727183136 learned_addr learned my addr 172.21.15.37:0/2727183136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.419+0000 7f0a32000640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a34151920 0x7f0a34151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 0x7f0c240debb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24056da0 0x7f0c2402f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c240e2f40 0x7f0c2402f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24007160 con 0x7f0c24056da0 2024-09-21T06:28:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24006fe0 con 0x7f0c24053c50 2024-09-21T06:28:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0c240072e0 con 0x7f0c240e2f40 2024-09-21T06:28:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== osd.0 v2:172.21.15.37:6808/787824175 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8298084e30 con 0x7f82980822d0 2024-09-21T06:28:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f3334c00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f3310083150 0x7f3310085640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f921413d840 con 0x7f921406edf0 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 -- 172.21.15.37:0/2758054875 shutdown_connections 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.417+0000 7ff220800640 1 -- 172.21.15.37:0/2758054875 wait complete. 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0a33400640 1 -- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 msgr2=0x7f0a34150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0a33400640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a18003ae0 0x7f0a34150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0a33400640 1 -- 172.21.15.37:0/2727183136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a34151920 msgr2=0x7f0a34151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 0x7f0c240debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c2a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c240e2f40 0x7f0c2402f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.420+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 0x7f0c240debb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:55126/0 (socket says 172.21.15.37:55126) 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c29e00640 1 -- 172.21.15.37:0/4171988791 learned_addr learned my addr 172.21.15.37:0/4171988791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0a33400640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a34151920 0x7f0a34151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0a33400640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3415ccd0 con 0x7f0a34152270 2024-09-21T06:28:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0a32000640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a34151920 0x7f0a34151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c29400640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24056da0 0x7f0c2402f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f2dc1600640 1 Processor -- start 2024-09-21T06:28:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c2a800640 1 -- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 msgr2=0x7f0c240debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f332fe00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3310078dc0 0x7f331007b2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3320034000 tx=0x7f33200046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c2a800640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 0x7f0c240debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c2a800640 1 -- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24056da0 msgr2=0x7f0c2402f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c2a800640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24056da0 0x7f0c2402f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c2a800640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2417d110 con 0x7f0c240e2f40 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f0c29e00640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c24053c50 0x7f0c240debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f2dc1600640 1 -- start start 2024-09-21T06:28:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.421+0000 7f3334c00640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f3310083150 0x7f3310085640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7ff220800640 1 Processor -- start 2024-09-21T06:28:23.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0c2a800640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c240e2f40 0x7f0c2402f970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c240e3520 tx=0x7f0c1800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a33400640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 0x7f0a341fcba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a34162180 tx=0x7f0a2c001c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7ff220800640 1 -- start start 2024-09-21T06:28:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011900 con 0x7f0a34152270 2024-09-21T06:28:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.423+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011790 con 0x7f0c240e2f40 2024-09-21T06:28:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.423+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c18017070 con 0x7f0c240e2f40 2024-09-21T06:28:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.423+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011cc0 con 0x7f0c240e2f40 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.423+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3310085d10 con 0x7f3310083150 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a2c018070 con 0x7f0a34152270 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011d20 con 0x7f0a34152270 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a340ca790 con 0x7f0a34152270 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.422+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a340c9cb0 con 0x7f0a34152270 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.423+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a3415e6e0 con 0x7f0a34152270 2024-09-21T06:28:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.424+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9200041030 con 0x7f921406edf0 2024-09-21T06:28:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.424+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2417d3f0 con 0x7f0c240e2f40 2024-09-21T06:28:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.424+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2417d9b0 con 0x7f0c240e2f40 2024-09-21T06:28:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2da0004800 0x7f2dbc156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 0x7f2dbc0d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dbc0d33c0 0x7f2dbc0730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f7c46000640 1 -- 172.21.15.37:0/3357044947 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34009420 con 0x7f7c2c004800 2024-09-21T06:28:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0cf6a0 con 0x7f2da0004800 2024-09-21T06:28:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0cf520 con 0x7f2dbc0d33c0 2024-09-21T06:28:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f2dc1600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0cf820 con 0x7f2dbc1fd130 2024-09-21T06:28:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f9211600640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f91f4078be0 0x7f91f407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f7c4d000640 1 -- 172.21.15.37:0/3357044947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c2c004800 msgr2=0x7f7c2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f7c4d000640 1 --2- 172.21.15.37:0/3357044947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c2c004800 0x7f7c2c004c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009800 tx=0x7f7c2c005830 comp rx=0 tx=0).stop 2024-09-21T06:28:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f7c47e00640 1 -- 172.21.15.37:0/3357044947 reap_dead start 2024-09-21T06:28:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 0x7f2dbc0d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.425+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c240077d0 con 0x7f0c240e2f40 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.426+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f92000bb5e0 con 0x7f921406edf0 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f9218e00640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f91f4078be0 0x7f91f407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 0x7f2dbc0d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44242/0 (socket says 172.21.15.37:44242) 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f2dbb400640 1 -- 172.21.15.37:0/2181766274 learned_addr learned my addr 172.21.15.37:0/2181766274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f2dbbe00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2da0004800 0x7f2dbc156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f7c4d000640 1 -- 172.21.15.37:0/3357044947 shutdown_connections 2024-09-21T06:28:23.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f7c4d000640 1 -- 172.21.15.37:0/3357044947 >> 172.21.15.37:0/3357044947 conn(0x7f7c480d05c0 msgr2=0x7f7c4806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7ff220800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21806e330 0x7ff218149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a2c01d460 con 0x7f0a34152270 2024-09-21T06:28:23.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f2dc0c00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dbc0d33c0 0x7f2dbc0730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f7c4d000640 1 -- 172.21.15.37:0/3357044947 shutdown_connections 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f9211600640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f91f4082320 0x7f91f40847b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f920002eec0 con 0x7f91f4082320 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.427+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92000c2050 con 0x7f921406edf0 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dbb400640 1 -- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dbc0d33c0 msgr2=0x7f2dbc0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f7c4d000640 1 -- 172.21.15.37:0/3357044947 wait complete. 2024-09-21T06:28:23.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff21de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21806e330 0x7ff218149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f921a200640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f91f4082320 0x7f91f40847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f9218e00640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f91f4078be0 0x7f91f407b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f921415bc20 tx=0x7f92040079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f921a200640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f91f4082320 0x7f91f40847b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff220800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 0x7ff2181f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff220800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21815fc50 0x7ff218147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff220800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff218070d20 con 0x7ff2180cf7d0 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff220800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff218070ba0 con 0x7ff21806e330 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff220800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180734a0 con 0x7ff21815fc50 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7ff21d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 0x7ff2181f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7ff21d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 0x7ff2181f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42370/0 (socket says 172.21.15.37:42370) 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7ff21d400640 1 -- 172.21.15.37:0/1932528850 learned_addr learned my addr 172.21.15.37:0/1932528850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7ff21de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21806e330 0x7ff218149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:55146/0 (socket says 172.21.15.37:55146) 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7ff21d400640 1 -- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21815fc50 msgr2=0x7ff218147650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7ff21d400640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21815fc50 0x7ff218147650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7ff21d400640 1 -- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21806e330 msgr2=0x7ff218149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7ff21d400640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21806e330 0x7ff218149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7ff21d400640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21815c2f0 con 0x7ff2180cf7d0 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.431+0000 7ff21d400640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 0x7ff2181f2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff214000f80 tx=0x7ff214003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dbb400640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dbc0d33c0 0x7f2dbc0730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dbb400640 1 -- 172.21.15.37:0/2181766274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2da0004800 msgr2=0x7f2dbc156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dbb400640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2da0004800 0x7f2dbc156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dbb400640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc070b90 con 0x7f2dbc1fd130 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f2dc0c00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dbc0d33c0 0x7f2dbc0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f2dbb400640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 0x7f2dbc0d2e80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c630 tx=0x7f2dac0060d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac003810 con 0x7f2dbc1fd130 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dac003aa0 con 0x7f2dbc1fd130 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03e440 con 0x7f2dbc1fd130 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f2dbbe00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2da0004800 0x7f2dbc156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.431+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc0d37d0 con 0x7f2dbc1fd130 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.431+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc158820 con 0x7f2dbc1fd130 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f0a2aa00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a10078bc0 0x7f0a1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.429+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0a2c09c540 con 0x7f0a34152270 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f0a2aa00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f0a10082300 0x7f0a10084790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a2c023e50 con 0x7f0a10082300 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c09c8f0 con 0x7f0a34152270 2024-09-21T06:28:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f0a32a00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a10078bc0 0x7f0a1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.430+0000 7f0a32000640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f0a10082300 0x7f0a10084790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.431+0000 7f0a32000640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f0a10082300 0x7f0a10084790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.428+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c1801c460 con 0x7f0c240e2f40 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.431+0000 7f0c22000640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c0c078be0 0x7f0c0c07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0c1809ae60 con 0x7f0c240e2f40 2024-09-21T06:28:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== osd.2 v2:172.21.15.37:6800/2037276396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f920002eec0 con 0x7f91f4082320 2024-09-21T06:28:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0c22000640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7f0c0c0823a0 0x7f0c0c084830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c0c084f00 con 0x7f0c0c0823a0 2024-09-21T06:28:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c1809b210 con 0x7f0c240e2f40 2024-09-21T06:28:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f7c4d000640 1 Processor -- start 2024-09-21T06:28:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214046070 con 0x7ff2180cf7d0 2024-09-21T06:28:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2140046c0 con 0x7ff2180cf7d0 2024-09-21T06:28:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214033070 con 0x7ff2180cf7d0 2024-09-21T06:28:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2dbc158360 con 0x7f2dbc1fd130 2024-09-21T06:28:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.433+0000 7f7c4d000640 1 -- start start 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a2c023e50 con 0x7f0a10082300 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.433+0000 7f0a32a00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a10078bc0 0x7f0a1007b0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c035000 tx=0x7f0a1c0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.432+0000 7f0c29e00640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c0c078be0 0x7f0c0c07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.433+0000 7f0c29400640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7f0c0c0823a0 0x7f0c0c084830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.433+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21815c5d0 con 0x7ff2180cf7d0 2024-09-21T06:28:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.433+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21815cb90 con 0x7ff2180cf7d0 2024-09-21T06:28:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.434+0000 7f0c29400640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7f0c0c0823a0 0x7f0c0c084830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.435+0000 7f0c29e00640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c0c078be0 0x7f0c0c07b0f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06c0e0 tx=0x7f0c2c0b0040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.434+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff21815cf70 con 0x7ff2180cf7d0 2024-09-21T06:28:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.435+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c0c084f00 con 0x7f0c0c0823a0 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.436+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2dac03e5e0 con 0x7f2dbc1fd130 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.436+0000 7f7c4d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c48144450 0x7f7c48150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.437+0000 7f7c4d000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 0x7f7c48146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.437+0000 7f7c4d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c481465d0 0x7f7c481470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.437+0000 7f7c4d000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c480b4420 con 0x7f7c481465d0 2024-09-21T06:28:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.437+0000 7f7c4d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7c480b42a0 con 0x7f7c48144450 2024-09-21T06:28:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.437+0000 7f2db3e00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d9c078b90 0x7f2d9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f7c4d000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7c480b45a0 con 0x7f7c48145c80 2024-09-21T06:28:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2dac0bb3e0 con 0x7f2dbc1fd130 2024-09-21T06:28:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f2db3e00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2d9c0822e0 0x7f2d9c084770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f7c46a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 0x7f7c48146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d9c084e40 con 0x7f2d9c0822e0 2024-09-21T06:28:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.438+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2dac0bb870 con 0x7f2dbc1fd130 2024-09-21T06:28:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f2dc0c00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2d9c0822e0 0x7f2d9c084770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f2dbbe00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d9c078b90 0x7f2d9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 0x7f7c48146090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:44258/0 (socket says 172.21.15.37:44258) 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 -- 172.21.15.37:0/1575993967 learned_addr learned my addr 172.21.15.37:0/1575993967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c47400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c48144450 0x7f7c48150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c481465d0 0x7f7c481470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 -- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c48144450 msgr2=0x7f7c48150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c48144450 0x7f7c48150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 -- 172.21.15.37:0/1575993967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c481465d0 msgr2=0x7f7c481470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f7c46a00640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c481465d0 0x7f7c481470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.439+0000 7f2dc0c00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2d9c0822e0 0x7f2d9c084770 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff214036070 con 0x7ff2180cf7d0 2024-09-21T06:28:23.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7f2dbbe00640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d9c078b90 0x7f2d9c07b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0521d0 tx=0x7f2da8005e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7f7c46a00640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48146c20 con 0x7f7c48145c80 2024-09-21T06:28:23.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7f7c47400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c48144450 0x7f7c48150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:23.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d9c084e40 con 0x7f2d9c0822e0 2024-09-21T06:28:23.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.440+0000 7f7c46a00640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 0x7f7c48146090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009800 tx=0x7f7c34032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34037040 con 0x7f7c48145c80 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c3403d430 con 0x7f7c48145c80 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34005ad0 con 0x7f7c48145c80 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7ff212000640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1fc078b90 0x7ff1fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481537d0 con 0x7f7c48145c80 2024-09-21T06:28:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c48153d90 con 0x7f7c48145c80 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff2140bc090 con 0x7ff2180cf7d0 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7ff212000640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7ff1fc082350 0x7ff1fc0847e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff214030950 con 0x7ff1fc082350 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.441+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2140bc480 con 0x7ff2180cf7d0 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.442+0000 7ff21e800640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7ff1fc082350 0x7ff1fc0847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.443+0000 7ff21de00640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1fc078b90 0x7ff1fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.444+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c48154170 con 0x7f7c48145c80 2024-09-21T06:28:23.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.444+0000 7ff21e800640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7ff1fc082350 0x7ff1fc0847e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.444+0000 7ff21de00640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1fc078b90 0x7ff1fc07b0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff2080058c0 tx=0x7ff208005d80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.445+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== osd.4 v2:172.21.15.150:6818/2746357825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff214030950 con 0x7ff1fc082350 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.446+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c3403d8c0 con 0x7f7c48145c80 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.447+0000 7f7c3f400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7c28078b90 0x7f7c2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.447+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7c340bb410 con 0x7f7c48145c80 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.448+0000 7f7c3f400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7f7c28082350 0x7f7c280847e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.448+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c34004510 con 0x7f7c28082350 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.448+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c340c2050 con 0x7f7c48145c80 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.449+0000 7f7c47400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7c28078b90 0x7f7c2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.449+0000 7f7c47e00640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7f7c28082350 0x7f7c280847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.450+0000 7f7c47400640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7c28078b90 0x7f7c2807b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c40004fd0 tx=0x7f7c40003970 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.450+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82b820e5e0 con 0x7f82980822d0 2024-09-21T06:28:23.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.451+0000 7f7c47e00640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7f7c28082350 0x7f7c280847e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.451+0000 7f82b5600640 1 -- 172.21.15.37:0/1818760827 <== osd.0 v2:172.21.15.37:6808/787824175 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82b820e5e0 con 0x7f82980822d0 2024-09-21T06:28:23.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.452+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f82980822d0 msgr2=0x7f8298084760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.452+0000 7f82c0600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f82980822d0 0x7f8298084760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.452+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== osd.7 v2:172.21.15.150:6826/1641077969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c34004510 con 0x7f7c28082350 2024-09-21T06:28:23.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8298078b90 msgr2=0x7f829807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8298078b90 0x7f829807b0a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82b0035000 tx=0x7f82b0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 msgr2=0x7f82b81f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 --2- 172.21.15.37:0/1818760827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82b80739f0 0x7f82b81f83c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c3a0 tx=0x7f82a800c860 comp rx=0 tx=0).stop 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82be600640 1 -- 172.21.15.37:0/1818760827 reap_dead start 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 shutdown_connections 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 >> 172.21.15.37:0/1818760827 conn(0x7f82b80d0350 msgr2=0x7f82b80ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.453+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 shutdown_connections 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.454+0000 7f82c0600640 1 -- 172.21.15.37:0/1818760827 wait complete. 2024-09-21T06:28:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.454+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3304002dd0 con 0x7f3310083150 2024-09-21T06:28:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.455+0000 7f3327e00640 1 -- 172.21.15.37:0/2858643975 <== osd.1 v2:172.21.15.37:6824/1275190192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3304002dd0 con 0x7f3310083150 2024-09-21T06:28:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.456+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f3310083150 msgr2=0x7f3310085640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.456+0000 7f3325600640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f3310083150 0x7f3310085640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3310078dc0 msgr2=0x7f331007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3325600640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3310078dc0 0x7f331007b2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3320034000 tx=0x7f33200046c0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 msgr2=0x7f33300c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3325600640 1 --2- 172.21.15.37:0/2858643975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3330153e80 0x7f33300c7ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f332800c730 tx=0x7f332800cbf0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3334c00640 1 -- 172.21.15.37:0/2858643975 reap_dead start 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.457+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 shutdown_connections 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.458+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 >> 172.21.15.37:0/2858643975 conn(0x7f33300d1960 msgr2=0x7f333016d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.458+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 shutdown_connections 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.458+0000 7f3325600640 1 -- 172.21.15.37:0/2858643975 wait complete. 2024-09-21T06:28:23.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.463+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c24052c40 con 0x7f0c0c0823a0 2024-09-21T06:28:23.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.464+0000 7f0c22000640 1 -- 172.21.15.37:0/4171988791 <== osd.3 v2:172.21.15.37:6815/1242254511 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c24052c40 con 0x7f0c0c0823a0 2024-09-21T06:28:23.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.465+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7f0c0c0823a0 msgr2=0x7f0c0c084830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.465+0000 7f0c2b200640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7f0c0c0823a0 0x7f0c0c084830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c0c078be0 msgr2=0x7f0c0c07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c0c078be0 0x7f0c0c07b0f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06c0e0 tx=0x7f0c2c0b0040 comp rx=0 tx=0).stop 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c240e2f40 msgr2=0x7f0c2402f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 --2- 172.21.15.37:0/4171988791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c240e2f40 0x7f0c2402f970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c240e3520 tx=0x7f0c1800c820 comp rx=0 tx=0).stop 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2a800640 1 -- 172.21.15.37:0/4171988791 reap_dead start 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 shutdown_connections 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.466+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 >> 172.21.15.37:0/4171988791 conn(0x7f0c2403a5f0 msgr2=0x7f0c2403d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.467+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 shutdown_connections 2024-09-21T06:28:23.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.467+0000 7f0c2b200640 1 -- 172.21.15.37:0/4171988791 wait complete. 2024-09-21T06:28:23.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.467+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f921406f200 con 0x7f91f4082320 2024-09-21T06:28:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.468+0000 7f9211600640 1 -- 172.21.15.37:0/3674294803 <== osd.2 v2:172.21.15.37:6800/2037276396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f921406f200 con 0x7f91f4082320 2024-09-21T06:28:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f91f4082320 msgr2=0x7f91f40847b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f91f4082320 0x7f91f40847b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f91f4078be0 msgr2=0x7f91f407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a34070490 con 0x7f0a10082300 2024-09-21T06:28:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f91f4078be0 0x7f91f407b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f921415bc20 tx=0x7f92040079d0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 msgr2=0x7f92141f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921ac00640 1 --2- 172.21.15.37:0/3674294803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f921406edf0 0x7f92141f5440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f9200003820 comp rx=0 tx=0).stop 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.469+0000 7f921a200640 1 -- 172.21.15.37:0/3674294803 reap_dead start 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.470+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 shutdown_connections 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.470+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 >> 172.21.15.37:0/3674294803 conn(0x7f92140d05c0 msgr2=0x7f9214151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.470+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 shutdown_connections 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.470+0000 7f921ac00640 1 -- 172.21.15.37:0/3674294803 wait complete. 2024-09-21T06:28:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.470+0000 7f0a2aa00640 1 -- 172.21.15.37:0/2727183136 <== osd.6 v2:172.21.15.150:6810/3705202380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a34070490 con 0x7f0a10082300 2024-09-21T06:28:23.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.472+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f0a10082300 msgr2=0x7f0a10084790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.472+0000 7f0a33e00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f0a10082300 0x7f0a10084790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.473+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a10078bc0 msgr2=0x7f0a1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.473+0000 7f0a33e00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a10078bc0 0x7f0a1007b0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c035000 tx=0x7f0a1c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.473+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 msgr2=0x7f0a341fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.473+0000 7f0a33e00640 1 --2- 172.21.15.37:0/2727183136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a34152270 0x7f0a341fcba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a34162180 tx=0x7f0a2c001c50 comp rx=0 tx=0).stop 2024-09-21T06:28:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.473+0000 7f0a33400640 1 -- 172.21.15.37:0/2727183136 reap_dead start 2024-09-21T06:28:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.477+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2dbc1585d0 con 0x7f2d9c0822e0 2024-09-21T06:28:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.477+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 shutdown_connections 2024-09-21T06:28:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.477+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 >> 172.21.15.37:0/2727183136 conn(0x7f0a340d41e0 msgr2=0x7f0a340b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.477+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 shutdown_connections 2024-09-21T06:28:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.478+0000 7f2db3e00640 1 -- 172.21.15.37:0/2181766274 <== osd.5 v2:172.21.15.150:6802/2870078774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2dbc1585d0 con 0x7f2d9c0822e0 2024-09-21T06:28:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.478+0000 7f0a33e00640 1 -- 172.21.15.37:0/2727183136 wait complete. 2024-09-21T06:28:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2d9c0822e0 msgr2=0x7f2d9c084770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2d9c0822e0 0x7f2d9c084770 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d9c078b90 msgr2=0x7f2d9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d9c078b90 0x7f2d9c07b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0521d0 tx=0x7f2da8005e30 comp rx=0 tx=0).stop 2024-09-21T06:28:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 msgr2=0x7f2dbc0d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 --2- 172.21.15.37:0/2181766274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dbc1fd130 0x7f2dbc0d2e80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c630 tx=0x7f2dac0060d0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc0c00640 1 -- 172.21.15.37:0/2181766274 reap_dead start 2024-09-21T06:28:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 shutdown_connections 2024-09-21T06:28:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.479+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 >> 172.21.15.37:0/2181766274 conn(0x7f2dbc0d41e0 msgr2=0x7f2dbc0b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.480+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 shutdown_connections 2024-09-21T06:28:23.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.480+0000 7f2dc1600640 1 -- 172.21.15.37:0/2181766274 wait complete. 2024-09-21T06:28:23.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.481+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 --> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff218070240 con 0x7ff1fc082350 2024-09-21T06:28:23.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.485+0000 7ff212000640 1 -- 172.21.15.37:0/1932528850 <== osd.4 v2:172.21.15.150:6818/2746357825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff218070240 con 0x7ff1fc082350 2024-09-21T06:28:23.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.485+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c481512d0 con 0x7f7c28082350 2024-09-21T06:28:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.487+0000 7f7c3f400640 1 -- 172.21.15.37:0/1575993967 <== osd.7 v2:172.21.15.150:6826/1641077969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7c481512d0 con 0x7f7c28082350 2024-09-21T06:28:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7f7c28082350 msgr2=0x7f7c280847e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7f7c28082350 0x7f7c280847e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.489 INFO:teuthology.orchestra.run.smithi037.stdout:64424509452 2024-09-21T06:28:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7c28078b90 msgr2=0x7f7c2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7c28078b90 0x7f7c2807b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c40004fd0 tx=0x7f7c40003970 comp rx=0 tx=0).stop 2024-09-21T06:28:23.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T06:28:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7ff1fc082350 msgr2=0x7ff1fc0847e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7ff1fc082350 0x7ff1fc0847e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 msgr2=0x7f7c48146090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 --2- 172.21.15.37:0/1575993967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c48145c80 0x7f7c48146090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009800 tx=0x7f7c34032c20 comp rx=0 tx=0).stop 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c47e00640 1 -- 172.21.15.37:0/1575993967 reap_dead start 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 shutdown_connections 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.488+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 >> 172.21.15.37:0/1575993967 conn(0x7f7c480d05c0 msgr2=0x7f7c48144b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 shutdown_connections 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1fc078b90 msgr2=0x7ff1fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7f7c4d000640 1 -- 172.21.15.37:0/1575993967 wait complete. 2024-09-21T06:28:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1fc078b90 0x7ff1fc07b0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff2080058c0 tx=0x7ff208005d80 comp rx=0 tx=0).stop 2024-09-21T06:28:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 msgr2=0x7ff2181f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 --2- 172.21.15.37:0/1932528850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2180cf7d0 0x7ff2181f2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff214000f80 tx=0x7ff214003f80 comp rx=0 tx=0).stop 2024-09-21T06:28:23.493 INFO:teuthology.orchestra.run.smithi037.stdout:64424509451 2024-09-21T06:28:23.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff21e800640 1 -- 172.21.15.37:0/1932528850 reap_dead start 2024-09-21T06:28:23.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T06:28:23.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 shutdown_connections 2024-09-21T06:28:23.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.489+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 >> 172.21.15.37:0/1932528850 conn(0x7ff2180b1b90 msgr2=0x7ff2180afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.490+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 shutdown_connections 2024-09-21T06:28:23.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.490+0000 7ff220800640 1 -- 172.21.15.37:0/1932528850 wait complete. 2024-09-21T06:28:23.501 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:23.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T06:28:23.505 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:23.505 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T06:28:23.516 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-09-21T06:28:23.516 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:23.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T06:28:23.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T06:28:23.521 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:23.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T06:28:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:23.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 -- 172.21.15.37:0/3914488606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b815e520 msgr2=0x7ff8b80d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 --2- 172.21.15.37:0/3914488606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b815e520 0x7ff8b80d2170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009830 tx=0x7ff8a802a410 comp rx=0 tx=0).stop 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 -- 172.21.15.37:0/3914488606 shutdown_connections 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 --2- 172.21.15.37:0/3914488606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b815e520 0x7ff8b80d2170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 --2- 172.21.15.37:0/3914488606 >> v2:172.21.15.150:3300/0 conn(0x7ff8b815f6d0 0x7ff8b80cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 -- 172.21.15.37:0/3914488606 >> 172.21.15.37:0/3914488606 conn(0x7ff8b80d41e0 msgr2=0x7ff8b80d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 -- 172.21.15.37:0/3914488606 shutdown_connections 2024-09-21T06:28:23.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.901+0000 7ff8bce00640 1 -- 172.21.15.37:0/3914488606 wait complete. 2024-09-21T06:28:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.905+0000 7ff8bce00640 1 Processor -- start 2024-09-21T06:28:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.905+0000 7ff8bce00640 1 -- start start 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.907+0000 7ff8bce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.907+0000 7ff8bce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b815e520 0x7ff8b80cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.907+0000 7ff8bce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b815f6d0 0x7ff8b80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8bce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b80d26b0 con 0x7ff8b815e520 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8bce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b815eb40 con 0x7ff8b815f6d0 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8bce00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b80d2830 con 0x7ff8b80cf620 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8b7400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8b6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b815e520 0x7ff8b80cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8b7400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58030/0 (socket says 172.21.15.37:58030) 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.908+0000 7ff8b7400640 1 -- 172.21.15.37:0/229408148 learned_addr learned my addr 172.21.15.37:0/229408148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7e00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b815f6d0 0x7ff8b80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7400640 1 -- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b815f6d0 msgr2=0x7ff8b80b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7400640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b815f6d0 0x7ff8b80b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7400640 1 -- 172.21.15.37:0/229408148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b815e520 msgr2=0x7ff8b80cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7400640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b815e520 0x7ff8b80cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.909+0000 7ff8b7400640 1 -- 172.21.15.37:0/229408148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b8151870 con 0x7ff8b80cf620 2024-09-21T06:28:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.910+0000 7ff8b7400640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000eb30 tx=0x7ff8b000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.911+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0010430 con 0x7ff8b80cf620 2024-09-21T06:28:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.911+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b00105d0 con 0x7ff8b80cf620 2024-09-21T06:28:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.911+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0010a20 con 0x7ff8b80cf620 2024-09-21T06:28:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.911+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b8151b50 con 0x7ff8b80cf620 2024-09-21T06:28:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.911+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b80b5180 con 0x7ff8b80cf620 2024-09-21T06:28:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.913+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b80d2170 con 0x7ff8b80cf620 2024-09-21T06:28:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.919+0000 7f33cea00640 1 -- 172.21.15.37:0/1522379226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 msgr2=0x7f33c80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.919+0000 7f33cea00640 1 --2- 172.21.15.37:0/1522379226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c80ceb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f33c81623b0 tx=0x7f33c002a9a0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 -- 172.21.15.37:0/1522379226 shutdown_connections 2024-09-21T06:28:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 --2- 172.21.15.37:0/1522379226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c80ceb90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7fc702800640 1 -- 172.21.15.37:0/2716944040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f40e2f40 msgr2=0x7fc6f403c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7fc702800640 1 --2- 172.21.15.37:0/2716944040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f40e2f40 0x7fc6f403c1f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6f403b9e0 tx=0x7fc6e802a400 comp rx=0 tx=0).stop 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 --2- 172.21.15.37:0/1522379226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c80b9ad0 0x7f33c80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 --2- 172.21.15.37:0/1522379226 >> v2:172.21.15.37:3300/0 conn(0x7f33c815ee80 0x7f33c80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 -- 172.21.15.37:0/1522379226 >> 172.21.15.37:0/1522379226 conn(0x7f33c809d830 msgr2=0x7f33c80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 -- 172.21.15.37:0/1522379226 shutdown_connections 2024-09-21T06:28:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.920+0000 7f33cea00640 1 -- 172.21.15.37:0/1522379226 wait complete. 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 -- 172.21.15.37:0/2716944040 shutdown_connections 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 --2- 172.21.15.37:0/2716944040 >> v2:172.21.15.150:3300/0 conn(0x7fc6f4053c50 0x7fc6f4054060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 --2- 172.21.15.37:0/2716944040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f40e2f40 0x7fc6f403c1f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 --2- 172.21.15.37:0/2716944040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4056da0 0x7fc6f4057c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 -- 172.21.15.37:0/2716944040 >> 172.21.15.37:0/2716944040 conn(0x7fc6f403a5f0 msgr2=0x7fc6f403aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 -- 172.21.15.37:0/2716944040 shutdown_connections 2024-09-21T06:28:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.919+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8b0010bc0 con 0x7ff8b80cf620 2024-09-21T06:28:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7ff8af400640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 0x7ff89807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff8b009c680 con 0x7ff8b80cf620 2024-09-21T06:28:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7fc702800640 1 -- 172.21.15.37:0/2716944040 wait complete. 2024-09-21T06:28:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.921+0000 7ff8b6a00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 0x7ff89807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.922+0000 7ff8b6a00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 0x7ff89807b1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80b5f00 tx=0x7ff8a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.923+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff8b0064110 con 0x7ff8b80cf620 2024-09-21T06:28:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 -- 172.21.15.37:0/1514619951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc009b90 msgr2=0x7f31dc0df170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 --2- 172.21.15.37:0/1514619951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc009b90 0x7f31dc0df170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31d8007310 tx=0x7f31d802b230 comp rx=0 tx=0).stop 2024-09-21T06:28:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 -- 172.21.15.37:0/3260766979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 msgr2=0x7fa624070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 --2- 172.21.15.37:0/3260766979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624070070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa614009830 tx=0x7fa61402a510 comp rx=0 tx=0).stop 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 -- 172.21.15.37:0/1514619951 shutdown_connections 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 --2- 172.21.15.37:0/1514619951 >> v2:172.21.15.150:3300/0 conn(0x7f31dc005c50 0x7f31dc036420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 --2- 172.21.15.37:0/1514619951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc009b90 0x7f31dc0df170 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7f31eb600640 1 -- 172.21.15.37:0/1514619951 >> 172.21.15.37:0/1514619951 conn(0x7f31dc051410 msgr2=0x7f31dc051850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 -- 172.21.15.37:0/3260766979 shutdown_connections 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 --2- 172.21.15.37:0/3260766979 >> v2:172.21.15.37:3300/0 conn(0x7fa62406ddf0 0x7fa62406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 --2- 172.21.15.37:0/3260766979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624070070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fa62b000640 1 -- 172.21.15.37:0/3260766979 >> 172.21.15.37:0/3260766979 conn(0x7fa6240b1b90 msgr2=0x7fa6240b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.924+0000 7fc702800640 1 Processor -- start 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7fa62b000640 1 -- 172.21.15.37:0/3260766979 shutdown_connections 2024-09-21T06:28:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7f31eb600640 1 -- 172.21.15.37:0/1514619951 shutdown_connections 2024-09-21T06:28:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7fa62b000640 1 -- 172.21.15.37:0/3260766979 wait complete. 2024-09-21T06:28:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7f31eb600640 1 -- 172.21.15.37:0/1514619951 wait complete. 2024-09-21T06:28:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7f33cea00640 1 Processor -- start 2024-09-21T06:28:23.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.925+0000 7fc702800640 1 -- start start 2024-09-21T06:28:23.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.926+0000 7f33cea00640 1 -- start start 2024-09-21T06:28:23.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.928+0000 7fc702800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7fc702800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 0x7fc6f417f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7fc702800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 0x7fc6f40deb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c80b9ad0 0x7f33c81f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7fc701400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7fc701400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58092/0 (socket says 172.21.15.37:58092) 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f31eb600640 1 Processor -- start 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c806f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33c80cf510 con 0x7f33c80b9ad0 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33c80cf390 con 0x7f33c80d1120 2024-09-21T06:28:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f33c80b6580 con 0x7f33c815ee80 2024-09-21T06:28:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7f31eb600640 1 -- start start 2024-09-21T06:28:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7fa62b000640 1 Processor -- start 2024-09-21T06:28:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7fc701400640 1 -- 172.21.15.37:0/3667158951 learned_addr learned my addr 172.21.15.37:0/3667158951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4006f20 con 0x7fc6f4056da0 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4006da0 con 0x7fc6f40e2f40 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f40070a0 con 0x7fc6f4053c50 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cca00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cca00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58100/0 (socket says 172.21.15.37:58100) 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.929+0000 7f33cca00640 1 -- 172.21.15.37:0/1593188311 learned_addr learned my addr 172.21.15.37:0/1593188311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7f33c7e00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c80b9ad0 0x7f33c81f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7fa62b000640 1 -- start start 2024-09-21T06:28:23.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.930+0000 7f33c7400640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701e00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 0x7fc6f40deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7f33cca00640 1 -- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 msgr2=0x7f33c806f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7f33cca00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c806f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7f33cca00640 1 -- 172.21.15.37:0/1593188311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c80b9ad0 msgr2=0x7f33c81f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7f33cca00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c80b9ad0 0x7f33c81f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7f33cca00640 1 -- 172.21.15.37:0/1593188311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c81f6700 con 0x7f33c815ee80 2024-09-21T06:28:23.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701400640 1 -- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 msgr2=0x7fc6f40deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc700a00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 0x7fc6f417f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701400640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 0x7fc6f40deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701400640 1 -- 172.21.15.37:0/3667158951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 msgr2=0x7fc6f417f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701400640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 0x7fc6f417f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.931+0000 7fc701400640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f4182ab0 con 0x7fc6f4053c50 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.932+0000 7fc701e00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 0x7fc6f40deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.932+0000 7fc700a00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 0x7fc6f417f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.932+0000 7f33cca00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33c002af90 tx=0x7f33c0003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0033050 con 0x7f33c815ee80 2024-09-21T06:28:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c806f780 con 0x7f33c815ee80 2024-09-21T06:28:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7fc701400640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e790 tx=0x7fc6f000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7fa62b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33c00043f0 con 0x7f33c815ee80 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0004820 con 0x7f33c815ee80 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c806e480 con 0x7f33c815ee80 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.933+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000eea0 con 0x7fc6f4053c50 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6f000cc60 con 0x7fc6f4053c50 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0004020 con 0x7fc6f4053c50 2024-09-21T06:28:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f4182d90 con 0x7fc6f4053c50 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa62b000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa62406ec50 0x7fa624152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa62b000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa62b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa624073f40 con 0x7fa62406ddf0 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa62b000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa624070e80 con 0x7fa62406ec50 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa62b000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa624071000 con 0x7fa62415ac20 2024-09-21T06:28:23.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f40dcec0 con 0x7fc6f4053c50 2024-09-21T06:28:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa623e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa623e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40264/0 (socket says 172.21.15.37:40264) 2024-09-21T06:28:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.934+0000 7fa623e00640 1 -- 172.21.15.37:0/3608921413 learned_addr learned my addr 172.21.15.37:0/3608921413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7f433ca00640 1 -- 172.21.15.37:0/2970819876 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338031b80 con 0x7f43400ceb90 2024-09-21T06:28:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623400640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa62406ec50 0x7fa624152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa629000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623e00640 1 -- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 msgr2=0x7fa624153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623e00640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623e00640 1 -- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa62406ec50 msgr2=0x7fa624152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623e00640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa62406ec50 0x7fa624152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.935+0000 7fa623e00640 1 -- 172.21.15.37:0/3608921413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240c9680 con 0x7fa62406ddf0 2024-09-21T06:28:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.936+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c806dec0 con 0x7f33c815ee80 2024-09-21T06:28:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.936+0000 7f4344c00640 1 -- 172.21.15.37:0/2970819876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 msgr2=0x7f43400cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.936+0000 7f4344c00640 1 --2- 172.21.15.37:0/2970819876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f43400cefa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4338005fd0 tx=0x7f433802a760 comp rx=0 tx=0).stop 2024-09-21T06:28:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.936+0000 7fa623e00640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa614009950 tx=0x7fa614033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc005c50 0x7f31dc0e2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 0x7f31dc01f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00bda0 con 0x7f31dc036960 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00bc20 con 0x7f31dc009b90 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31eb600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc01ccb0 con 0x7f31dc005c50 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 -- 172.21.15.37:0/2970819876 shutdown_connections 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 --2- 172.21.15.37:0/2970819876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f43400cefa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 --2- 172.21.15.37:0/2970819876 >> v2:172.21.15.37:3300/0 conn(0x7f434015ee80 0x7f43400d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 -- 172.21.15.37:0/2970819876 >> 172.21.15.37:0/2970819876 conn(0x7f434009d830 msgr2=0x7f43400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 -- 172.21.15.37:0/2970819876 shutdown_connections 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d8005290 con 0x7fc6f4053c50 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:41058/0 (socket says 172.21.15.37:41058) 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31e3e00640 1 -- 172.21.15.37:0/3745857035 learned_addr learned my addr 172.21.15.37:0/3745857035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f31e9600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 0x7f31dc01f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7fcaf3a00640 1 -- 172.21.15.37:0/1682922173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad0004800 msgr2=0x7fcad0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/1682922173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad0004800 0x7fcad0004c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaec08d980 tx=0x7fcad0005830 comp rx=0 tx=0).stop 2024-09-21T06:28:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614005c90 con 0x7fa62406ddf0 2024-09-21T06:28:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7f31e8c00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc005c50 0x7f31dc0e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7fcaf1a00640 1 -- 172.21.15.37:0/1682922173 reap_dead start 2024-09-21T06:28:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33c0036070 con 0x7f33c815ee80 2024-09-21T06:28:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa614031e50 con 0x7fa62406ddf0 2024-09-21T06:28:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.938+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140332f0 con 0x7fa62406ddf0 2024-09-21T06:28:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.939+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6240c8080 con 0x7fa62406ddf0 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.939+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6240c8640 con 0x7fa62406ddf0 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.939+0000 7f31e3e00640 1 -- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc005c50 msgr2=0x7f31dc0e2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.939+0000 7f31e3e00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc005c50 0x7f31dc0e2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7f31e3e00640 1 -- 172.21.15.37:0/3745857035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 msgr2=0x7f31dc01f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.939+0000 7f33bfe00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 0x7f33a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f33c00bc290 con 0x7f33c815ee80 2024-09-21T06:28:23.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7fcaf3a00640 1 -- 172.21.15.37:0/1682922173 shutdown_connections 2024-09-21T06:28:23.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7fcaf3a00640 1 -- 172.21.15.37:0/1682922173 >> 172.21.15.37:0/1682922173 conn(0x7fcaec09d830 msgr2=0x7fcaec0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.937+0000 7f4344c00640 1 -- 172.21.15.37:0/2970819876 wait complete. 2024-09-21T06:28:23.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7f33c7e00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 0x7f33a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.940+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6f003bd10 con 0x7fc6f4053c50 2024-09-21T06:28:23.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f31e3e00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 0x7f31dc01f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f31e3e00640 1 -- 172.21.15.37:0/3745857035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc0e11d0 con 0x7f31dc009b90 2024-09-21T06:28:23.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7fcaf3a00640 1 -- 172.21.15.37:0/1682922173 shutdown_connections 2024-09-21T06:28:23.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62406f930 con 0x7fa62406ddf0 2024-09-21T06:28:23.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f4344c00640 1 Processor -- start 2024-09-21T06:28:23.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f31e9600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 0x7f31dc01f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:23.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f4344c00640 1 -- start start 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7f31e3e00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b440 tx=0x7f31d000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0015020 con 0x7f31dc009b90 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31d0011890 con 0x7f31dc009b90 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc0e14b0 con 0x7f31dc009b90 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc0e1a70 con 0x7f31dc009b90 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0011ce0 con 0x7f31dc009b90 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7fc6f9600640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 0x7fc6dc07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.942+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc6f009b7d0 con 0x7fc6f4053c50 2024-09-21T06:28:23.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.941+0000 7fcaf3a00640 1 -- 172.21.15.37:0/1682922173 wait complete. 2024-09-21T06:28:23.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.943+0000 7fc700a00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 0x7fc6dc07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400b9ad0 0x7f43400ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6f000f3c0 con 0x7fc6f4053c50 2024-09-21T06:28:23.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7fc700a00640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 0x7fc6dc07b1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8034000 tx=0x7fc6e8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43400b6700 con 0x7f43400b9ad0 2024-09-21T06:28:23.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc0e0f40 con 0x7f31dc009b90 2024-09-21T06:28:23.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43400b6580 con 0x7f434015ee80 2024-09-21T06:28:23.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.944+0000 7f4344c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43400b6880 con 0x7f43400ceb90 2024-09-21T06:28:23.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.945+0000 7f433e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:41062/0 (socket says 172.21.15.37:41062) 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433d400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433d400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58134/0 (socket says 172.21.15.37:58134) 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433d400640 1 -- 172.21.15.37:0/1167256325 learned_addr learned my addr 172.21.15.37:0/1167256325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433d400640 1 -- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 msgr2=0x7f434015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f433d400640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.945+0000 7fcaf3a00640 1 Processor -- start 2024-09-21T06:28:23.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f433d400640 1 -- 172.21.15.37:0/1167256325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400b9ad0 msgr2=0x7f43400ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f433d400640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400b9ad0 0x7f43400ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7fcaf3a00640 1 -- start start 2024-09-21T06:28:23.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f433d400640 1 -- 172.21.15.37:0/1167256325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4340206550 con 0x7f43400ceb90 2024-09-21T06:28:23.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f433e800640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.946+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31d0018020 con 0x7f31dc009b90 2024-09-21T06:28:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f31e0c00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 0x7f31c407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f31d009b0d0 con 0x7f31dc009b90 2024-09-21T06:28:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.947+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa614033490 con 0x7fa62406ddf0 2024-09-21T06:28:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.948+0000 7f31e8c00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 0x7f31c407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.948+0000 7f31e8c00640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 0x7f31c407b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31d8034000 tx=0x7f31d8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.949+0000 7fcaf3a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 0x7fcaec14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7fcaf3a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 0x7fcaec0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7f33c7e00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 0x7f33a807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33b800a770 tx=0x7f33b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33c81f6700 con 0x7f33c815ee80 2024-09-21T06:28:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7fa61be00640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 0x7fa60407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7fcaf1000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 0x7fcaec14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7fcaf1000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 0x7fcaec14c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58146/0 (socket says 172.21.15.37:58146) 2024-09-21T06:28:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.950+0000 7fcaf3a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 0x7fcaec1f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7f433d400640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c410 tx=0x7f432c00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7fa623400640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 0x7fa60407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7fcae3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 0x7fcaec0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa6140bc1b0 con 0x7fa62406ddf0 2024-09-21T06:28:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c007020 con 0x7f43400ceb90 2024-09-21T06:28:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7fcaf1000640 1 -- 172.21.15.37:0/3247877781 learned_addr learned my addr 172.21.15.37:0/3247877781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f432c0072b0 con 0x7f43400ceb90 2024-09-21T06:28:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.951+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0067d0 con 0x7f43400ceb90 2024-09-21T06:28:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.952+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4340202460 con 0x7f43400ceb90 2024-09-21T06:28:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.952+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0cf1f0 con 0x7fcaec0cc460 2024-09-21T06:28:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.952+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4340202a20 con 0x7f43400ceb90 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.952+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0cf070 con 0x7fcaec06cde0 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.952+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0cf370 con 0x7fcad0004800 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.953+0000 7fcaf1a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 0x7fcaec1f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcaf1a00640 1 -- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 msgr2=0x7fcaec14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcaf1a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 0x7fcaec14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcaf1a00640 1 -- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 msgr2=0x7fcaec0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcaf1a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 0x7fcaec0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43402062c0 con 0x7f43400ceb90 2024-09-21T06:28:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcaf1a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0cd8b0 con 0x7fcaec0cc460 2024-09-21T06:28:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.954+0000 7fcae3e00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 0x7fcaec0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.955+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31d001f020 con 0x7f31dc009b90 2024-09-21T06:28:23.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.955+0000 7fcaf1a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 0x7fcaec1f1440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ba5f0 tx=0x7fcae400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.955+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4003970 con 0x7fcaec0cc460 2024-09-21T06:28:23.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcae4003b30 con 0x7fcaec0cc460 2024-09-21T06:28:23.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fa623400640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 0x7fa60407b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b1950 tx=0x7fa610009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4004030 con 0x7fcaec0cc460 2024-09-21T06:28:23.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa61404a030 con 0x7fa62406ddf0 2024-09-21T06:28:23.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec0cdb90 con 0x7fcaec0cc460 2024-09-21T06:28:23.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.956+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec0cde70 con 0x7fcaec0cc460 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.958+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f432c019020 con 0x7f43400ceb90 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.960+0000 7f4336000640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 0x7f432007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.961+0000 7f433de00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 0x7f432007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.961+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcae40041d0 con 0x7fcaec0cc460 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.961+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac4005290 con 0x7fcaec0cc460 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.961+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f432c098b30 con 0x7f43400ceb90 2024-09-21T06:28:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.962+0000 7f433de00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 0x7f432007b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43300058c0 tx=0x7f4330005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.963+0000 7fcae9600640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 0x7fcac807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.963+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcae409be10 con 0x7fcaec0cc460 2024-09-21T06:28:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.963+0000 7fcaf1000640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 0x7fcac807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.965+0000 7fcaf1000640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 0x7fcac807b1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcad4000c00 tx=0x7fcad4006040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.968+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f432c062930 con 0x7f43400ceb90 2024-09-21T06:28:23.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.972+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcae4020060 con 0x7fcaec0cc460 2024-09-21T06:28:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.984+0000 7ff4d1000640 1 -- 172.21.15.37:0/1322465023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 msgr2=0x7ff4cc162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.984+0000 7ff4d1000640 1 --2- 172.21.15.37:0/1322465023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc162350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 -- 172.21.15.37:0/1322465023 shutdown_connections 2024-09-21T06:28:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 --2- 172.21.15.37:0/1322465023 >> v2:172.21.15.150:3300/0 conn(0x7ff4cc15a5b0 0x7ff4cc06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 --2- 172.21.15.37:0/1322465023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc162350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 -- 172.21.15.37:0/1322465023 >> 172.21.15.37:0/1322465023 conn(0x7ff4cc0d4290 msgr2=0x7ff4cc0d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:23.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 -- 172.21.15.37:0/1322465023 shutdown_connections 2024-09-21T06:28:23.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.985+0000 7ff4d1000640 1 -- 172.21.15.37:0/1322465023 wait complete. 2024-09-21T06:28:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.990+0000 7ff4d1000640 1 Processor -- start 2024-09-21T06:28:23.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.990+0000 7ff4d1000640 1 -- start start 2024-09-21T06:28:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.993+0000 7ff4d1000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.993+0000 7ff4d1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 0x7ff4cc1480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.994+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.994+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58172/0 (socket says 172.21.15.37:58172) 2024-09-21T06:28:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.994+0000 7ff4c9e00640 1 -- 172.21.15.37:0/85838593 learned_addr learned my addr 172.21.15.37:0/85838593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.994+0000 7ff4c9400640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 0x7ff4cc1480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.994+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 0x7ff4cc148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.995+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc15ab40 con 0x7ff4cc162890 2024-09-21T06:28:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.995+0000 7ff4ca800640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 0x7ff4cc148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.995+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc15a9c0 con 0x7ff4cc15a5b0 2024-09-21T06:28:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.995+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc15acc0 con 0x7ff4cc0ced80 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.997+0000 7ff4ca800640 1 -- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 msgr2=0x7ff4cc072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.997+0000 7ff4ca800640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.997+0000 7ff4ca800640 1 -- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 msgr2=0x7ff4cc1480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.997+0000 7ff4ca800640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 0x7ff4cc1480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.997+0000 7ff4ca800640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc159a30 con 0x7ff4cc162890 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.998+0000 7ff4ca800640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 0x7ff4cc148610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc08d640 tx=0x7ff4b800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.998+0000 7ff4c9400640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 0x7ff4cc1480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:23.998+0000 7ff4c9e00640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.002+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4cc162890 2024-09-21T06:28:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.002+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc072ab0 con 0x7ff4cc162890 2024-09-21T06:28:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.002+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc073b10 con 0x7ff4cc162890 2024-09-21T06:28:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.003+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b8004310 con 0x7ff4cc162890 2024-09-21T06:28:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.003+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4cc162890 2024-09-21T06:28:24.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.009+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a4005290 con 0x7ff4cc162890 2024-09-21T06:28:24.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.011+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4b8005020 con 0x7ff4cc162890 2024-09-21T06:28:24.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.013+0000 7ff4c2000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 0x7ff4ac07b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:24.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.013+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff4b809af10 con 0x7ff4cc162890 2024-09-21T06:28:24.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.018+0000 7ff4c9e00640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 0x7ff4ac07b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:24.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.019+0000 7ff4c9e00640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 0x7ff4ac07b2a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009420 tx=0x7ff4c4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:24.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.022+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4b8020020 con 0x7ff4cc162890 2024-09-21T06:28:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.388+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff8b8163110 con 0x7ff8b80cf620 2024-09-21T06:28:24.393 INFO:teuthology.orchestra.run.smithi037.stdout:64424509451 2024-09-21T06:28:24.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.389+0000 7ff8af400640 1 -- 172.21.15.37:0/229408148 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff8b0068b40 con 0x7ff8b80cf620 2024-09-21T06:28:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.393+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 msgr2=0x7ff89807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.393+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 0x7ff89807b1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80b5f00 tx=0x7ff8a4009040 comp rx=0 tx=0).stop 2024-09-21T06:28:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 msgr2=0x7ff8b80cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000eb30 tx=0x7ff8b000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 shutdown_connections 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff898078c90 0x7ff89807b1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b815e520 0x7ff8b80cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b815f6d0 0x7ff8b80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 --2- 172.21.15.37:0/229408148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8b80cf620 0x7ff8b80cab50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.394+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 >> 172.21.15.37:0/229408148 conn(0x7ff8b80d41e0 msgr2=0x7ff8b815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.395+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 shutdown_connections 2024-09-21T06:28:24.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.395+0000 7ff8bce00640 1 -- 172.21.15.37:0/229408148 wait complete. 2024-09-21T06:28:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.403+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc6d8005850 con 0x7fc6f4053c50 2024-09-21T06:28:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.403+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f33c80b1610 con 0x7f33c815ee80 2024-09-21T06:28:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.404+0000 7fc6f9600640 1 -- 172.21.15.37:0/3667158951 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6f00638b0 con 0x7fc6f4053c50 2024-09-21T06:28:24.405 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-09-21T06:28:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.404+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa624168b30 con 0x7fa62406ddf0 2024-09-21T06:28:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.404+0000 7f33bfe00640 1 -- 172.21.15.37:0/1593188311 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33c0084380 con 0x7f33c815ee80 2024-09-21T06:28:24.406 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-09-21T06:28:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.405+0000 7fa61be00640 1 -- 172.21.15.37:0/3608921413 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa61403f030 con 0x7fa62406ddf0 2024-09-21T06:28:24.410 INFO:teuthology.orchestra.run.smithi037.stdout:55834574857 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.408+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 msgr2=0x7fc6dc07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.408+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 0x7fc6dc07b1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8034000 tx=0x7fc6e8004730 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 msgr2=0x7fc6f4182160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e790 tx=0x7fc6f000ec50 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 shutdown_connections 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6dc078c90 0x7fc6dc07b1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6f4056da0 0x7fc6f417f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f40e2f40 0x7fc6f40deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 --2- 172.21.15.37:0/3667158951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6f4053c50 0x7fc6f4182160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 >> 172.21.15.37:0/3667158951 conn(0x7fc6f403a5f0 msgr2=0x7fc6f40e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 shutdown_connections 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7fc702800640 1 -- 172.21.15.37:0/3667158951 wait complete. 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 msgr2=0x7f33a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.409+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 0x7f33a807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33b800a770 tx=0x7f33b8008040 comp rx=0 tx=0).stop 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 msgr2=0x7f33c80a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33c002af90 tx=0x7f33c0003ef0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 shutdown_connections 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33a8078b90 0x7f33a807b0a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33c815ee80 0x7f33c80a17c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c80d1120 0x7f33c806f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 --2- 172.21.15.37:0/1593188311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c80b9ad0 0x7f33c81f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 >> 172.21.15.37:0/1593188311 conn(0x7f33c809d830 msgr2=0x7f33c80b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 msgr2=0x7fa60407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 0x7fa60407b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b1950 tx=0x7fa610009260 comp rx=0 tx=0).stop 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 msgr2=0x7fa6240c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa614009950 tx=0x7fa614033c20 comp rx=0 tx=0).stop 2024-09-21T06:28:24.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.410+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 shutdown_connections 2024-09-21T06:28:24.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7f33cea00640 1 -- 172.21.15.37:0/1593188311 wait complete. 2024-09-21T06:28:24.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 shutdown_connections 2024-09-21T06:28:24.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa604078b90 0x7fa60407b0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa62415ac20 0x7fa624153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa62406ec50 0x7fa624152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 --2- 172.21.15.37:0/3608921413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa62406ddf0 0x7fa6240c7490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 >> 172.21.15.37:0/3608921413 conn(0x7fa6240b1b90 msgr2=0x7fa6240b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 shutdown_connections 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.411+0000 7fa62b000640 1 -- 172.21.15.37:0/3608921413 wait complete. 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.413+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f31dc01d290 con 0x7f31dc009b90 2024-09-21T06:28:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.414+0000 7f31e0c00640 1 -- 172.21.15.37:0/3745857035 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31d000d040 con 0x7f31dc009b90 2024-09-21T06:28:24.418 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.419+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 msgr2=0x7f31c407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.419+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 0x7f31c407b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31d8034000 tx=0x7f31d8004730 comp rx=0 tx=0).stop 2024-09-21T06:28:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.419+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 msgr2=0x7f31dc01f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.419+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b440 tx=0x7f31d000b900 comp rx=0 tx=0).stop 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 shutdown_connections 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31c4078c60 0x7f31c407b170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31dc036960 0x7f31dc01f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc009b90 0x7f31dc01f020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 --2- 172.21.15.37:0/3745857035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc005c50 0x7f31dc0e2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 >> 172.21.15.37:0/3745857035 conn(0x7f31dc051410 msgr2=0x7f31dc034250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 shutdown_connections 2024-09-21T06:28:24.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.421+0000 7f31eb600640 1 -- 172.21.15.37:0/3745857035 wait complete. 2024-09-21T06:28:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.428+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f43400b6ca0 con 0x7f43400ceb90 2024-09-21T06:28:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.429+0000 7f4336000640 1 -- 172.21.15.37:0/1167256325 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f432c066d10 con 0x7f43400ceb90 2024-09-21T06:28:24.431 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:24.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.1 2024-09-21T06:28:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 msgr2=0x7f432007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 0x7f432007b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43300058c0 tx=0x7f4330005850 comp rx=0 tx=0).stop 2024-09-21T06:28:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcac4005580 con 0x7fcaec0cc460 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 msgr2=0x7f4340205820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c410 tx=0x7f432c00c8d0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 shutdown_connections 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4320078b90 0x7f432007b0a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f434015ee80 0x7f434015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43400ceb90 0x7f4340205820 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 --2- 172.21.15.37:0/1167256325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400b9ad0 0x7f43400ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.434+0000 7fcae9600640 1 -- 172.21.15.37:0/3247877781 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcae4063930 con 0x7fcaec0cc460 2024-09-21T06:28:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.433+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 >> 172.21.15.37:0/1167256325 conn(0x7f434009d830 msgr2=0x7f4340160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.434+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 shutdown_connections 2024-09-21T06:28:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.434+0000 7f4344c00640 1 -- 172.21.15.37:0/1167256325 wait complete. 2024-09-21T06:28:24.436 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 msgr2=0x7fcac807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 0x7fcac807b1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcad4000c00 tx=0x7fcad4006040 comp rx=0 tx=0).stop 2024-09-21T06:28:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 msgr2=0x7fcaec1f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 0x7fcaec1f1440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ba5f0 tx=0x7fcae400f3c0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 shutdown_connections 2024-09-21T06:28:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcac8078c90 0x7fcac807b1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaec0cc460 0x7fcaec1f1440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaec06cde0 0x7fcaec0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 --2- 172.21.15.37:0/3247877781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcad0004800 0x7fcaec14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.437+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 >> 172.21.15.37:0/3247877781 conn(0x7fcaec09d830 msgr2=0x7fcaec15c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.439+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 shutdown_connections 2024-09-21T06:28:24.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.439+0000 7fcaf3a00640 1 -- 172.21.15.37:0/3247877781 wait complete. 2024-09-21T06:28:24.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-09-21T06:28:24.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-09-21T06:28:24.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-09-21T06:28:24.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2024-09-21T06:28:24.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-09-21T06:28:24.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.463+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff4a4005580 con 0x7ff4cc162890 2024-09-21T06:28:24.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.464+0000 7ff4c2000640 1 -- 172.21.15.37:0/85838593 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4cc072ab0 con 0x7ff4cc162890 2024-09-21T06:28:24.465 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 msgr2=0x7ff4ac07b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 0x7ff4ac07b2a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009420 tx=0x7ff4c4004ab0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 msgr2=0x7ff4cc148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 0x7ff4cc148610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc08d640 tx=0x7ff4b800d9f0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 shutdown_connections 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff4ac078d90 0x7ff4ac07b2a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4cc162890 0x7ff4cc148610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4cc15a5b0 0x7ff4cc1480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 --2- 172.21.15.37:0/85838593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4cc0ced80 0x7ff4cc072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 >> 172.21.15.37:0/85838593 conn(0x7ff4cc0d4290 msgr2=0x7ff4cc0b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.466+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 shutdown_connections 2024-09-21T06:28:24.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:24.467+0000 7ff4d1000640 1 -- 172.21.15.37:0/85838593 wait complete. 2024-09-21T06:28:24.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-21T06:28:24.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-09-21T06:28:25.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T06:28:25.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T06:28:25.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T06:28:25.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T06:28:25.460 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T06:28:25.465 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T06:28:25.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T06:28:25.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T06:28:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.804+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3815944028 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 msgr2=0x7ff2cc0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.804+0000 7ff2d2e00640 1 --2- 172.21.15.37:0/3815944028 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc0b31e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009900 tx=0x7ff2bc02a670 comp rx=0 tx=0).stop 2024-09-21T06:28:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.804+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3815944028 shutdown_connections 2024-09-21T06:28:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.804+0000 7ff2d2e00640 1 --2- 172.21.15.37:0/3815944028 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc0b31e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.804+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3815944028 >> 172.21.15.37:0/3815944028 conn(0x7ff2cc0d0350 msgr2=0x7ff2cc0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.805+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3815944028 shutdown_connections 2024-09-21T06:28:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.805+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3815944028 wait complete. 2024-09-21T06:28:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.810+0000 7ff2d2e00640 1 Processor -- start 2024-09-21T06:28:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.810+0000 7ff2d2e00640 1 -- start start 2024-09-21T06:28:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.813+0000 7ff2d2e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 0x7ff2cc13f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.813+0000 7ff2d2e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 0x7ff2cc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.814+0000 7ff2d2e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.814+0000 7ff2d2e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc15f5e0 con 0x7ff2cc06f500 2024-09-21T06:28:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.814+0000 7ff2d2e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc15f460 con 0x7ff2cc074880 2024-09-21T06:28:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.814+0000 7ff2d2e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc15f760 con 0x7ff2cc0af4e0 2024-09-21T06:28:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.815+0000 7ff2cb400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 0x7ff2cc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.815+0000 7ff2cb400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 0x7ff2cc13fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:41084/0 (socket says 172.21.15.37:41084) 2024-09-21T06:28:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.815+0000 7ff2cb400640 1 -- 172.21.15.37:0/3721012795 learned_addr learned my addr 172.21.15.37:0/3721012795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.815+0000 7ff2d0e00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.816+0000 7ff2cbe00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 0x7ff2cc13f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.817+0000 7ff2cbe00640 1 -- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 msgr2=0x7ff2cc13ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.817+0000 7ff2cbe00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc13ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.817+0000 7ff2cbe00640 1 -- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 msgr2=0x7ff2cc13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.817+0000 7ff2cbe00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 0x7ff2cc13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.817+0000 7ff2cbe00640 1 -- 172.21.15.37:0/3721012795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc0ce240 con 0x7ff2cc06f500 2024-09-21T06:28:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.818+0000 7ff2cbe00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 0x7ff2cc13f270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc031000 tx=0x7ff2bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.819+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc035040 con 0x7ff2cc06f500 2024-09-21T06:28:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.819+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2bc0039b0 con 0x7ff2cc06f500 2024-09-21T06:28:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.820+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc003dc0 con 0x7ff2cc06f500 2024-09-21T06:28:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.820+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3721012795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc0ce520 con 0x7ff2cc06f500 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.820+0000 7ff2d2e00640 1 -- 172.21.15.37:0/3721012795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc0ceae0 con 0x7ff2cc06f500 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.827+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2bc0047a0 con 0x7ff2cc06f500 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.828+0000 7ff2c3e00640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 0x7ff2ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.829+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff2bc0c3ea0 con 0x7ff2cc06f500 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.830+0000 7ff2cb400640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 0x7ff2ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.830+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc15ed50 con 0x7ff2cc06f500 2024-09-21T06:28:25.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.841+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2bc044070 con 0x7ff2cc06f500 2024-09-21T06:28:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.842+0000 7ff2cb400640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 0x7ff2ac07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc156220 tx=0x7ff2b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.880+0000 7ff27f200640 1 -- 172.21.15.37:0/1170883192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff278056da0 msgr2=0x7ff278057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.880+0000 7ff27f200640 1 --2- 172.21.15.37:0/1170883192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff278056da0 0x7ff278057c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009830 tx=0x7ff26c02a520 comp rx=0 tx=0).stop 2024-09-21T06:28:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7ff27f200640 1 -- 172.21.15.37:0/1170883192 shutdown_connections 2024-09-21T06:28:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 -- 172.21.15.37:0/898987580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 msgr2=0x7f3614006650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 --2- 172.21.15.37:0/898987580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614006650 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f361c07ad70 tx=0x7f361c0a6350 comp rx=0 tx=0).stop 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7ff27f200640 1 --2- 172.21.15.37:0/1170883192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff278056da0 0x7ff278057c80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7ff27f200640 1 -- 172.21.15.37:0/1170883192 >> 172.21.15.37:0/1170883192 conn(0x7ff27803a5f0 msgr2=0x7ff27803aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 -- 172.21.15.37:0/898987580 shutdown_connections 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 --2- 172.21.15.37:0/898987580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3614036b30 0x7f36140df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 --2- 172.21.15.37:0/898987580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614006650 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 -- 172.21.15.37:0/898987580 >> 172.21.15.37:0/898987580 conn(0x7f3614051410 msgr2=0x7f3614051850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7ff27f200640 1 -- 172.21.15.37:0/1170883192 shutdown_connections 2024-09-21T06:28:25.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7f3622c00640 1 -- 172.21.15.37:0/898987580 shutdown_connections 2024-09-21T06:28:25.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.881+0000 7ff27f200640 1 -- 172.21.15.37:0/1170883192 wait complete. 2024-09-21T06:28:25.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.883+0000 7f3622c00640 1 -- 172.21.15.37:0/898987580 wait complete. 2024-09-21T06:28:25.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.885+0000 7ff27f200640 1 Processor -- start 2024-09-21T06:28:25.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.885+0000 7ff27f200640 1 -- start start 2024-09-21T06:28:25.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 -- 172.21.15.37:0/2462774307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cf6b0 msgr2=0x7f2e3815f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2462774307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cf6b0 0x7f2e3815f5b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009830 tx=0x7f2e2802a5b0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 -- 172.21.15.37:0/2462774307 shutdown_connections 2024-09-21T06:28:25.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2462774307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cf6b0 0x7f2e3815f5b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 -- 172.21.15.37:0/2462774307 >> 172.21.15.37:0/2462774307 conn(0x7f2e380b1b90 msgr2=0x7f2e380b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 0x7ff27804f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 0x7ff2780d6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 0x7ff278050ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780072a0 con 0x7ff278056da0 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 -- 172.21.15.37:0/2462774307 shutdown_connections 2024-09-21T06:28:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f2e3f800640 1 -- 172.21.15.37:0/2462774307 wait complete. 2024-09-21T06:28:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff27803c6d0 con 0x7ff2780e2f40 2024-09-21T06:28:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7ff27f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff278007420 con 0x7ff278054360 2024-09-21T06:28:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.888+0000 7f3622c00640 1 Processor -- start 2024-09-21T06:28:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27de00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 0x7ff27804f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27de00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 0x7ff27804f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58212/0 (socket says 172.21.15.37:58212) 2024-09-21T06:28:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 0x7ff278050ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7f3622c00640 1 -- start start 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 0x7ff278050ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:41128/0 (socket says 172.21.15.37:41128) 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27e800640 1 -- 172.21.15.37:0/2217702738 learned_addr learned my addr 172.21.15.37:0/2217702738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.889+0000 7ff27d400640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 0x7ff2780d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.890+0000 7ff27d400640 1 -- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 msgr2=0x7ff27804f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.890+0000 7ff27d400640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 0x7ff27804f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.890+0000 7ff27d400640 1 -- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 msgr2=0x7ff278050ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.890+0000 7ff27d400640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 0x7ff278050ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.890+0000 7ff27d400640 1 -- 172.21.15.37:0/2217702738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff278030450 con 0x7ff278056da0 2024-09-21T06:28:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.891+0000 7ff27d400640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 0x7ff2780d6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c760 tx=0x7ff27000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.891+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2700050d0 con 0x7ff278056da0 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.891+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff270005270 con 0x7ff278056da0 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.891+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2700035f0 con 0x7ff278056da0 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.891+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2780df2c0 con 0x7ff278056da0 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614021be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3614009b90 0x7f36140e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3614006b90 con 0x7f3614009b90 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f361400be30 con 0x7f3614005c50 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f3622c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3614006d10 con 0x7f36140e56c0 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.892+0000 7f2e3f800640 1 Processor -- start 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27804e670 con 0x7ff278056da0 2024-09-21T06:28:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7f3620c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7f3620c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58226/0 (socket says 172.21.15.37:58226) 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7f3620c00640 1 -- 172.21.15.37:0/1125329891 learned_addr learned my addr 172.21.15.37:0/1125329891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7f361be00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614021be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.893+0000 7f2e3f800640 1 -- start start 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.894+0000 7f3620c00640 1 -- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 msgr2=0x7f3614021be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.894+0000 7f3620c00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614021be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.894+0000 7f3620c00640 1 -- 172.21.15.37:0/1125329891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3614009b90 msgr2=0x7f36140e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.894+0000 7f3620c00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3614009b90 0x7f36140e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.894+0000 7f3620c00640 1 -- 172.21.15.37:0/1125329891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361401e5a0 con 0x7f36140e56c0 2024-09-21T06:28:25.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff278057c80 con 0x7ff278056da0 2024-09-21T06:28:25.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f3620c00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f361c06bcf0 tx=0x7f361c0aed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0b3050 con 0x7f36140e56c0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f361c0bf070 con 0x7f36140e56c0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0b8b00 con 0x7f36140e56c0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f3622c00640 1 -- 172.21.15.37:0/1125329891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361401e880 con 0x7f36140e56c0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.895+0000 7f3622c00640 1 -- 172.21.15.37:0/1125329891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f361401ee40 con 0x7f36140e56c0 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7fee69600640 1 -- 172.21.15.37:0/3710078732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee64162880 msgr2=0x7fee640cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7fee69600640 1 --2- 172.21.15.37:0/3710078732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee64162880 0x7fee640cf4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fee54005fd0 tx=0x7fee5402a930 comp rx=0 tx=0).stop 2024-09-21T06:28:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38073000 0x7f2e380b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff270003790 con 0x7ff278056da0 2024-09-21T06:28:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7fee69600640 1 -- 172.21.15.37:0/3710078732 shutdown_connections 2024-09-21T06:28:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7fee69600640 1 --2- 172.21.15.37:0/3710078732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee64162880 0x7fee640cf4b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7fee69600640 1 -- 172.21.15.37:0/3710078732 >> 172.21.15.37:0/3710078732 conn(0x7fee640d4360 msgr2=0x7fee640d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3614038ef0 con 0x7f36140e56c0 2024-09-21T06:28:25.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e380cf6b0 0x7f2e3815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3806ddf0 con 0x7f2e38073000 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3815fd90 con 0x7f2e380cfac0 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3f800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3806df70 con 0x7f2e380cf6b0 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:41148/0 (socket says 172.21.15.37:41148) 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.896+0000 7f2e3d800640 1 -- 172.21.15.37:0/2535510030 learned_addr learned my addr 172.21.15.37:0/2535510030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f2e3d800640 1 -- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e380cf6b0 msgr2=0x7f2e3815d6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f2e3d800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e380cf6b0 0x7f2e3815d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f2e3d800640 1 -- 172.21.15.37:0/2535510030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38073000 msgr2=0x7f2e380b57c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f2e3d800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38073000 0x7f2e380b57c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.897+0000 7f2e3d800640 1 -- 172.21.15.37:0/2535510030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e380b2f70 con 0x7f2e380cfac0 2024-09-21T06:28:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.899+0000 7ff276000640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 0x7ff26007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.899+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff27009a7a0 con 0x7ff278056da0 2024-09-21T06:28:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.900+0000 7ff27de00640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 0x7ff26007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.898+0000 7fee69600640 1 -- 172.21.15.37:0/3710078732 shutdown_connections 2024-09-21T06:28:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.898+0000 7fee69600640 1 -- 172.21.15.37:0/3710078732 wait complete. 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.900+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f361c0b8ca0 con 0x7f36140e56c0 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.900+0000 7f2e3d800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c3c0 tx=0x7f2e20001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.901+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20017070 con 0x7f2e380cfac0 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.901+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e200053c0 con 0x7f2e380cfac0 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.901+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003850 con 0x7f2e380cfac0 2024-09-21T06:28:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.901+0000 7ff27de00640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 0x7ff26007b170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff26c034000 tx=0x7ff26c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.901+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e380b3250 con 0x7f2e380cfac0 2024-09-21T06:28:25.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.902+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e380b3810 con 0x7f2e380cfac0 2024-09-21T06:28:25.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.903+0000 7fee69600640 1 Processor -- start 2024-09-21T06:28:25.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.903+0000 7fee69600640 1 -- start start 2024-09-21T06:28:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.904+0000 7f360fe00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 0x7f35fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.904+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f361c137b60 con 0x7f36140e56c0 2024-09-21T06:28:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.905+0000 7f361be00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 0x7f35fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.905+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f361c0ffd90 con 0x7f36140e56c0 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee640cf9f0 0x7fee640ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 0x7fee640c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 0x7fee640cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee64154f10 con 0x7fee640d2760 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee640b5700 con 0x7fee640cf9f0 2024-09-21T06:28:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.906+0000 7fee69600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fee64155090 con 0x7fee64153510 2024-09-21T06:28:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.907+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff270020020 con 0x7ff278056da0 2024-09-21T06:28:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.907+0000 7f361be00640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 0x7f35fc07b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36100046b0 tx=0x7f3610009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.907+0000 7fee68c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 0x7fee640cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.907+0000 7fee68c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 0x7fee640cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58248/0 (socket says 172.21.15.37:58248) 2024-09-21T06:28:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.907+0000 7fee68c00640 1 -- 172.21.15.37:0/3798563854 learned_addr learned my addr 172.21.15.37:0/3798563854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.908+0000 7fee63400640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 0x7fee640c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.908+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3806e460 con 0x7f2e380cfac0 2024-09-21T06:28:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 -- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 msgr2=0x7fee640cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 0x7fee640cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 -- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee640cf9f0 msgr2=0x7fee640ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee640cf9f0 0x7fee640ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 -- 172.21.15.37:0/3798563854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee64158480 con 0x7fee640d2760 2024-09-21T06:28:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7fee63400640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 0x7fee640c8c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee54009000 tx=0x7fee540039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7f58b1800640 1 -- 172.21.15.37:0/2211816196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac15e1c0 msgr2=0x7f58ac15e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.909+0000 7f58b1800640 1 --2- 172.21.15.37:0/2211816196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac15e1c0 0x7f58ac15e5d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58ac072940 tx=0x7f58a402a6c0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54030c60 con 0x7fee640d2760 2024-09-21T06:28:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7f58b1800640 1 -- 172.21.15.37:0/2211816196 shutdown_connections 2024-09-21T06:28:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7f58b1800640 1 --2- 172.21.15.37:0/2211816196 >> v2:172.21.15.37:3300/0 conn(0x7f58ac06edf0 0x7f58ac06f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7f58b1800640 1 --2- 172.21.15.37:0/2211816196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac15e1c0 0x7f58ac15e5d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7f58b1800640 1 -- 172.21.15.37:0/2211816196 >> 172.21.15.37:0/2211816196 conn(0x7f58ac0d05c0 msgr2=0x7f58ac06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee5403e070 con 0x7fee640d2760 2024-09-21T06:28:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.910+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54032330 con 0x7fee640d2760 2024-09-21T06:28:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.911+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee640cca80 con 0x7fee640d2760 2024-09-21T06:28:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.911+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee640c8420 con 0x7fee640d2760 2024-09-21T06:28:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.911+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2e2001a070 con 0x7f2e380cfac0 2024-09-21T06:28:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.912+0000 7f58b1800640 1 -- 172.21.15.37:0/2211816196 shutdown_connections 2024-09-21T06:28:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.913+0000 7f58b1800640 1 -- 172.21.15.37:0/2211816196 wait complete. 2024-09-21T06:28:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.916+0000 7f2e34c00640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 0x7f2e1407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.916+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2e2009a3a0 con 0x7f2e380cfac0 2024-09-21T06:28:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.917+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee640cf4b0 con 0x7fee640d2760 2024-09-21T06:28:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.917+0000 7f58b1800640 1 Processor -- start 2024-09-21T06:28:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.917+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2e200625d0 con 0x7f2e380cfac0 2024-09-21T06:28:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.917+0000 7f2e3ce00640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 0x7f2e1407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.917+0000 7f58b1800640 1 -- start start 2024-09-21T06:28:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.919+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee540324d0 con 0x7fee640d2760 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.919+0000 7f2e3ce00640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 0x7f2e1407b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e30003df0 tx=0x7f2e30003d60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 -- 172.21.15.37:0/1784447755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00b63e0 msgr2=0x7f5ed00b67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/1784447755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00b63e0 0x7f5ed00b67f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0005fd0 tx=0x7f5ec002a860 comp rx=0 tx=0).stop 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 -- 172.21.15.37:0/1784447755 shutdown_connections 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/1784447755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00b63e0 0x7f5ed00b67f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/1784447755 >> v2:172.21.15.150:3300/0 conn(0x7f5ed00d0c20 0x7f5ed00d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.920+0000 7f5ed5e00640 1 -- 172.21.15.37:0/1784447755 >> 172.21.15.37:0/1784447755 conn(0x7f5ed009d830 msgr2=0x7f5ed00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7f5ed5e00640 1 -- 172.21.15.37:0/1784447755 shutdown_connections 2024-09-21T06:28:25.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7fee5be00640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 0x7fee4007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7f58b1800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58ac06edf0 0x7f58ac1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7f58b1800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac151080 0x7f58ac1f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7f58b1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fee540327c0 con 0x7fee640d2760 2024-09-21T06:28:25.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7fee63e00640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 0x7fee4007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b1800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06f740 con 0x7f58ac15e1c0 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b1800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06f8c0 con 0x7f58ac06edf0 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b1800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac06fa40 con 0x7f58ac151080 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40360/0 (socket says 172.21.15.37:40360) 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.922+0000 7f58b0e00640 1 -- 172.21.15.37:0/221436111 learned_addr learned my addr 172.21.15.37:0/221436111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.921+0000 7f5ed5e00640 1 -- 172.21.15.37:0/1784447755 wait complete. 2024-09-21T06:28:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58abe00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58ac06edf0 0x7f58ac1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58ab400640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac151080 0x7f58ac1f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58b0e00640 1 -- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac151080 msgr2=0x7f58ac1f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58b0e00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac151080 0x7f58ac1f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58b0e00640 1 -- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58ac06edf0 msgr2=0x7f58ac1f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58b0e00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58ac06edf0 0x7f58ac1f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.923+0000 7f58b0e00640 1 -- 172.21.15.37:0/221436111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac1fa600 con 0x7f58ac15e1c0 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.924+0000 7f58b0e00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5898003930 tx=0x7f5898003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.924+0000 7fee63e00640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 0x7fee4007b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f60 tx=0x7fee5c0079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.925+0000 7f5ed5e00640 1 Processor -- start 2024-09-21T06:28:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.925+0000 7f5ed5e00640 1 -- start start 2024-09-21T06:28:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.928+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 0x7f5ed0150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.929+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 0x7f5ed0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.929+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.929+0000 7f5ed5e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00b6eb0 con 0x7f5ed00b63e0 2024-09-21T06:28:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ec6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 0x7f5ed0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ecf400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ecf400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58276/0 (socket says 172.21.15.37:58276) 2024-09-21T06:28:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ecf400640 1 -- 172.21.15.37:0/598083229 learned_addr learned my addr 172.21.15.37:0/598083229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00b6d30 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed015ee80 con 0x7f5ed00d0c20 2024-09-21T06:28:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.930+0000 7f5ecea00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 0x7f5ed0150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.931+0000 7f5ec6000640 1 -- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 msgr2=0x7f5ed0155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.931+0000 7f5ec6000640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.931+0000 7f5ec6000640 1 -- 172.21.15.37:0/598083229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 msgr2=0x7f5ed0150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f5ec6000640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 0x7f5ed0150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f5ec6000640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed0152770 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f5ecf400640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f5ecea00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 0x7f5ed0150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898009cc0 con 0x7f58ac15e1c0 2024-09-21T06:28:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.932+0000 7f5ec6000640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 0x7f5ed0152230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00d20f0 tx=0x7f5ec0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004240 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec0003e50 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004660 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac1fa8e0 con 0x7f58ac15e1c0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac1faea0 con 0x7f58ac15e1c0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5898005020 con 0x7f58ac15e1c0 2024-09-21T06:28:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898009e60 con 0x7f58ac15e1c0 2024-09-21T06:28:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed0152bf0 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.933+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed01531b0 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.934+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee540847a0 con 0x7fee640d2760 2024-09-21T06:28:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.935+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5884005290 con 0x7f58ac15e1c0 2024-09-21T06:28:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.937+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5898005450 con 0x7f58ac15e1c0 2024-09-21T06:28:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.938+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed0152940 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.939+0000 7f58a3e00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 0x7f588c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.939+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f589809b020 con 0x7f58ac15e1c0 2024-09-21T06:28:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.939+0000 7f58abe00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 0x7f588c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.938+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ec000c040 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.940+0000 7f5ec7e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 0x7f5eac07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.940+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5ec00bc250 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.940+0000 7f58abe00640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 0x7f588c07b120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58a4033000 tx=0x7f58a4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.942+0000 7f5ecea00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 0x7f5eac07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.943+0000 7f5ecea00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 0x7f5eac07b1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006fd0 tx=0x7f5eb8009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.946+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ed00b3b40 con 0x7f5ed00b9ad0 2024-09-21T06:28:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.947+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5898063110 con 0x7f58ac15e1c0 2024-09-21T06:28:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 -- 172.21.15.37:0/3064491724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 msgr2=0x7f02e80d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 --2- 172.21.15.37:0/3064491724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80d26a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f02e0005fd0 tx=0x7f02e002a560 comp rx=0 tx=0).stop 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 -- 172.21.15.37:0/3064491724 shutdown_connections 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 --2- 172.21.15.37:0/3064491724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80d26a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 --2- 172.21.15.37:0/3064491724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cef00 0x7f02e80cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.955+0000 7f02eda00640 1 -- 172.21.15.37:0/3064491724 >> 172.21.15.37:0/3064491724 conn(0x7f02e80d41e0 msgr2=0x7f02e80d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.956+0000 7f02eda00640 1 -- 172.21.15.37:0/3064491724 shutdown_connections 2024-09-21T06:28:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.956+0000 7f02eda00640 1 -- 172.21.15.37:0/3064491724 wait complete. 2024-09-21T06:28:25.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.960+0000 7f02eda00640 1 Processor -- start 2024-09-21T06:28:25.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.960+0000 7f02eda00640 1 -- start start 2024-09-21T06:28:25.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.963+0000 7f02eda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.964+0000 7f02eda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02e815f6d0 0x7f02e8149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.964+0000 7f02eda00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.964+0000 7f02eda00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f02e80d2d60 con 0x7f02e80d2290 2024-09-21T06:28:25.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.964+0000 7f02eda00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02e80d2be0 con 0x7f02e815f6d0 2024-09-21T06:28:25.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.964+0000 7f02ed000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.965+0000 7f02ed000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:58284/0 (socket says 172.21.15.37:58284) 2024-09-21T06:28:25.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.965+0000 7f02ed000640 1 -- 172.21.15.37:0/306989248 learned_addr learned my addr 172.21.15.37:0/306989248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:25.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.965+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f02e80d2ee0 con 0x7f02e80cd460 2024-09-21T06:28:25.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02e7e00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02ed000640 1 -- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02e815f6d0 msgr2=0x7f02e8149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02ed000640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02e815f6d0 0x7f02e8149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02ed000640 1 -- 172.21.15.37:0/306989248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 msgr2=0x7f02e80b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02ed000640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.966+0000 7f02ed000640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e80cddb0 con 0x7f02e80cd460 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02ed000640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006010 tx=0x7f02e0003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e003e030 con 0x7f02e80cd460 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02e0003e50 con 0x7f02e80cd460 2024-09-21T06:28:25.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0035040 con 0x7f02e80cd460 2024-09-21T06:28:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e80caf60 con 0x7f02e80cd460 2024-09-21T06:28:25.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.967+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81f5bb0 con 0x7f02e80cd460 2024-09-21T06:28:25.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.969+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e80d3460 con 0x7f02e80cd460 2024-09-21T06:28:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.974+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f02e0041030 con 0x7f02e80cd460 2024-09-21T06:28:25.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.975+0000 7f02e4c00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 0x7f02c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.976+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f02e00bbdd0 con 0x7f02e80cd460 2024-09-21T06:28:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.976+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02e80cddb0 con 0x7f02e80cd460 2024-09-21T06:28:25.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.978+0000 7f02e7e00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 0x7f02c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:25.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:25.979+0000 7f02e7e00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 0x7f02c407b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006fd0 tx=0x7f02d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:26.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.270+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff2cc0cf7f0 con 0x7ff2cc06f500 2024-09-21T06:28:26.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.271+0000 7ff2c3e00640 1 -- 172.21.15.37:0/3721012795 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2bc08b8c0 con 0x7ff2cc06f500 2024-09-21T06:28:26.275 INFO:teuthology.orchestra.run.smithi037.stdout:64424509452 2024-09-21T06:28:26.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.276+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 msgr2=0x7ff2ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.276+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 0x7ff2ac07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc156220 tx=0x7ff2b8009040 comp rx=0 tx=0).stop 2024-09-21T06:28:26.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.276+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 msgr2=0x7ff2cc13f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.276+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 0x7ff2cc13f270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc031000 tx=0x7ff2bc0045e0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.277+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 shutdown_connections 2024-09-21T06:28:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.277+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2ac078b90 0x7ff2ac07b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.277+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2cc0af4e0 0x7ff2cc13ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.278+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2cc074880 0x7ff2cc13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.278+0000 7ff2c1600640 1 --2- 172.21.15.37:0/3721012795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2cc06f500 0x7ff2cc13f270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.278+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 >> 172.21.15.37:0/3721012795 conn(0x7ff2cc0d0350 msgr2=0x7ff2cc06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.278+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 shutdown_connections 2024-09-21T06:28:26.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.278+0000 7ff2c1600640 1 -- 172.21.15.37:0/3721012795 wait complete. 2024-09-21T06:28:26.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.1 2024-09-21T06:28:26.318 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.351+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f36140221c0 con 0x7f36140e56c0 2024-09-21T06:28:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.352+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5884005580 con 0x7f58ac15e1c0 2024-09-21T06:28:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.352+0000 7f360fe00640 1 -- 172.21.15.37:0/1125329891 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f361c0ff690 con 0x7f36140e56c0 2024-09-21T06:28:26.353 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-09-21T06:28:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.352+0000 7f58a3e00640 1 -- 172.21.15.37:0/221436111 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5898062a10 con 0x7f58ac15e1c0 2024-09-21T06:28:26.356 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:26.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.355+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff27804c3d0 con 0x7ff278056da0 2024-09-21T06:28:26.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.355+0000 7ff276000640 1 -- 172.21.15.37:0/2217702738 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff270062910 con 0x7ff278056da0 2024-09-21T06:28:26.357 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.355+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 msgr2=0x7f35fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.355+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 0x7f35fc07b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36100046b0 tx=0x7f3610009040 comp rx=0 tx=0).stop 2024-09-21T06:28:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.355+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 msgr2=0x7f361401f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f361c06bcf0 tx=0x7f361c0aed40 comp rx=0 tx=0).stop 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 shutdown_connections 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35fc078bc0 0x7f35fc07b0d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3614009b90 0x7f36140e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36140e56c0 0x7f361401f360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 --2- 172.21.15.37:0/1125329891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3614005c50 0x7f3614021be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 >> 172.21.15.37:0/1125329891 conn(0x7f3614051410 msgr2=0x7f3614052a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 msgr2=0x7f588c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 0x7f588c07b120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58a4033000 tx=0x7f58a4004ab0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 msgr2=0x7f58ac0caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5898003930 tx=0x7f5898003df0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 shutdown_connections 2024-09-21T06:28:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 shutdown_connections 2024-09-21T06:28:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f588c078c10 0x7f588c07b120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58ac15e1c0 0x7f58ac0caf50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58ac151080 0x7f58ac1f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 --2- 172.21.15.37:0/221436111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58ac06edf0 0x7f58ac1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 >> 172.21.15.37:0/221436111 conn(0x7f58ac0d05c0 msgr2=0x7f58ac0b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.356+0000 7f360d600640 1 -- 172.21.15.37:0/1125329891 wait complete. 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 shutdown_connections 2024-09-21T06:28:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.357+0000 7f58b1800640 1 -- 172.21.15.37:0/221436111 wait complete. 2024-09-21T06:28:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.358+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 msgr2=0x7ff26007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.358+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 0x7ff26007b170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff26c034000 tx=0x7ff26c004730 comp rx=0 tx=0).stop 2024-09-21T06:28:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.358+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 msgr2=0x7ff2780d6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.358+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 0x7ff2780d6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c760 tx=0x7ff27000cc20 comp rx=0 tx=0).stop 2024-09-21T06:28:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 shutdown_connections 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff260078c60 0x7ff26007b170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278056da0 0x7ff2780d6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2780e2f40 0x7ff278050ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 --2- 172.21.15.37:0/2217702738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff278054360 0x7ff27804f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 >> 172.21.15.37:0/2217702738 conn(0x7ff27803a5f0 msgr2=0x7ff2780e6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 shutdown_connections 2024-09-21T06:28:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.359+0000 7ff27f200640 1 -- 172.21.15.37:0/2217702738 wait complete. 2024-09-21T06:28:26.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-09-21T06:28:26.387 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.390+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2e3815f790 con 0x7f2e380cfac0 2024-09-21T06:28:26.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.391+0000 7f2e34c00640 1 -- 172.21.15.37:0/2535510030 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e20061ed0 con 0x7f2e380cfac0 2024-09-21T06:28:26.392 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-09-21T06:28:26.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-09-21T06:28:26.395 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 msgr2=0x7f2e1407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 0x7f2e1407b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e30003df0 tx=0x7f2e30003d60 comp rx=0 tx=0).stop 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 msgr2=0x7f2e3815dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c3c0 tx=0x7f2e20001e70 comp rx=0 tx=0).stop 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 shutdown_connections 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2e14078c60 0x7f2e1407b170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e380cfac0 0x7f2e3815dc10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e380cf6b0 0x7f2e3815d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 --2- 172.21.15.37:0/2535510030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38073000 0x7f2e380b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.394+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 >> 172.21.15.37:0/2535510030 conn(0x7f2e380b1b90 msgr2=0x7f2e3815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.395+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 shutdown_connections 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.395+0000 7f2e3f800640 1 -- 172.21.15.37:0/2535510030 wait complete. 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.395+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fee640cc7e0 con 0x7fee640d2760 2024-09-21T06:28:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.396+0000 7fee5be00640 1 -- 172.21.15.37:0/3798563854 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee640cc7e0 con 0x7fee640d2760 2024-09-21T06:28:26.398 INFO:teuthology.orchestra.run.smithi037.stdout:64424509451 2024-09-21T06:28:26.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-09-21T06:28:26.399 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 msgr2=0x7fee4007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 0x7fee4007b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f60 tx=0x7fee5c0079b0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 msgr2=0x7fee640c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 0x7fee640c8c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee54009000 tx=0x7fee540039f0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 shutdown_connections 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee40078bc0 0x7fee4007b0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee640d2760 0x7fee640c8c20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee64153510 0x7fee640cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 --2- 172.21.15.37:0/3798563854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee640cf9f0 0x7fee640ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 >> 172.21.15.37:0/3798563854 conn(0x7fee640d4360 msgr2=0x7fee6406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 shutdown_connections 2024-09-21T06:28:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.399+0000 7fee69600640 1 -- 172.21.15.37:0/3798563854 wait complete. 2024-09-21T06:28:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.403+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5ed00d5010 con 0x7f5ed00b9ad0 2024-09-21T06:28:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.403+0000 7f5ec7e00640 1 -- 172.21.15.37:0/598083229 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ec0009850 con 0x7f5ed00b9ad0 2024-09-21T06:28:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 msgr2=0x7f5eac07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 0x7f5eac07b1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006fd0 tx=0x7f5eb8009060 comp rx=0 tx=0).stop 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 msgr2=0x7f5ed0152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 0x7f5ed0152230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00d20f0 tx=0x7f5ec0003ba0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 shutdown_connections 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5eac078c90 0x7f5eac07b1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed00d0c20 0x7f5ed0155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed00b9ad0 0x7f5ed0152230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 --2- 172.21.15.37:0/598083229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00b63e0 0x7f5ed0150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 >> 172.21.15.37:0/598083229 conn(0x7f5ed009d830 msgr2=0x7f5ed00b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.406+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 shutdown_connections 2024-09-21T06:28:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.407+0000 7f5ed5e00640 1 -- 172.21.15.37:0/598083229 wait complete. 2024-09-21T06:28:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.410+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f02e8149950 con 0x7f02e80cd460 2024-09-21T06:28:26.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.411+0000 7f02e4c00640 1 -- 172.21.15.37:0/306989248 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02e80cf850 con 0x7f02e80cd460 2024-09-21T06:28:26.412 INFO:teuthology.orchestra.run.smithi037.stdout:64424509450 2024-09-21T06:28:26.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.414+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 msgr2=0x7f02c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.414+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 0x7f02c407b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006fd0 tx=0x7f02d8008040 comp rx=0 tx=0).stop 2024-09-21T06:28:26.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.414+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 msgr2=0x7f02e80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.414+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006010 tx=0x7f02e0003960 comp rx=0 tx=0).stop 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 shutdown_connections 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02c4078b90 0x7f02c407b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02e80cd460 0x7f02e80cd870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02e815f6d0 0x7f02e8149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 --2- 172.21.15.37:0/306989248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e80d2290 0x7f02e80b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 >> 172.21.15.37:0/306989248 conn(0x7f02e80d41e0 msgr2=0x7f02e80b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 shutdown_connections 2024-09-21T06:28:26.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.415+0000 7f02eda00640 1 -- 172.21.15.37:0/306989248 wait complete. 2024-09-21T06:28:26.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.0 2024-09-21T06:28:26.427 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-09-21T06:28:26.428 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-09-21T06:28:26.438 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-09-21T06:28:26.441 DEBUG:teuthology.parallel:result is None 2024-09-21T06:28:26.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:28:26.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:28:26.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 -- 172.21.15.37:0/4238889658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 msgr2=0x7fa618004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 --2- 172.21.15.37:0/4238889658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa618004060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6240017d0 tx=0x7fa618005830 comp rx=0 tx=0).stop 2024-09-21T06:28:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633400640 1 -- 172.21.15.37:0/4238889658 reap_dead start 2024-09-21T06:28:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 -- 172.21.15.37:0/4238889658 shutdown_connections 2024-09-21T06:28:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 -- 172.21.15.37:0/4238889658 >> 172.21.15.37:0/4238889658 conn(0x7fa6340d1930 msgr2=0x7fa6340d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 -- 172.21.15.37:0/4238889658 shutdown_connections 2024-09-21T06:28:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.708+0000 7fa633e00640 1 -- 172.21.15.37:0/4238889658 wait complete. 2024-09-21T06:28:26.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.710+0000 7fa633e00640 1 Processor -- start 2024-09-21T06:28:26.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.710+0000 7fa633e00640 1 -- start start 2024-09-21T06:28:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 0x7fa6340b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa63414d300 0x7fa63414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa634155020 con 0x7fa618003c10 2024-09-21T06:28:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa634154ea0 con 0x7fa63414d300 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa6341551a0 con 0x7fa6340b9220 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa632a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa632a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40420/0 (socket says 172.21.15.37:40420) 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa632a00640 1 -- 172.21.15.37:0/3731127491 learned_addr learned my addr 172.21.15.37:0/3731127491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa633400640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa63414d300 0x7fa63414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa62a000640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 0x7fa6340b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.712+0000 7fa632a00640 1 -- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 msgr2=0x7fa6340b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa632a00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 0x7fa6340b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa632a00640 1 -- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa63414d300 msgr2=0x7fa63414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa632a00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa63414d300 0x7fa63414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa632a00640 1 -- 172.21.15.37:0/3731127491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341f54f0 con 0x7fa618003c10 2024-09-21T06:28:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa62a000640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 0x7fa6340b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa632a00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa624000f60 tx=0x7fa624006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624043070 con 0x7fa618003c10 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6240038e0 con 0x7fa618003c10 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341f5760 con 0x7fa618003c10 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003d30 con 0x7fa618003c10 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.713+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341f5d20 con 0x7fa618003c10 2024-09-21T06:28:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.714+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa634155730 con 0x7fa618003c10 2024-09-21T06:28:26.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.716+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa624033480 con 0x7fa618003c10 2024-09-21T06:28:26.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.717+0000 7fa62b400640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 0x7fa61007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.717+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa6240bae80 con 0x7fa618003c10 2024-09-21T06:28:26.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.717+0000 7fa62a000640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 0x7fa61007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.718+0000 7fa62a000640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 0x7fa61007b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6341ec4d0 tx=0x7fa61c000f70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.722+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa624083030 con 0x7fa618003c10 2024-09-21T06:28:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.916+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6340b6c70 con 0x7fa610078bc0 2024-09-21T06:28:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.923+0000 7fa62b400640 1 -- 172.21.15.37:0/3731127491 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7fa6340b6c70 con 0x7fa610078bc0 2024-09-21T06:28:26.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:26.938 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:28:26.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 msgr2=0x7fa61007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 0x7fa61007b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6341ec4d0 tx=0x7fa61c000f70 comp rx=0 tx=0).stop 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 msgr2=0x7fa63414fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa624000f60 tx=0x7fa624006940 comp rx=0 tx=0).stop 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 shutdown_connections 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa610078bc0 0x7fa61007b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa63414d300 0x7fa63414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6340b9220 0x7fa6340b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 --2- 172.21.15.37:0/3731127491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618003c10 0x7fa63414fd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.927+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 >> 172.21.15.37:0/3731127491 conn(0x7fa6340d1930 msgr2=0x7fa634160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.928+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 shutdown_connections 2024-09-21T06:28:26.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:26.928+0000 7fa633e00640 1 -- 172.21.15.37:0/3731127491 wait complete. 2024-09-21T06:28:26.952 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2024-09-21T06:28:25.093210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219672,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755048,"statfs":{"total":773094113280,"available":772869169152,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003897"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275298+0000","last_change":"2024-09-21T06:28:00.220046+0000","last_active":"2024-09-21T06:28:06.275298+0000","last_peered":"2024-09-21T06:28:06.275298+0000","last_clean":"2024-09-21T06:28:06.275298+0000","last_became_active":"2024-09-21T06:28:00.219127+0000","last_became_peered":"2024-09-21T06:28:00.219127+0000","last_unstale":"2024-09-21T06:28:06.275298+0000","last_undegraded":"2024-09-21T06:28:06.275298+0000","last_fullsized":"2024-09-21T06:28:06.275298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:49.256177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918731+0000","last_change":"2024-09-21T06:28:01.226620+0000","last_active":"2024-09-21T06:28:06.918731+0000","last_peered":"2024-09-21T06:28:06.918731+0000","last_clean":"2024-09-21T06:28:06.918731+0000","last_became_active":"2024-09-21T06:28:01.224002+0000","last_became_peered":"2024-09-21T06:28:01.224002+0000","last_unstale":"2024-09-21T06:28:06.918731+0000","last_undegraded":"2024-09-21T06:28:06.918731+0000","last_fullsized":"2024-09-21T06:28:06.918731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.430586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276059+0000","last_change":"2024-09-21T06:28:05.289106+0000","last_active":"2024-09-21T06:28:06.276059+0000","last_peered":"2024-09-21T06:28:06.276059+0000","last_clean":"2024-09-21T06:28:06.276059+0000","last_became_active":"2024-09-21T06:28:05.288635+0000","last_became_peered":"2024-09-21T06:28:05.288635+0000","last_unstale":"2024-09-21T06:28:06.276059+0000","last_undegraded":"2024-09-21T06:28:06.276059+0000","last_fullsized":"2024-09-21T06:28:06.276059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:30.614325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918864+0000","last_change":"2024-09-21T06:28:00.216173+0000","last_active":"2024-09-21T06:28:06.918864+0000","last_peered":"2024-09-21T06:28:06.918864+0000","last_clean":"2024-09-21T06:28:06.918864+0000","last_became_active":"2024-09-21T06:28:00.215794+0000","last_became_peered":"2024-09-21T06:28:00.215794+0000","last_unstale":"2024-09-21T06:28:06.918864+0000","last_undegraded":"2024-09-21T06:28:06.918864+0000","last_fullsized":"2024-09-21T06:28:06.918864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:18.915364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918573+0000","last_change":"2024-09-21T06:28:01.238244+0000","last_active":"2024-09-21T06:28:06.918573+0000","last_peered":"2024-09-21T06:28:06.918573+0000","last_clean":"2024-09-21T06:28:06.918573+0000","last_became_active":"2024-09-21T06:28:01.237538+0000","last_became_peered":"2024-09-21T06:28:01.237538+0000","last_unstale":"2024-09-21T06:28:06.918573+0000","last_undegraded":"2024-09-21T06:28:06.918573+0000","last_fullsized":"2024-09-21T06:28:06.918573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:11.476210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271237+0000","last_change":"2024-09-21T06:28:05.267181+0000","last_active":"2024-09-21T06:28:06.271237+0000","last_peered":"2024-09-21T06:28:06.271237+0000","last_clean":"2024-09-21T06:28:06.271237+0000","last_became_active":"2024-09-21T06:28:05.266697+0000","last_became_peered":"2024-09-21T06:28:05.266697+0000","last_unstale":"2024-09-21T06:28:06.271237+0000","last_undegraded":"2024-09-21T06:28:06.271237+0000","last_fullsized":"2024-09-21T06:28:06.271237+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.144842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918246+0000","last_change":"2024-09-21T06:28:00.230877+0000","last_active":"2024-09-21T06:28:06.918246+0000","last_peered":"2024-09-21T06:28:06.918246+0000","last_clean":"2024-09-21T06:28:06.918246+0000","last_became_active":"2024-09-21T06:28:00.230578+0000","last_became_peered":"2024-09-21T06:28:00.230578+0000","last_unstale":"2024-09-21T06:28:06.918246+0000","last_undegraded":"2024-09-21T06:28:06.918246+0000","last_fullsized":"2024-09-21T06:28:06.918246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919137+0000","last_change":"2024-09-21T06:28:01.255720+0000","last_active":"2024-09-21T06:28:06.919137+0000","last_peered":"2024-09-21T06:28:06.919137+0000","last_clean":"2024-09-21T06:28:06.919137+0000","last_became_active":"2024-09-21T06:28:01.254968+0000","last_became_peered":"2024-09-21T06:28:01.254968+0000","last_unstale":"2024-09-21T06:28:06.919137+0000","last_undegraded":"2024-09-21T06:28:06.919137+0000","last_fullsized":"2024-09-21T06:28:06.919137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271315+0000","last_change":"2024-09-21T06:28:05.277963+0000","last_active":"2024-09-21T06:28:06.271315+0000","last_peered":"2024-09-21T06:28:06.271315+0000","last_clean":"2024-09-21T06:28:06.271315+0000","last_became_active":"2024-09-21T06:28:05.277006+0000","last_became_peered":"2024-09-21T06:28:05.277006+0000","last_unstale":"2024-09-21T06:28:06.271315+0000","last_undegraded":"2024-09-21T06:28:06.271315+0000","last_fullsized":"2024-09-21T06:28:06.271315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:35.624735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918079+0000","last_change":"2024-09-21T06:28:00.230976+0000","last_active":"2024-09-21T06:28:06.918079+0000","last_peered":"2024-09-21T06:28:06.918079+0000","last_clean":"2024-09-21T06:28:06.918079+0000","last_became_active":"2024-09-21T06:28:00.230660+0000","last_became_peered":"2024-09-21T06:28:00.230660+0000","last_unstale":"2024-09-21T06:28:06.918079+0000","last_undegraded":"2024-09-21T06:28:06.918079+0000","last_fullsized":"2024-09-21T06:28:06.918079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:59.230778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567463+0000","last_change":"2024-09-21T06:28:01.234299+0000","last_active":"2024-09-21T06:28:06.567463+0000","last_peered":"2024-09-21T06:28:06.567463+0000","last_clean":"2024-09-21T06:28:06.567463+0000","last_became_active":"2024-09-21T06:28:01.233998+0000","last_became_peered":"2024-09-21T06:28:01.233998+0000","last_unstale":"2024-09-21T06:28:06.567463+0000","last_undegraded":"2024-09-21T06:28:06.567463+0000","last_fullsized":"2024-09-21T06:28:06.567463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:08.335780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271460+0000","last_change":"2024-09-21T06:28:05.271851+0000","last_active":"2024-09-21T06:28:06.271460+0000","last_peered":"2024-09-21T06:28:06.271460+0000","last_clean":"2024-09-21T06:28:06.271460+0000","last_became_active":"2024-09-21T06:28:05.265434+0000","last_became_peered":"2024-09-21T06:28:05.265434+0000","last_unstale":"2024-09-21T06:28:06.271460+0000","last_undegraded":"2024-09-21T06:28:06.271460+0000","last_fullsized":"2024-09-21T06:28:06.271460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:39.030315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566989+0000","last_change":"2024-09-21T06:28:01.226065+0000","last_active":"2024-09-21T06:28:06.566989+0000","last_peered":"2024-09-21T06:28:06.566989+0000","last_clean":"2024-09-21T06:28:06.566989+0000","last_became_active":"2024-09-21T06:28:01.225761+0000","last_became_peered":"2024-09-21T06:28:01.225761+0000","last_unstale":"2024-09-21T06:28:06.566989+0000","last_undegraded":"2024-09-21T06:28:06.566989+0000","last_fullsized":"2024-09-21T06:28:06.566989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:02.936599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567107+0000","last_change":"2024-09-21T06:28:00.220356+0000","last_active":"2024-09-21T06:28:06.567107+0000","last_peered":"2024-09-21T06:28:06.567107+0000","last_clean":"2024-09-21T06:28:06.567107+0000","last_became_active":"2024-09-21T06:28:00.219737+0000","last_became_peered":"2024-09-21T06:28:00.219737+0000","last_unstale":"2024-09-21T06:28:06.567107+0000","last_undegraded":"2024-09-21T06:28:06.567107+0000","last_fullsized":"2024-09-21T06:28:06.567107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:58.072689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567253+0000","last_change":"2024-09-21T06:28:05.264666+0000","last_active":"2024-09-21T06:28:06.567253+0000","last_peered":"2024-09-21T06:28:06.567253+0000","last_clean":"2024-09-21T06:28:06.567253+0000","last_became_active":"2024-09-21T06:28:05.264145+0000","last_became_peered":"2024-09-21T06:28:05.264145+0000","last_unstale":"2024-09-21T06:28:06.567253+0000","last_undegraded":"2024-09-21T06:28:06.567253+0000","last_fullsized":"2024-09-21T06:28:06.567253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.384335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276438+0000","last_change":"2024-09-21T06:28:01.238641+0000","last_active":"2024-09-21T06:28:06.276438+0000","last_peered":"2024-09-21T06:28:06.276438+0000","last_clean":"2024-09-21T06:28:06.276438+0000","last_became_active":"2024-09-21T06:28:01.237104+0000","last_became_peered":"2024-09-21T06:28:01.237104+0000","last_unstale":"2024-09-21T06:28:06.276438+0000","last_undegraded":"2024-09-21T06:28:06.276438+0000","last_fullsized":"2024-09-21T06:28:06.276438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919263+0000","last_change":"2024-09-21T06:28:00.222233+0000","last_active":"2024-09-21T06:28:06.919263+0000","last_peered":"2024-09-21T06:28:06.919263+0000","last_clean":"2024-09-21T06:28:06.919263+0000","last_became_active":"2024-09-21T06:28:00.221686+0000","last_became_peered":"2024-09-21T06:28:00.221686+0000","last_unstale":"2024-09-21T06:28:06.919263+0000","last_undegraded":"2024-09-21T06:28:06.919263+0000","last_fullsized":"2024-09-21T06:28:06.919263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:40.731260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917906+0000","last_change":"2024-09-21T06:28:05.275795+0000","last_active":"2024-09-21T06:28:06.917906+0000","last_peered":"2024-09-21T06:28:06.917906+0000","last_clean":"2024-09-21T06:28:06.917906+0000","last_became_active":"2024-09-21T06:28:05.275276+0000","last_became_peered":"2024-09-21T06:28:05.275276+0000","last_unstale":"2024-09-21T06:28:06.917906+0000","last_undegraded":"2024-09-21T06:28:06.917906+0000","last_fullsized":"2024-09-21T06:28:06.917906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:55.953841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273709+0000","last_change":"2024-09-21T06:28:01.231657+0000","last_active":"2024-09-21T06:28:06.273709+0000","last_peered":"2024-09-21T06:28:06.273709+0000","last_clean":"2024-09-21T06:28:06.273709+0000","last_became_active":"2024-09-21T06:28:01.223226+0000","last_became_peered":"2024-09-21T06:28:01.223226+0000","last_unstale":"2024-09-21T06:28:06.273709+0000","last_undegraded":"2024-09-21T06:28:06.273709+0000","last_fullsized":"2024-09-21T06:28:06.273709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274062+0000","last_change":"2024-09-21T06:28:00.227556+0000","last_active":"2024-09-21T06:28:06.274062+0000","last_peered":"2024-09-21T06:28:06.274062+0000","last_clean":"2024-09-21T06:28:06.274062+0000","last_became_active":"2024-09-21T06:28:00.227125+0000","last_became_peered":"2024-09-21T06:28:00.227125+0000","last_unstale":"2024-09-21T06:28:06.274062+0000","last_undegraded":"2024-09-21T06:28:06.274062+0000","last_fullsized":"2024-09-21T06:28:06.274062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:03.936656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277113+0000","last_change":"2024-09-21T06:28:05.283986+0000","last_active":"2024-09-21T06:28:06.277113+0000","last_peered":"2024-09-21T06:28:06.277113+0000","last_clean":"2024-09-21T06:28:06.277113+0000","last_became_active":"2024-09-21T06:28:05.283382+0000","last_became_peered":"2024-09-21T06:28:05.283382+0000","last_unstale":"2024-09-21T06:28:06.277113+0000","last_undegraded":"2024-09-21T06:28:06.277113+0000","last_fullsized":"2024-09-21T06:28:06.277113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.496971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566814+0000","last_change":"2024-09-21T06:28:01.223370+0000","last_active":"2024-09-21T06:28:06.566814+0000","last_peered":"2024-09-21T06:28:06.566814+0000","last_clean":"2024-09-21T06:28:06.566814+0000","last_became_active":"2024-09-21T06:28:01.223018+0000","last_became_peered":"2024-09-21T06:28:01.223018+0000","last_unstale":"2024-09-21T06:28:06.566814+0000","last_undegraded":"2024-09-21T06:28:06.566814+0000","last_fullsized":"2024-09-21T06:28:06.566814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:57.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917258+0000","last_change":"2024-09-21T06:28:00.225692+0000","last_active":"2024-09-21T06:28:06.917258+0000","last_peered":"2024-09-21T06:28:06.917258+0000","last_clean":"2024-09-21T06:28:06.917258+0000","last_became_active":"2024-09-21T06:28:00.224466+0000","last_became_peered":"2024-09-21T06:28:00.224466+0000","last_unstale":"2024-09-21T06:28:06.917258+0000","last_undegraded":"2024-09-21T06:28:06.917258+0000","last_fullsized":"2024-09-21T06:28:06.917258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.485812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271902+0000","last_change":"2024-09-21T06:28:05.282444+0000","last_active":"2024-09-21T06:28:06.271902+0000","last_peered":"2024-09-21T06:28:06.271902+0000","last_clean":"2024-09-21T06:28:06.271902+0000","last_became_active":"2024-09-21T06:28:05.282050+0000","last_became_peered":"2024-09-21T06:28:05.282050+0000","last_unstale":"2024-09-21T06:28:06.271902+0000","last_undegraded":"2024-09-21T06:28:06.271902+0000","last_fullsized":"2024-09-21T06:28:06.271902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274767+0000","last_change":"2024-09-21T06:28:01.225464+0000","last_active":"2024-09-21T06:28:06.274767+0000","last_peered":"2024-09-21T06:28:06.274767+0000","last_clean":"2024-09-21T06:28:06.274767+0000","last_became_active":"2024-09-21T06:28:01.223116+0000","last_became_peered":"2024-09-21T06:28:01.223116+0000","last_unstale":"2024-09-21T06:28:06.274767+0000","last_undegraded":"2024-09-21T06:28:06.274767+0000","last_fullsized":"2024-09-21T06:28:06.274767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567905+0000","last_change":"2024-09-21T06:28:00.230433+0000","last_active":"2024-09-21T06:28:06.567905+0000","last_peered":"2024-09-21T06:28:06.567905+0000","last_clean":"2024-09-21T06:28:06.567905+0000","last_became_active":"2024-09-21T06:28:00.228324+0000","last_became_peered":"2024-09-21T06:28:00.228324+0000","last_unstale":"2024-09-21T06:28:06.567905+0000","last_undegraded":"2024-09-21T06:28:06.567905+0000","last_fullsized":"2024-09-21T06:28:06.567905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:40.653033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271246+0000","last_change":"2024-09-21T06:28:05.280019+0000","last_active":"2024-09-21T06:28:06.271246+0000","last_peered":"2024-09-21T06:28:06.271246+0000","last_clean":"2024-09-21T06:28:06.271246+0000","last_became_active":"2024-09-21T06:28:05.279566+0000","last_became_peered":"2024-09-21T06:28:05.279566+0000","last_unstale":"2024-09-21T06:28:06.271246+0000","last_undegraded":"2024-09-21T06:28:06.271246+0000","last_fullsized":"2024-09-21T06:28:06.271246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:46.736598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275157+0000","last_change":"2024-09-21T06:28:01.227702+0000","last_active":"2024-09-21T06:28:06.275157+0000","last_peered":"2024-09-21T06:28:06.275157+0000","last_clean":"2024-09-21T06:28:06.275157+0000","last_became_active":"2024-09-21T06:28:01.227351+0000","last_became_peered":"2024-09-21T06:28:01.227351+0000","last_unstale":"2024-09-21T06:28:06.275157+0000","last_undegraded":"2024-09-21T06:28:06.275157+0000","last_fullsized":"2024-09-21T06:28:06.275157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275087+0000","last_change":"2024-09-21T06:28:00.227291+0000","last_active":"2024-09-21T06:28:06.275087+0000","last_peered":"2024-09-21T06:28:06.275087+0000","last_clean":"2024-09-21T06:28:06.275087+0000","last_became_active":"2024-09-21T06:28:00.226969+0000","last_became_peered":"2024-09-21T06:28:00.226969+0000","last_unstale":"2024-09-21T06:28:06.275087+0000","last_undegraded":"2024-09-21T06:28:06.275087+0000","last_fullsized":"2024-09-21T06:28:06.275087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:31.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273969+0000","last_change":"2024-09-21T06:28:05.277580+0000","last_active":"2024-09-21T06:28:06.273969+0000","last_peered":"2024-09-21T06:28:06.273969+0000","last_clean":"2024-09-21T06:28:06.273969+0000","last_became_active":"2024-09-21T06:28:05.277134+0000","last_became_peered":"2024-09-21T06:28:05.277134+0000","last_unstale":"2024-09-21T06:28:06.273969+0000","last_undegraded":"2024-09-21T06:28:06.273969+0000","last_fullsized":"2024-09-21T06:28:06.273969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919851+0000","last_change":"2024-09-21T06:28:01.235542+0000","last_active":"2024-09-21T06:28:06.919851+0000","last_peered":"2024-09-21T06:28:06.919851+0000","last_clean":"2024-09-21T06:28:06.919851+0000","last_became_active":"2024-09-21T06:28:01.235235+0000","last_became_peered":"2024-09-21T06:28:01.235235+0000","last_unstale":"2024-09-21T06:28:06.919851+0000","last_undegraded":"2024-09-21T06:28:06.919851+0000","last_fullsized":"2024-09-21T06:28:06.919851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567780+0000","last_change":"2024-09-21T06:28:00.219554+0000","last_active":"2024-09-21T06:28:06.567780+0000","last_peered":"2024-09-21T06:28:06.567780+0000","last_clean":"2024-09-21T06:28:06.567780+0000","last_became_active":"2024-09-21T06:28:00.219180+0000","last_became_peered":"2024-09-21T06:28:00.219180+0000","last_unstale":"2024-09-21T06:28:06.567780+0000","last_undegraded":"2024-09-21T06:28:06.567780+0000","last_fullsized":"2024-09-21T06:28:06.567780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.267700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919430+0000","last_change":"2024-09-21T06:28:05.268962+0000","last_active":"2024-09-21T06:28:06.919430+0000","last_peered":"2024-09-21T06:28:06.919430+0000","last_clean":"2024-09-21T06:28:06.919430+0000","last_became_active":"2024-09-21T06:28:05.268415+0000","last_became_peered":"2024-09-21T06:28:05.268415+0000","last_unstale":"2024-09-21T06:28:06.919430+0000","last_undegraded":"2024-09-21T06:28:06.919430+0000","last_fullsized":"2024-09-21T06:28:06.919430+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.107198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271569+0000","last_change":"2024-09-21T06:28:01.230048+0000","last_active":"2024-09-21T06:28:06.271569+0000","last_peered":"2024-09-21T06:28:06.271569+0000","last_clean":"2024-09-21T06:28:06.271569+0000","last_became_active":"2024-09-21T06:28:01.229747+0000","last_became_peered":"2024-09-21T06:28:01.229747+0000","last_unstale":"2024-09-21T06:28:06.271569+0000","last_undegraded":"2024-09-21T06:28:06.271569+0000","last_fullsized":"2024-09-21T06:28:06.271569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918630+0000","last_change":"2024-09-21T06:28:00.221155+0000","last_active":"2024-09-21T06:28:06.918630+0000","last_peered":"2024-09-21T06:28:06.918630+0000","last_clean":"2024-09-21T06:28:06.918630+0000","last_became_active":"2024-09-21T06:28:00.220728+0000","last_became_peered":"2024-09-21T06:28:00.220728+0000","last_unstale":"2024-09-21T06:28:06.918630+0000","last_undegraded":"2024-09-21T06:28:06.918630+0000","last_fullsized":"2024-09-21T06:28:06.918630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.642076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919099+0000","last_change":"2024-09-21T06:28:05.276248+0000","last_active":"2024-09-21T06:28:06.919099+0000","last_peered":"2024-09-21T06:28:06.919099+0000","last_clean":"2024-09-21T06:28:06.919099+0000","last_became_active":"2024-09-21T06:28:05.271420+0000","last_became_peered":"2024-09-21T06:28:05.271420+0000","last_unstale":"2024-09-21T06:28:06.919099+0000","last_undegraded":"2024-09-21T06:28:06.919099+0000","last_fullsized":"2024-09-21T06:28:06.919099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:03.357365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917672+0000","last_change":"2024-09-21T06:28:01.240349+0000","last_active":"2024-09-21T06:28:06.917672+0000","last_peered":"2024-09-21T06:28:06.917672+0000","last_clean":"2024-09-21T06:28:06.917672+0000","last_became_active":"2024-09-21T06:28:01.239325+0000","last_became_peered":"2024-09-21T06:28:01.239325+0000","last_unstale":"2024-09-21T06:28:06.917672+0000","last_undegraded":"2024-09-21T06:28:06.917672+0000","last_fullsized":"2024-09-21T06:28:06.917672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567402+0000","last_change":"2024-09-21T06:28:00.225175+0000","last_active":"2024-09-21T06:28:06.567402+0000","last_peered":"2024-09-21T06:28:06.567402+0000","last_clean":"2024-09-21T06:28:06.567402+0000","last_became_active":"2024-09-21T06:28:00.224858+0000","last_became_peered":"2024-09-21T06:28:00.224858+0000","last_unstale":"2024-09-21T06:28:06.567402+0000","last_undegraded":"2024-09-21T06:28:06.567402+0000","last_fullsized":"2024-09-21T06:28:06.567402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:41.165675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919693+0000","last_change":"2024-09-21T06:28:05.283982+0000","last_active":"2024-09-21T06:28:06.919693+0000","last_peered":"2024-09-21T06:28:06.919693+0000","last_clean":"2024-09-21T06:28:06.919693+0000","last_became_active":"2024-09-21T06:28:05.283527+0000","last_became_peered":"2024-09-21T06:28:05.283527+0000","last_unstale":"2024-09-21T06:28:06.919693+0000","last_undegraded":"2024-09-21T06:28:06.919693+0000","last_fullsized":"2024-09-21T06:28:06.919693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:17.117103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917636+0000","last_change":"2024-09-21T06:28:01.224161+0000","last_active":"2024-09-21T06:28:06.917636+0000","last_peered":"2024-09-21T06:28:06.917636+0000","last_clean":"2024-09-21T06:28:06.917636+0000","last_became_active":"2024-09-21T06:28:01.223787+0000","last_became_peered":"2024-09-21T06:28:01.223787+0000","last_unstale":"2024-09-21T06:28:06.917636+0000","last_undegraded":"2024-09-21T06:28:06.917636+0000","last_fullsized":"2024-09-21T06:28:06.917636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:03.351441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276110+0000","last_change":"2024-09-21T06:28:00.218460+0000","last_active":"2024-09-21T06:28:06.276110+0000","last_peered":"2024-09-21T06:28:06.276110+0000","last_clean":"2024-09-21T06:28:06.276110+0000","last_became_active":"2024-09-21T06:28:00.218046+0000","last_became_peered":"2024-09-21T06:28:00.218046+0000","last_unstale":"2024-09-21T06:28:06.276110+0000","last_undegraded":"2024-09-21T06:28:06.276110+0000","last_fullsized":"2024-09-21T06:28:06.276110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919620+0000","last_change":"2024-09-21T06:28:05.274088+0000","last_active":"2024-09-21T06:28:06.919620+0000","last_peered":"2024-09-21T06:28:06.919620+0000","last_clean":"2024-09-21T06:28:06.919620+0000","last_became_active":"2024-09-21T06:28:05.273184+0000","last_became_peered":"2024-09-21T06:28:05.273184+0000","last_unstale":"2024-09-21T06:28:06.919620+0000","last_undegraded":"2024-09-21T06:28:06.919620+0000","last_fullsized":"2024-09-21T06:28:06.919620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:17.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918180+0000","last_change":"2024-09-21T06:28:01.230531+0000","last_active":"2024-09-21T06:28:06.918180+0000","last_peered":"2024-09-21T06:28:06.918180+0000","last_clean":"2024-09-21T06:28:06.918180+0000","last_became_active":"2024-09-21T06:28:01.230236+0000","last_became_peered":"2024-09-21T06:28:01.230236+0000","last_unstale":"2024-09-21T06:28:06.918180+0000","last_undegraded":"2024-09-21T06:28:06.918180+0000","last_fullsized":"2024-09-21T06:28:06.918180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.172198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567087+0000","last_change":"2024-09-21T06:28:00.233247+0000","last_active":"2024-09-21T06:28:06.567087+0000","last_peered":"2024-09-21T06:28:06.567087+0000","last_clean":"2024-09-21T06:28:06.567087+0000","last_became_active":"2024-09-21T06:28:00.232905+0000","last_became_peered":"2024-09-21T06:28:00.232905+0000","last_unstale":"2024-09-21T06:28:06.567087+0000","last_undegraded":"2024-09-21T06:28:06.567087+0000","last_fullsized":"2024-09-21T06:28:06.567087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918161+0000","last_change":"2024-09-21T06:28:05.266249+0000","last_active":"2024-09-21T06:28:06.918161+0000","last_peered":"2024-09-21T06:28:06.918161+0000","last_clean":"2024-09-21T06:28:06.918161+0000","last_became_active":"2024-09-21T06:28:05.265779+0000","last_became_peered":"2024-09-21T06:28:05.265779+0000","last_unstale":"2024-09-21T06:28:06.918161+0000","last_undegraded":"2024-09-21T06:28:06.918161+0000","last_fullsized":"2024-09-21T06:28:06.918161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:58.459609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274361+0000","last_change":"2024-09-21T06:28:01.236228+0000","last_active":"2024-09-21T06:28:06.274361+0000","last_peered":"2024-09-21T06:28:06.274361+0000","last_clean":"2024-09-21T06:28:06.274361+0000","last_became_active":"2024-09-21T06:28:01.229874+0000","last_became_peered":"2024-09-21T06:28:01.229874+0000","last_unstale":"2024-09-21T06:28:06.274361+0000","last_undegraded":"2024-09-21T06:28:06.274361+0000","last_fullsized":"2024-09-21T06:28:06.274361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:08.119892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567235+0000","last_change":"2024-09-21T06:28:00.219928+0000","last_active":"2024-09-21T06:28:06.567235+0000","last_peered":"2024-09-21T06:28:06.567235+0000","last_clean":"2024-09-21T06:28:06.567235+0000","last_became_active":"2024-09-21T06:28:00.219561+0000","last_became_peered":"2024-09-21T06:28:00.219561+0000","last_unstale":"2024-09-21T06:28:06.567235+0000","last_undegraded":"2024-09-21T06:28:06.567235+0000","last_fullsized":"2024-09-21T06:28:06.567235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:22.801415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918345+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:28:06.918345+0000","last_peered":"2024-09-21T06:28:06.918345+0000","last_clean":"2024-09-21T06:28:06.918345+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:28:06.918345+0000","last_undegraded":"2024-09-21T06:28:06.918345+0000","last_fullsized":"2024-09-21T06:28:06.918345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567826+0000","last_change":"2024-09-21T06:28:05.274952+0000","last_active":"2024-09-21T06:28:06.567826+0000","last_peered":"2024-09-21T06:28:06.567826+0000","last_clean":"2024-09-21T06:28:06.567826+0000","last_became_active":"2024-09-21T06:28:05.274514+0000","last_became_peered":"2024-09-21T06:28:05.274514+0000","last_unstale":"2024-09-21T06:28:06.567826+0000","last_undegraded":"2024-09-21T06:28:06.567826+0000","last_fullsized":"2024-09-21T06:28:06.567826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:04.636189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567574+0000","last_change":"2024-09-21T06:28:01.224594+0000","last_active":"2024-09-21T06:28:06.567574+0000","last_peered":"2024-09-21T06:28:06.567574+0000","last_clean":"2024-09-21T06:28:06.567574+0000","last_became_active":"2024-09-21T06:28:01.224267+0000","last_became_peered":"2024-09-21T06:28:01.224267+0000","last_unstale":"2024-09-21T06:28:06.567574+0000","last_undegraded":"2024-09-21T06:28:06.567574+0000","last_fullsized":"2024-09-21T06:28:06.567574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:57.968897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919963+0000","last_change":"2024-09-21T06:28:00.226006+0000","last_active":"2024-09-21T06:28:06.919963+0000","last_peered":"2024-09-21T06:28:06.919963+0000","last_clean":"2024-09-21T06:28:06.919963+0000","last_became_active":"2024-09-21T06:28:00.225678+0000","last_became_peered":"2024-09-21T06:28:00.225678+0000","last_unstale":"2024-09-21T06:28:06.919963+0000","last_undegraded":"2024-09-21T06:28:06.919963+0000","last_fullsized":"2024-09-21T06:28:06.919963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:54.460846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271274+0000","last_change":"2024-09-21T06:28:05.280689+0000","last_active":"2024-09-21T06:28:06.271274+0000","last_peered":"2024-09-21T06:28:06.271274+0000","last_clean":"2024-09-21T06:28:06.271274+0000","last_became_active":"2024-09-21T06:28:05.272594+0000","last_became_peered":"2024-09-21T06:28:05.272594+0000","last_unstale":"2024-09-21T06:28:06.271274+0000","last_undegraded":"2024-09-21T06:28:06.271274+0000","last_fullsized":"2024-09-21T06:28:06.271274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:55.258366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917995+0000","last_change":"2024-09-21T06:28:01.230915+0000","last_active":"2024-09-21T06:28:06.917995+0000","last_peered":"2024-09-21T06:28:06.917995+0000","last_clean":"2024-09-21T06:28:06.917995+0000","last_became_active":"2024-09-21T06:28:01.230702+0000","last_became_peered":"2024-09-21T06:28:01.230702+0000","last_unstale":"2024-09-21T06:28:06.917995+0000","last_undegraded":"2024-09-21T06:28:06.917995+0000","last_fullsized":"2024-09-21T06:28:06.917995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:34.107851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918893+0000","last_change":"2024-09-21T06:28:00.214857+0000","last_active":"2024-09-21T06:28:06.918893+0000","last_peered":"2024-09-21T06:28:06.918893+0000","last_clean":"2024-09-21T06:28:06.918893+0000","last_became_active":"2024-09-21T06:28:00.214582+0000","last_became_peered":"2024-09-21T06:28:00.214582+0000","last_unstale":"2024-09-21T06:28:06.918893+0000","last_undegraded":"2024-09-21T06:28:06.918893+0000","last_fullsized":"2024-09-21T06:28:06.918893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:49.842050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918122+0000","last_change":"2024-09-21T06:28:05.273006+0000","last_active":"2024-09-21T06:28:06.918122+0000","last_peered":"2024-09-21T06:28:06.918122+0000","last_clean":"2024-09-21T06:28:06.918122+0000","last_became_active":"2024-09-21T06:28:05.272504+0000","last_became_peered":"2024-09-21T06:28:05.272504+0000","last_unstale":"2024-09-21T06:28:06.918122+0000","last_undegraded":"2024-09-21T06:28:06.918122+0000","last_fullsized":"2024-09-21T06:28:06.918122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:52.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.247357+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.242943+0000","last_became_peered":"2024-09-21T06:28:01.242943+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274916+0000","last_change":"2024-09-21T06:28:00.219919+0000","last_active":"2024-09-21T06:28:06.274916+0000","last_peered":"2024-09-21T06:28:06.274916+0000","last_clean":"2024-09-21T06:28:06.274916+0000","last_became_active":"2024-09-21T06:28:00.219544+0000","last_became_peered":"2024-09-21T06:28:00.219544+0000","last_unstale":"2024-09-21T06:28:06.274916+0000","last_undegraded":"2024-09-21T06:28:06.274916+0000","last_fullsized":"2024-09-21T06:28:06.274916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:21.858722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567045+0000","last_change":"2024-09-21T06:28:05.258119+0000","last_active":"2024-09-21T06:28:06.567045+0000","last_peered":"2024-09-21T06:28:06.567045+0000","last_clean":"2024-09-21T06:28:06.567045+0000","last_became_active":"2024-09-21T06:28:05.257394+0000","last_became_peered":"2024-09-21T06:28:05.257394+0000","last_unstale":"2024-09-21T06:28:06.567045+0000","last_undegraded":"2024-09-21T06:28:06.567045+0000","last_fullsized":"2024-09-21T06:28:06.567045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:13.794574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271877+0000","last_change":"2024-09-21T06:28:01.239234+0000","last_active":"2024-09-21T06:28:06.271877+0000","last_peered":"2024-09-21T06:28:06.271877+0000","last_clean":"2024-09-21T06:28:06.271877+0000","last_became_active":"2024-09-21T06:28:01.238978+0000","last_became_peered":"2024-09-21T06:28:01.238978+0000","last_unstale":"2024-09-21T06:28:06.271877+0000","last_undegraded":"2024-09-21T06:28:06.271877+0000","last_fullsized":"2024-09-21T06:28:06.271877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918626+0000","last_change":"2024-09-21T06:28:00.224968+0000","last_active":"2024-09-21T06:28:06.918626+0000","last_peered":"2024-09-21T06:28:06.918626+0000","last_clean":"2024-09-21T06:28:06.918626+0000","last_became_active":"2024-09-21T06:28:00.224751+0000","last_became_peered":"2024-09-21T06:28:00.224751+0000","last_unstale":"2024-09-21T06:28:06.918626+0000","last_undegraded":"2024-09-21T06:28:06.918626+0000","last_fullsized":"2024-09-21T06:28:06.918626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919013+0000","last_change":"2024-09-21T06:28:05.268891+0000","last_active":"2024-09-21T06:28:06.919013+0000","last_peered":"2024-09-21T06:28:06.919013+0000","last_clean":"2024-09-21T06:28:06.919013+0000","last_became_active":"2024-09-21T06:28:05.268010+0000","last_became_peered":"2024-09-21T06:28:05.268010+0000","last_unstale":"2024-09-21T06:28:06.919013+0000","last_undegraded":"2024-09-21T06:28:06.919013+0000","last_fullsized":"2024-09-21T06:28:06.919013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:04.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566758+0000","last_change":"2024-09-21T06:28:01.225615+0000","last_active":"2024-09-21T06:28:06.566758+0000","last_peered":"2024-09-21T06:28:06.566758+0000","last_clean":"2024-09-21T06:28:06.566758+0000","last_became_active":"2024-09-21T06:28:01.225277+0000","last_became_peered":"2024-09-21T06:28:01.225277+0000","last_unstale":"2024-09-21T06:28:06.566758+0000","last_undegraded":"2024-09-21T06:28:06.566758+0000","last_fullsized":"2024-09-21T06:28:06.566758+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.531813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273831+0000","last_change":"2024-09-21T06:28:00.215632+0000","last_active":"2024-09-21T06:28:06.273831+0000","last_peered":"2024-09-21T06:28:06.273831+0000","last_clean":"2024-09-21T06:28:06.273831+0000","last_became_active":"2024-09-21T06:28:00.215285+0000","last_became_peered":"2024-09-21T06:28:00.215285+0000","last_unstale":"2024-09-21T06:28:06.273831+0000","last_undegraded":"2024-09-21T06:28:06.273831+0000","last_fullsized":"2024-09-21T06:28:06.273831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918621+0000","last_change":"2024-09-21T06:28:05.274989+0000","last_active":"2024-09-21T06:28:06.918621+0000","last_peered":"2024-09-21T06:28:06.918621+0000","last_clean":"2024-09-21T06:28:06.918621+0000","last_became_active":"2024-09-21T06:28:05.274517+0000","last_became_peered":"2024-09-21T06:28:05.274517+0000","last_unstale":"2024-09-21T06:28:06.918621+0000","last_undegraded":"2024-09-21T06:28:06.918621+0000","last_fullsized":"2024-09-21T06:28:06.918621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:09.260032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918268+0000","last_change":"2024-09-21T06:28:01.221685+0000","last_active":"2024-09-21T06:28:06.918268+0000","last_peered":"2024-09-21T06:28:06.918268+0000","last_clean":"2024-09-21T06:28:06.918268+0000","last_became_active":"2024-09-21T06:28:01.221360+0000","last_became_peered":"2024-09-21T06:28:01.221360+0000","last_unstale":"2024-09-21T06:28:06.918268+0000","last_undegraded":"2024-09-21T06:28:06.918268+0000","last_fullsized":"2024-09-21T06:28:06.918268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917445+0000","last_change":"2024-09-21T06:28:00.235459+0000","last_active":"2024-09-21T06:28:06.917445+0000","last_peered":"2024-09-21T06:28:06.917445+0000","last_clean":"2024-09-21T06:28:06.917445+0000","last_became_active":"2024-09-21T06:28:00.235195+0000","last_became_peered":"2024-09-21T06:28:00.235195+0000","last_unstale":"2024-09-21T06:28:06.917445+0000","last_undegraded":"2024-09-21T06:28:06.917445+0000","last_fullsized":"2024-09-21T06:28:06.917445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918627+0000","last_change":"2024-09-21T06:28:05.271996+0000","last_active":"2024-09-21T06:28:06.918627+0000","last_peered":"2024-09-21T06:28:06.918627+0000","last_clean":"2024-09-21T06:28:06.918627+0000","last_became_active":"2024-09-21T06:28:05.270956+0000","last_became_peered":"2024-09-21T06:28:05.270956+0000","last_unstale":"2024-09-21T06:28:06.918627+0000","last_undegraded":"2024-09-21T06:28:06.918627+0000","last_fullsized":"2024-09-21T06:28:06.918627+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:21.181116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566881+0000","last_change":"2024-09-21T06:28:01.237451+0000","last_active":"2024-09-21T06:28:06.566881+0000","last_peered":"2024-09-21T06:28:06.566881+0000","last_clean":"2024-09-21T06:28:06.566881+0000","last_became_active":"2024-09-21T06:28:01.237064+0000","last_became_peered":"2024-09-21T06:28:01.237064+0000","last_unstale":"2024-09-21T06:28:06.566881+0000","last_undegraded":"2024-09-21T06:28:06.566881+0000","last_fullsized":"2024-09-21T06:28:06.566881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567273+0000","last_change":"2024-09-21T06:28:00.224624+0000","last_active":"2024-09-21T06:28:06.567273+0000","last_peered":"2024-09-21T06:28:06.567273+0000","last_clean":"2024-09-21T06:28:06.567273+0000","last_became_active":"2024-09-21T06:28:00.224259+0000","last_became_peered":"2024-09-21T06:28:00.224259+0000","last_unstale":"2024-09-21T06:28:06.567273+0000","last_undegraded":"2024-09-21T06:28:06.567273+0000","last_fullsized":"2024-09-21T06:28:06.567273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919872+0000","last_change":"2024-09-21T06:28:05.283392+0000","last_active":"2024-09-21T06:28:06.919872+0000","last_peered":"2024-09-21T06:28:06.919872+0000","last_clean":"2024-09-21T06:28:06.919872+0000","last_became_active":"2024-09-21T06:28:05.271401+0000","last_became_peered":"2024-09-21T06:28:05.271401+0000","last_unstale":"2024-09-21T06:28:06.919872+0000","last_undegraded":"2024-09-21T06:28:06.919872+0000","last_fullsized":"2024-09-21T06:28:06.919872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:21.192646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919152+0000","last_change":"2024-09-21T06:28:01.224123+0000","last_active":"2024-09-21T06:28:06.919152+0000","last_peered":"2024-09-21T06:28:06.919152+0000","last_clean":"2024-09-21T06:28:06.919152+0000","last_became_active":"2024-09-21T06:28:01.223773+0000","last_became_peered":"2024-09-21T06:28:01.223773+0000","last_unstale":"2024-09-21T06:28:06.919152+0000","last_undegraded":"2024-09-21T06:28:06.919152+0000","last_fullsized":"2024-09-21T06:28:06.919152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:59.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276563+0000","last_change":"2024-09-21T06:28:00.232406+0000","last_active":"2024-09-21T06:28:06.276563+0000","last_peered":"2024-09-21T06:28:06.276563+0000","last_clean":"2024-09-21T06:28:06.276563+0000","last_became_active":"2024-09-21T06:28:00.232143+0000","last_became_peered":"2024-09-21T06:28:00.232143+0000","last_unstale":"2024-09-21T06:28:06.276563+0000","last_undegraded":"2024-09-21T06:28:06.276563+0000","last_fullsized":"2024-09-21T06:28:06.276563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:33.343595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567590+0000","last_change":"2024-09-21T06:28:05.274346+0000","last_active":"2024-09-21T06:28:06.567590+0000","last_peered":"2024-09-21T06:28:06.567590+0000","last_clean":"2024-09-21T06:28:06.567590+0000","last_became_active":"2024-09-21T06:28:05.273714+0000","last_became_peered":"2024-09-21T06:28:05.273714+0000","last_unstale":"2024-09-21T06:28:06.567590+0000","last_undegraded":"2024-09-21T06:28:06.567590+0000","last_fullsized":"2024-09-21T06:28:06.567590+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:09.113671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918127+0000","last_change":"2024-09-21T06:28:01.238637+0000","last_active":"2024-09-21T06:28:06.918127+0000","last_peered":"2024-09-21T06:28:06.918127+0000","last_clean":"2024-09-21T06:28:06.918127+0000","last_became_active":"2024-09-21T06:28:01.236539+0000","last_became_peered":"2024-09-21T06:28:01.236539+0000","last_unstale":"2024-09-21T06:28:06.918127+0000","last_undegraded":"2024-09-21T06:28:06.918127+0000","last_fullsized":"2024-09-21T06:28:06.918127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.692846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:14.350747+0000","last_change":"2024-09-21T06:28:00.217119+0000","last_active":"2024-09-21T06:28:14.350747+0000","last_peered":"2024-09-21T06:28:14.350747+0000","last_clean":"2024-09-21T06:28:14.350747+0000","last_became_active":"2024-09-21T06:28:00.216806+0000","last_became_peered":"2024-09-21T06:28:00.216806+0000","last_unstale":"2024-09-21T06:28:14.350747+0000","last_undegraded":"2024-09-21T06:28:14.350747+0000","last_fullsized":"2024-09-21T06:28:14.350747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:17.032531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274402+0000","last_change":"2024-09-21T06:28:05.273619+0000","last_active":"2024-09-21T06:28:06.274402+0000","last_peered":"2024-09-21T06:28:06.274402+0000","last_clean":"2024-09-21T06:28:06.274402+0000","last_became_active":"2024-09-21T06:28:05.273193+0000","last_became_peered":"2024-09-21T06:28:05.273193+0000","last_unstale":"2024-09-21T06:28:06.274402+0000","last_undegraded":"2024-09-21T06:28:06.274402+0000","last_fullsized":"2024-09-21T06:28:06.274402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:31.630020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567019+0000","last_change":"2024-09-21T06:28:01.233057+0000","last_active":"2024-09-21T06:28:06.567019+0000","last_peered":"2024-09-21T06:28:06.567019+0000","last_clean":"2024-09-21T06:28:06.567019+0000","last_became_active":"2024-09-21T06:28:01.232602+0000","last_became_peered":"2024-09-21T06:28:01.232602+0000","last_unstale":"2024-09-21T06:28:06.567019+0000","last_undegraded":"2024-09-21T06:28:06.567019+0000","last_fullsized":"2024-09-21T06:28:06.567019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:34.157716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276884+0000","last_change":"2024-09-21T06:28:00.218229+0000","last_active":"2024-09-21T06:28:06.276884+0000","last_peered":"2024-09-21T06:28:06.276884+0000","last_clean":"2024-09-21T06:28:06.276884+0000","last_became_active":"2024-09-21T06:28:00.217635+0000","last_became_peered":"2024-09-21T06:28:00.217635+0000","last_unstale":"2024-09-21T06:28:06.276884+0000","last_undegraded":"2024-09-21T06:28:06.276884+0000","last_fullsized":"2024-09-21T06:28:06.276884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271664+0000","last_change":"2024-09-21T06:28:05.277550+0000","last_active":"2024-09-21T06:28:06.271664+0000","last_peered":"2024-09-21T06:28:06.271664+0000","last_clean":"2024-09-21T06:28:06.271664+0000","last_became_active":"2024-09-21T06:28:05.276077+0000","last_became_peered":"2024-09-21T06:28:05.276077+0000","last_unstale":"2024-09-21T06:28:06.271664+0000","last_undegraded":"2024-09-21T06:28:06.271664+0000","last_fullsized":"2024-09-21T06:28:06.271664+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.300870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918857+0000","last_change":"2024-09-21T06:28:01.232773+0000","last_active":"2024-09-21T06:28:06.918857+0000","last_peered":"2024-09-21T06:28:06.918857+0000","last_clean":"2024-09-21T06:28:06.918857+0000","last_became_active":"2024-09-21T06:28:01.232081+0000","last_became_peered":"2024-09-21T06:28:01.232081+0000","last_unstale":"2024-09-21T06:28:06.918857+0000","last_undegraded":"2024-09-21T06:28:06.918857+0000","last_fullsized":"2024-09-21T06:28:06.918857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:54.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917778+0000","last_change":"2024-09-21T06:28:00.234206+0000","last_active":"2024-09-21T06:28:06.917778+0000","last_peered":"2024-09-21T06:28:06.917778+0000","last_clean":"2024-09-21T06:28:06.917778+0000","last_became_active":"2024-09-21T06:28:00.233696+0000","last_became_peered":"2024-09-21T06:28:00.233696+0000","last_unstale":"2024-09-21T06:28:06.917778+0000","last_undegraded":"2024-09-21T06:28:06.917778+0000","last_fullsized":"2024-09-21T06:28:06.917778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275902+0000","last_change":"2024-09-21T06:28:05.283000+0000","last_active":"2024-09-21T06:28:06.275902+0000","last_peered":"2024-09-21T06:28:06.275902+0000","last_clean":"2024-09-21T06:28:06.275902+0000","last_became_active":"2024-09-21T06:28:05.282638+0000","last_became_peered":"2024-09-21T06:28:05.282638+0000","last_unstale":"2024-09-21T06:28:06.275902+0000","last_undegraded":"2024-09-21T06:28:06.275902+0000","last_fullsized":"2024-09-21T06:28:06.275902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271814+0000","last_change":"2024-09-21T06:28:01.221987+0000","last_active":"2024-09-21T06:28:06.271814+0000","last_peered":"2024-09-21T06:28:06.271814+0000","last_clean":"2024-09-21T06:28:06.271814+0000","last_became_active":"2024-09-21T06:28:01.221678+0000","last_became_peered":"2024-09-21T06:28:01.221678+0000","last_unstale":"2024-09-21T06:28:06.271814+0000","last_undegraded":"2024-09-21T06:28:06.271814+0000","last_fullsized":"2024-09-21T06:28:06.271814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:20.730550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919718+0000","last_change":"2024-09-21T06:28:00.202531+0000","last_active":"2024-09-21T06:28:06.919718+0000","last_peered":"2024-09-21T06:28:06.919718+0000","last_clean":"2024-09-21T06:28:06.919718+0000","last_became_active":"2024-09-21T06:28:00.202126+0000","last_became_peered":"2024-09-21T06:28:00.202126+0000","last_unstale":"2024-09-21T06:28:06.919718+0000","last_undegraded":"2024-09-21T06:28:06.919718+0000","last_fullsized":"2024-09-21T06:28:06.919718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274058+0000","last_change":"2024-09-21T06:28:05.278157+0000","last_active":"2024-09-21T06:28:06.274058+0000","last_peered":"2024-09-21T06:28:06.274058+0000","last_clean":"2024-09-21T06:28:06.274058+0000","last_became_active":"2024-09-21T06:28:05.277532+0000","last_became_peered":"2024-09-21T06:28:05.277532+0000","last_unstale":"2024-09-21T06:28:06.274058+0000","last_undegraded":"2024-09-21T06:28:06.274058+0000","last_fullsized":"2024-09-21T06:28:06.274058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.737605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274991+0000","last_change":"2024-09-21T06:28:05.278200+0000","last_active":"2024-09-21T06:28:06.274991+0000","last_peered":"2024-09-21T06:28:06.274991+0000","last_clean":"2024-09-21T06:28:06.274991+0000","last_became_active":"2024-09-21T06:28:05.277575+0000","last_became_peered":"2024-09-21T06:28:05.277575+0000","last_unstale":"2024-09-21T06:28:06.274991+0000","last_undegraded":"2024-09-21T06:28:06.274991+0000","last_fullsized":"2024-09-21T06:28:06.274991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:05.311819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273908+0000","last_change":"2024-09-21T06:28:01.228981+0000","last_active":"2024-09-21T06:28:06.273908+0000","last_peered":"2024-09-21T06:28:06.273908+0000","last_clean":"2024-09-21T06:28:06.273908+0000","last_became_active":"2024-09-21T06:28:01.222592+0000","last_became_peered":"2024-09-21T06:28:01.222592+0000","last_unstale":"2024-09-21T06:28:06.273908+0000","last_undegraded":"2024-09-21T06:28:06.273908+0000","last_fullsized":"2024-09-21T06:28:06.273908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566966+0000","last_change":"2024-09-21T06:28:00.214521+0000","last_active":"2024-09-21T06:28:06.566966+0000","last_peered":"2024-09-21T06:28:06.566966+0000","last_clean":"2024-09-21T06:28:06.566966+0000","last_became_active":"2024-09-21T06:28:00.212408+0000","last_became_peered":"2024-09-21T06:28:00.212408+0000","last_unstale":"2024-09-21T06:28:06.566966+0000","last_undegraded":"2024-09-21T06:28:06.566966+0000","last_fullsized":"2024-09-21T06:28:06.566966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567614+0000","last_change":"2024-09-21T06:28:05.265492+0000","last_active":"2024-09-21T06:28:06.567614+0000","last_peered":"2024-09-21T06:28:06.567614+0000","last_clean":"2024-09-21T06:28:06.567614+0000","last_became_active":"2024-09-21T06:28:05.264988+0000","last_became_peered":"2024-09-21T06:28:05.264988+0000","last_unstale":"2024-09-21T06:28:06.567614+0000","last_undegraded":"2024-09-21T06:28:06.567614+0000","last_fullsized":"2024-09-21T06:28:06.567614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:02.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917735+0000","last_change":"2024-09-21T06:28:01.228392+0000","last_active":"2024-09-21T06:28:06.917735+0000","last_peered":"2024-09-21T06:28:06.917735+0000","last_clean":"2024-09-21T06:28:06.917735+0000","last_became_active":"2024-09-21T06:28:01.228163+0000","last_became_peered":"2024-09-21T06:28:01.228163+0000","last_unstale":"2024-09-21T06:28:06.917735+0000","last_undegraded":"2024-09-21T06:28:06.917735+0000","last_fullsized":"2024-09-21T06:28:06.917735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:24.791034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918738+0000","last_change":"2024-09-21T06:28:00.228862+0000","last_active":"2024-09-21T06:28:06.918738+0000","last_peered":"2024-09-21T06:28:06.918738+0000","last_clean":"2024-09-21T06:28:06.918738+0000","last_became_active":"2024-09-21T06:28:00.228490+0000","last_became_peered":"2024-09-21T06:28:00.228490+0000","last_unstale":"2024-09-21T06:28:06.918738+0000","last_undegraded":"2024-09-21T06:28:06.918738+0000","last_fullsized":"2024-09-21T06:28:06.918738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:09:00.810925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567138+0000","last_change":"2024-09-21T06:28:00.234315+0000","last_active":"2024-09-21T06:28:06.567138+0000","last_peered":"2024-09-21T06:28:06.567138+0000","last_clean":"2024-09-21T06:28:06.567138+0000","last_became_active":"2024-09-21T06:28:00.233567+0000","last_became_peered":"2024-09-21T06:28:00.233567+0000","last_unstale":"2024-09-21T06:28:06.567138+0000","last_undegraded":"2024-09-21T06:28:06.567138+0000","last_fullsized":"2024-09-21T06:28:06.567138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274635+0000","last_change":"2024-09-21T06:28:01.236968+0000","last_active":"2024-09-21T06:28:06.274635+0000","last_peered":"2024-09-21T06:28:06.274635+0000","last_clean":"2024-09-21T06:28:06.274635+0000","last_became_active":"2024-09-21T06:28:01.236577+0000","last_became_peered":"2024-09-21T06:28:01.236577+0000","last_unstale":"2024-09-21T06:28:06.274635+0000","last_undegraded":"2024-09-21T06:28:06.274635+0000","last_fullsized":"2024-09-21T06:28:06.274635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:01.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274854+0000","last_change":"2024-09-21T06:28:05.271567+0000","last_active":"2024-09-21T06:28:06.274854+0000","last_peered":"2024-09-21T06:28:06.274854+0000","last_clean":"2024-09-21T06:28:06.274854+0000","last_became_active":"2024-09-21T06:28:05.271160+0000","last_became_peered":"2024-09-21T06:28:05.271160+0000","last_unstale":"2024-09-21T06:28:06.274854+0000","last_undegraded":"2024-09-21T06:28:06.274854+0000","last_fullsized":"2024-09-21T06:28:06.274854+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:02.495413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918355+0000","last_change":"2024-09-21T06:28:00.215612+0000","last_active":"2024-09-21T06:28:06.918355+0000","last_peered":"2024-09-21T06:28:06.918355+0000","last_clean":"2024-09-21T06:28:06.918355+0000","last_became_active":"2024-09-21T06:28:00.214929+0000","last_became_peered":"2024-09-21T06:28:00.214929+0000","last_unstale":"2024-09-21T06:28:06.918355+0000","last_undegraded":"2024-09-21T06:28:06.918355+0000","last_fullsized":"2024-09-21T06:28:06.918355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:41.140041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918598+0000","last_change":"2024-09-21T06:28:01.234866+0000","last_active":"2024-09-21T06:28:06.918598+0000","last_peered":"2024-09-21T06:28:06.918598+0000","last_clean":"2024-09-21T06:28:06.918598+0000","last_became_active":"2024-09-21T06:28:01.234568+0000","last_became_peered":"2024-09-21T06:28:01.234568+0000","last_unstale":"2024-09-21T06:28:06.918598+0000","last_undegraded":"2024-09-21T06:28:06.918598+0000","last_fullsized":"2024-09-21T06:28:06.918598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:00.807517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276818+0000","last_change":"2024-09-21T06:28:05.287452+0000","last_active":"2024-09-21T06:28:06.276818+0000","last_peered":"2024-09-21T06:28:06.276818+0000","last_clean":"2024-09-21T06:28:06.276818+0000","last_became_active":"2024-09-21T06:28:05.286963+0000","last_became_peered":"2024-09-21T06:28:05.286963+0000","last_unstale":"2024-09-21T06:28:06.276818+0000","last_undegraded":"2024-09-21T06:28:06.276818+0000","last_fullsized":"2024-09-21T06:28:06.276818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:03.321463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271084+0000","last_change":"2024-09-21T06:28:01.232705+0000","last_active":"2024-09-21T06:28:06.271084+0000","last_peered":"2024-09-21T06:28:06.271084+0000","last_clean":"2024-09-21T06:28:06.271084+0000","last_became_active":"2024-09-21T06:28:01.232305+0000","last_became_peered":"2024-09-21T06:28:01.232305+0000","last_unstale":"2024-09-21T06:28:06.271084+0000","last_undegraded":"2024-09-21T06:28:06.271084+0000","last_fullsized":"2024-09-21T06:28:06.271084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275922+0000","last_change":"2024-09-21T06:28:01.235529+0000","last_active":"2024-09-21T06:28:06.275922+0000","last_peered":"2024-09-21T06:28:06.275922+0000","last_clean":"2024-09-21T06:28:06.275922+0000","last_became_active":"2024-09-21T06:28:01.235170+0000","last_became_peered":"2024-09-21T06:28:01.235170+0000","last_unstale":"2024-09-21T06:28:06.275922+0000","last_undegraded":"2024-09-21T06:28:06.275922+0000","last_fullsized":"2024-09-21T06:28:06.275922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566964+0000","last_change":"2024-09-21T06:28:01.242450+0000","last_active":"2024-09-21T06:28:06.566964+0000","last_peered":"2024-09-21T06:28:06.566964+0000","last_clean":"2024-09-21T06:28:06.566964+0000","last_became_active":"2024-09-21T06:28:01.242078+0000","last_became_peered":"2024-09-21T06:28:01.242078+0000","last_unstale":"2024-09-21T06:28:06.566964+0000","last_undegraded":"2024-09-21T06:28:06.566964+0000","last_fullsized":"2024-09-21T06:28:06.566964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:58.450653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918263+0000","last_change":"2024-09-21T06:28:01.230631+0000","last_active":"2024-09-21T06:28:06.918263+0000","last_peered":"2024-09-21T06:28:06.918263+0000","last_clean":"2024-09-21T06:28:06.918263+0000","last_became_active":"2024-09-21T06:28:01.223844+0000","last_became_peered":"2024-09-21T06:28:01.223844+0000","last_unstale":"2024-09-21T06:28:06.918263+0000","last_undegraded":"2024-09-21T06:28:06.918263+0000","last_fullsized":"2024-09-21T06:28:06.918263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:28.649005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919766+0000","last_change":"2024-09-21T06:28:01.248984+0000","last_active":"2024-09-21T06:28:06.919766+0000","last_peered":"2024-09-21T06:28:06.919766+0000","last_clean":"2024-09-21T06:28:06.919766+0000","last_became_active":"2024-09-21T06:28:01.248639+0000","last_became_peered":"2024-09-21T06:28:01.248639+0000","last_unstale":"2024-09-21T06:28:06.919766+0000","last_undegraded":"2024-09-21T06:28:06.919766+0000","last_fullsized":"2024-09-21T06:28:06.919766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.659556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919689+0000","last_change":"2024-09-21T06:28:01.226847+0000","last_active":"2024-09-21T06:28:06.919689+0000","last_peered":"2024-09-21T06:28:06.919689+0000","last_clean":"2024-09-21T06:28:06.919689+0000","last_became_active":"2024-09-21T06:28:01.226536+0000","last_became_peered":"2024-09-21T06:28:01.226536+0000","last_unstale":"2024-09-21T06:28:06.919689+0000","last_undegraded":"2024-09-21T06:28:06.919689+0000","last_fullsized":"2024-09-21T06:28:06.919689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271732+0000","last_change":"2024-09-21T06:28:01.223972+0000","last_active":"2024-09-21T06:28:06.271732+0000","last_peered":"2024-09-21T06:28:06.271732+0000","last_clean":"2024-09-21T06:28:06.271732+0000","last_became_active":"2024-09-21T06:28:01.223601+0000","last_became_peered":"2024-09-21T06:28:01.223601+0000","last_unstale":"2024-09-21T06:28:06.271732+0000","last_undegraded":"2024-09-21T06:28:06.271732+0000","last_fullsized":"2024-09-21T06:28:06.271732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918404+0000","last_change":"2024-09-21T06:28:01.233118+0000","last_active":"2024-09-21T06:28:06.918404+0000","last_peered":"2024-09-21T06:28:06.918404+0000","last_clean":"2024-09-21T06:28:06.918404+0000","last_became_active":"2024-09-21T06:28:01.232482+0000","last_became_peered":"2024-09-21T06:28:01.232482+0000","last_unstale":"2024-09-21T06:28:06.918404+0000","last_undegraded":"2024-09-21T06:28:06.918404+0000","last_fullsized":"2024-09-21T06:28:06.918404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:39.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566851+0000","last_change":"2024-09-21T06:28:01.243008+0000","last_active":"2024-09-21T06:28:06.566851+0000","last_peered":"2024-09-21T06:28:06.566851+0000","last_clean":"2024-09-21T06:28:06.566851+0000","last_became_active":"2024-09-21T06:28:01.242713+0000","last_became_peered":"2024-09-21T06:28:01.242713+0000","last_unstale":"2024-09-21T06:28:06.566851+0000","last_undegraded":"2024-09-21T06:28:06.566851+0000","last_fullsized":"2024-09-21T06:28:06.566851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:17.147886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917756+0000","last_change":"2024-09-21T06:28:01.216205+0000","last_active":"2024-09-21T06:28:06.917756+0000","last_peered":"2024-09-21T06:28:06.917756+0000","last_clean":"2024-09-21T06:28:06.917756+0000","last_became_active":"2024-09-21T06:28:01.215869+0000","last_became_peered":"2024-09-21T06:28:01.215869+0000","last_unstale":"2024-09-21T06:28:06.917756+0000","last_undegraded":"2024-09-21T06:28:06.917756+0000","last_fullsized":"2024-09-21T06:28:06.917756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.725175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.246071+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.245739+0000","last_became_peered":"2024-09-21T06:28:01.245739+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274517+0000","last_change":"2024-09-21T06:28:01.232435+0000","last_active":"2024-09-21T06:28:06.274517+0000","last_peered":"2024-09-21T06:28:06.274517+0000","last_clean":"2024-09-21T06:28:06.274517+0000","last_became_active":"2024-09-21T06:28:01.229739+0000","last_became_peered":"2024-09-21T06:28:01.229739+0000","last_unstale":"2024-09-21T06:28:06.274517+0000","last_undegraded":"2024-09-21T06:28:06.274517+0000","last_fullsized":"2024-09-21T06:28:06.274517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:59.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917974+0000","last_change":"2024-09-21T06:28:01.242252+0000","last_active":"2024-09-21T06:28:06.917974+0000","last_peered":"2024-09-21T06:28:06.917974+0000","last_clean":"2024-09-21T06:28:06.917974+0000","last_became_active":"2024-09-21T06:28:01.241926+0000","last_became_peered":"2024-09-21T06:28:01.241926+0000","last_unstale":"2024-09-21T06:28:06.917974+0000","last_undegraded":"2024-09-21T06:28:06.917974+0000","last_fullsized":"2024-09-21T06:28:06.917974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:31.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567677+0000","last_change":"2024-09-21T06:28:01.232426+0000","last_active":"2024-09-21T06:28:06.567677+0000","last_peered":"2024-09-21T06:28:06.567677+0000","last_clean":"2024-09-21T06:28:06.567677+0000","last_became_active":"2024-09-21T06:28:01.232067+0000","last_became_peered":"2024-09-21T06:28:01.232067+0000","last_unstale":"2024-09-21T06:28:06.567677+0000","last_undegraded":"2024-09-21T06:28:06.567677+0000","last_fullsized":"2024-09-21T06:28:06.567677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:28.274577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277142+0000","last_change":"2024-09-21T06:28:01.247978+0000","last_active":"2024-09-21T06:28:06.277142+0000","last_peered":"2024-09-21T06:28:06.277142+0000","last_clean":"2024-09-21T06:28:06.277142+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.277142+0000","last_undegraded":"2024-09-21T06:28:06.277142+0000","last_fullsized":"2024-09-21T06:28:06.277142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567819+0000","last_change":"2024-09-21T06:28:01.236566+0000","last_active":"2024-09-21T06:28:06.567819+0000","last_peered":"2024-09-21T06:28:06.567819+0000","last_clean":"2024-09-21T06:28:06.567819+0000","last_became_active":"2024-09-21T06:28:01.236209+0000","last_became_peered":"2024-09-21T06:28:01.236209+0000","last_unstale":"2024-09-21T06:28:06.567819+0000","last_undegraded":"2024-09-21T06:28:06.567819+0000","last_fullsized":"2024-09-21T06:28:06.567819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:55.410838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566732+0000","last_change":"2024-09-21T06:28:01.227863+0000","last_active":"2024-09-21T06:28:06.566732+0000","last_peered":"2024-09-21T06:28:06.566732+0000","last_clean":"2024-09-21T06:28:06.566732+0000","last_became_active":"2024-09-21T06:28:01.227553+0000","last_became_peered":"2024-09-21T06:28:01.227553+0000","last_unstale":"2024-09-21T06:28:06.566732+0000","last_undegraded":"2024-09-21T06:28:06.566732+0000","last_fullsized":"2024-09-21T06:28:06.566732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:51.034026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917669+0000","last_change":"2024-09-21T06:28:01.242950+0000","last_active":"2024-09-21T06:28:06.917669+0000","last_peered":"2024-09-21T06:28:06.917669+0000","last_clean":"2024-09-21T06:28:06.917669+0000","last_became_active":"2024-09-21T06:28:01.242715+0000","last_became_peered":"2024-09-21T06:28:01.242715+0000","last_unstale":"2024-09-21T06:28:06.917669+0000","last_undegraded":"2024-09-21T06:28:06.917669+0000","last_fullsized":"2024-09-21T06:28:06.917669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:12.827134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919061+0000","last_change":"2024-09-21T06:28:01.231009+0000","last_active":"2024-09-21T06:28:06.919061+0000","last_peered":"2024-09-21T06:28:06.919061+0000","last_clean":"2024-09-21T06:28:06.919061+0000","last_became_active":"2024-09-21T06:28:01.230706+0000","last_became_peered":"2024-09-21T06:28:01.230706+0000","last_unstale":"2024-09-21T06:28:06.919061+0000","last_undegraded":"2024-09-21T06:28:06.919061+0000","last_fullsized":"2024-09-21T06:28:06.919061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:38.274224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919349+0000","last_change":"2024-09-21T06:28:01.235786+0000","last_active":"2024-09-21T06:28:06.919349+0000","last_peered":"2024-09-21T06:28:06.919349+0000","last_clean":"2024-09-21T06:28:06.919349+0000","last_became_active":"2024-09-21T06:28:01.235551+0000","last_became_peered":"2024-09-21T06:28:01.235551+0000","last_unstale":"2024-09-21T06:28:06.919349+0000","last_undegraded":"2024-09-21T06:28:06.919349+0000","last_fullsized":"2024-09-21T06:28:06.919349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:00.237301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917934+0000","last_change":"2024-09-21T06:28:01.242543+0000","last_active":"2024-09-21T06:28:06.917934+0000","last_peered":"2024-09-21T06:28:06.917934+0000","last_clean":"2024-09-21T06:28:06.917934+0000","last_became_active":"2024-09-21T06:28:01.242225+0000","last_became_peered":"2024-09-21T06:28:01.242225+0000","last_unstale":"2024-09-21T06:28:06.917934+0000","last_undegraded":"2024-09-21T06:28:06.917934+0000","last_fullsized":"2024-09-21T06:28:06.917934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:39.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918078+0000","last_change":"2024-09-21T06:28:01.237214+0000","last_active":"2024-09-21T06:28:06.918078+0000","last_peered":"2024-09-21T06:28:06.918078+0000","last_clean":"2024-09-21T06:28:06.918078+0000","last_became_active":"2024-09-21T06:28:01.235938+0000","last_became_peered":"2024-09-21T06:28:01.235938+0000","last_unstale":"2024-09-21T06:28:06.918078+0000","last_undegraded":"2024-09-21T06:28:06.918078+0000","last_fullsized":"2024-09-21T06:28:06.918078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.564737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917663+0000","last_change":"2024-09-21T06:28:01.251436+0000","last_active":"2024-09-21T06:28:06.917663+0000","last_peered":"2024-09-21T06:28:06.917663+0000","last_clean":"2024-09-21T06:28:06.917663+0000","last_became_active":"2024-09-21T06:28:01.250562+0000","last_became_peered":"2024-09-21T06:28:01.250562+0000","last_unstale":"2024-09-21T06:28:06.917663+0000","last_undegraded":"2024-09-21T06:28:06.917663+0000","last_fullsized":"2024-09-21T06:28:06.917663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:58.692504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919694+0000","last_change":"2024-09-21T06:28:01.226763+0000","last_active":"2024-09-21T06:28:06.919694+0000","last_peered":"2024-09-21T06:28:06.919694+0000","last_clean":"2024-09-21T06:28:06.919694+0000","last_became_active":"2024-09-21T06:28:01.226438+0000","last_became_peered":"2024-09-21T06:28:01.226438+0000","last_unstale":"2024-09-21T06:28:06.919694+0000","last_undegraded":"2024-09-21T06:28:06.919694+0000","last_fullsized":"2024-09-21T06:28:06.919694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917570+0000","last_change":"2024-09-21T06:28:01.242596+0000","last_active":"2024-09-21T06:28:06.917570+0000","last_peered":"2024-09-21T06:28:06.917570+0000","last_clean":"2024-09-21T06:28:06.917570+0000","last_became_active":"2024-09-21T06:28:01.242205+0000","last_became_peered":"2024-09-21T06:28:01.242205+0000","last_unstale":"2024-09-21T06:28:06.917570+0000","last_undegraded":"2024-09-21T06:28:06.917570+0000","last_fullsized":"2024-09-21T06:28:06.917570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:44.145821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917382+0000","last_change":"2024-09-21T06:28:01.249034+0000","last_active":"2024-09-21T06:28:06.917382+0000","last_peered":"2024-09-21T06:28:06.917382+0000","last_clean":"2024-09-21T06:28:06.917382+0000","last_became_active":"2024-09-21T06:28:01.248727+0000","last_became_peered":"2024-09-21T06:28:01.248727+0000","last_unstale":"2024-09-21T06:28:06.917382+0000","last_undegraded":"2024-09-21T06:28:06.917382+0000","last_fullsized":"2024-09-21T06:28:06.917382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271467+0000","last_change":"2024-09-21T06:28:01.215764+0000","last_active":"2024-09-21T06:28:06.271467+0000","last_peered":"2024-09-21T06:28:06.271467+0000","last_clean":"2024-09-21T06:28:06.271467+0000","last_became_active":"2024-09-21T06:28:01.215412+0000","last_became_peered":"2024-09-21T06:28:01.215412+0000","last_unstale":"2024-09-21T06:28:06.271467+0000","last_undegraded":"2024-09-21T06:28:06.271467+0000","last_fullsized":"2024-09-21T06:28:06.271467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917795+0000","last_change":"2024-09-21T06:28:01.253480+0000","last_active":"2024-09-21T06:28:06.917795+0000","last_peered":"2024-09-21T06:28:06.917795+0000","last_clean":"2024-09-21T06:28:06.917795+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.917795+0000","last_undegraded":"2024-09-21T06:28:06.917795+0000","last_fullsized":"2024-09-21T06:28:06.917795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:51.213607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271436+0000","last_change":"2024-09-21T06:28:01.231505+0000","last_active":"2024-09-21T06:28:06.271436+0000","last_peered":"2024-09-21T06:28:06.271436+0000","last_clean":"2024-09-21T06:28:06.271436+0000","last_became_active":"2024-09-21T06:28:01.223703+0000","last_became_peered":"2024-09-21T06:28:01.223703+0000","last_unstale":"2024-09-21T06:28:06.271436+0000","last_undegraded":"2024-09-21T06:28:06.271436+0000","last_fullsized":"2024-09-21T06:28:06.271436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918618+0000","last_change":"2024-09-21T06:28:01.236472+0000","last_active":"2024-09-21T06:28:06.918618+0000","last_peered":"2024-09-21T06:28:06.918618+0000","last_clean":"2024-09-21T06:28:06.918618+0000","last_became_active":"2024-09-21T06:28:01.221503+0000","last_became_peered":"2024-09-21T06:28:01.221503+0000","last_unstale":"2024-09-21T06:28:06.918618+0000","last_undegraded":"2024-09-21T06:28:06.918618+0000","last_fullsized":"2024-09-21T06:28:06.918618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:35.021658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918538+0000","last_change":"2024-09-21T06:28:01.227458+0000","last_active":"2024-09-21T06:28:06.918538+0000","last_peered":"2024-09-21T06:28:06.918538+0000","last_clean":"2024-09-21T06:28:06.918538+0000","last_became_active":"2024-09-21T06:28:01.227145+0000","last_became_peered":"2024-09-21T06:28:01.227145+0000","last_unstale":"2024-09-21T06:28:06.918538+0000","last_undegraded":"2024-09-21T06:28:06.918538+0000","last_fullsized":"2024-09-21T06:28:06.918538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:24.909945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918495+0000","last_change":"2024-09-21T06:28:01.233932+0000","last_active":"2024-09-21T06:28:06.918495+0000","last_peered":"2024-09-21T06:28:06.918495+0000","last_clean":"2024-09-21T06:28:06.918495+0000","last_became_active":"2024-09-21T06:28:01.233617+0000","last_became_peered":"2024-09-21T06:28:01.233617+0000","last_unstale":"2024-09-21T06:28:06.918495+0000","last_undegraded":"2024-09-21T06:28:06.918495+0000","last_fullsized":"2024-09-21T06:28:06.918495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:27.585260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:28:26.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.222+0000 7f7a36400640 1 -- 172.21.15.37:0/1457059611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a18003a00 msgr2=0x7f7a18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.222+0000 7f7a36400640 1 --2- 172.21.15.37:0/1457059611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a18003a00 0x7f7a18003e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c4a0 tx=0x7f7a18005830 comp rx=0 tx=0).stop 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.222+0000 7f7a35a00640 1 -- 172.21.15.37:0/1457059611 reap_dead start 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.223+0000 7f7a36400640 1 -- 172.21.15.37:0/1457059611 shutdown_connections 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.223+0000 7f7a36400640 1 -- 172.21.15.37:0/1457059611 >> 172.21.15.37:0/1457059611 conn(0x7f7a3006f940 msgr2=0x7f7a300b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.223+0000 7f7a36400640 1 -- 172.21.15.37:0/1457059611 shutdown_connections 2024-09-21T06:28:27.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.223+0000 7f7a36400640 1 -- 172.21.15.37:0/1457059611 wait complete. 2024-09-21T06:28:27.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.225+0000 7f7a36400640 1 Processor -- start 2024-09-21T06:28:27.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.225+0000 7f7a36400640 1 -- start start 2024-09-21T06:28:27.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.226+0000 7f7a36400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a18003a00 0x7f7a30140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.226+0000 7f7a36400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30142ea0 0x7f7a301432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.226+0000 7f7a36400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a36400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a300ce0f0 con 0x7f7a30141f50 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a36400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a300cdf70 con 0x7f7a30142ea0 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a36400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3006e300 con 0x7f7a18003a00 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a18003a00 0x7f7a30140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40430/0 (socket says 172.21.15.37:40430) 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 -- 172.21.15.37:0/3302373578 learned_addr learned my addr 172.21.15.37:0/3302373578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a2fe00640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30142ea0 0x7f7a301432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 -- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a18003a00 msgr2=0x7f7a30140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a18003a00 0x7f7a30140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 -- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30142ea0 msgr2=0x7f7a301432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30142ea0 0x7f7a301432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.227+0000 7f7a35a00640 1 -- 172.21.15.37:0/3302373578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a301437f0 con 0x7f7a30141f50 2024-09-21T06:28:27.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a35a00640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a300b3000 tx=0x7f7a2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:27.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28003b80 con 0x7f7a30141f50 2024-09-21T06:28:27.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a28003d20 con 0x7f7a30141f50 2024-09-21T06:28:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28004d80 con 0x7f7a30141f50 2024-09-21T06:28:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a30140eb0 con 0x7f7a30141f50 2024-09-21T06:28:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.228+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a30141470 con 0x7f7a30141f50 2024-09-21T06:28:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.229+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3015c370 con 0x7f7a30141f50 2024-09-21T06:28:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.230+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a2801d020 con 0x7f7a30141f50 2024-09-21T06:28:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.231+0000 7f7a2cc00640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 0x7f7a1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.231+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7a2809ac10 con 0x7f7a30141f50 2024-09-21T06:28:27.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.231+0000 7f7a35000640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 0x7f7a1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.232+0000 7f7a35000640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 0x7f7a1007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a24033000 tx=0x7f7a30073cf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.236+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a301437f0 con 0x7f7a30141f50 2024-09-21T06:28:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.430+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a300d0290 con 0x7f7a10078b90 2024-09-21T06:28:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.438+0000 7f7a2cc00640 1 -- 172.21.15.37:0/3302373578 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f7a300d0290 con 0x7f7a10078b90 2024-09-21T06:28:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:27.446 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:28:27.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 msgr2=0x7f7a1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 0x7f7a1007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a24033000 tx=0x7f7a30073cf0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 msgr2=0x7f7a30142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a300b3000 tx=0x7f7a2800cc40 comp rx=0 tx=0).stop 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 shutdown_connections 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a10078b90 0x7f7a1007b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a30141f50 0x7f7a30142360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30142ea0 0x7f7a301432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 --2- 172.21.15.37:0/3302373578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a18003a00 0x7f7a30140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 >> 172.21.15.37:0/3302373578 conn(0x7f7a3006f940 msgr2=0x7f7a300d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:27.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 shutdown_connections 2024-09-21T06:28:27.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.443+0000 7f7a36400640 1 -- 172.21.15.37:0/3302373578 wait complete. 2024-09-21T06:28:27.466 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-09-21T06:28:27.094015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219672,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755048,"statfs":{"total":773094113280,"available":772869169152,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003917"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275298+0000","last_change":"2024-09-21T06:28:00.220046+0000","last_active":"2024-09-21T06:28:06.275298+0000","last_peered":"2024-09-21T06:28:06.275298+0000","last_clean":"2024-09-21T06:28:06.275298+0000","last_became_active":"2024-09-21T06:28:00.219127+0000","last_became_peered":"2024-09-21T06:28:00.219127+0000","last_unstale":"2024-09-21T06:28:06.275298+0000","last_undegraded":"2024-09-21T06:28:06.275298+0000","last_fullsized":"2024-09-21T06:28:06.275298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:49.256177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918731+0000","last_change":"2024-09-21T06:28:01.226620+0000","last_active":"2024-09-21T06:28:06.918731+0000","last_peered":"2024-09-21T06:28:06.918731+0000","last_clean":"2024-09-21T06:28:06.918731+0000","last_became_active":"2024-09-21T06:28:01.224002+0000","last_became_peered":"2024-09-21T06:28:01.224002+0000","last_unstale":"2024-09-21T06:28:06.918731+0000","last_undegraded":"2024-09-21T06:28:06.918731+0000","last_fullsized":"2024-09-21T06:28:06.918731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.430586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276059+0000","last_change":"2024-09-21T06:28:05.289106+0000","last_active":"2024-09-21T06:28:06.276059+0000","last_peered":"2024-09-21T06:28:06.276059+0000","last_clean":"2024-09-21T06:28:06.276059+0000","last_became_active":"2024-09-21T06:28:05.288635+0000","last_became_peered":"2024-09-21T06:28:05.288635+0000","last_unstale":"2024-09-21T06:28:06.276059+0000","last_undegraded":"2024-09-21T06:28:06.276059+0000","last_fullsized":"2024-09-21T06:28:06.276059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:30.614325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918864+0000","last_change":"2024-09-21T06:28:00.216173+0000","last_active":"2024-09-21T06:28:06.918864+0000","last_peered":"2024-09-21T06:28:06.918864+0000","last_clean":"2024-09-21T06:28:06.918864+0000","last_became_active":"2024-09-21T06:28:00.215794+0000","last_became_peered":"2024-09-21T06:28:00.215794+0000","last_unstale":"2024-09-21T06:28:06.918864+0000","last_undegraded":"2024-09-21T06:28:06.918864+0000","last_fullsized":"2024-09-21T06:28:06.918864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:18.915364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918573+0000","last_change":"2024-09-21T06:28:01.238244+0000","last_active":"2024-09-21T06:28:06.918573+0000","last_peered":"2024-09-21T06:28:06.918573+0000","last_clean":"2024-09-21T06:28:06.918573+0000","last_became_active":"2024-09-21T06:28:01.237538+0000","last_became_peered":"2024-09-21T06:28:01.237538+0000","last_unstale":"2024-09-21T06:28:06.918573+0000","last_undegraded":"2024-09-21T06:28:06.918573+0000","last_fullsized":"2024-09-21T06:28:06.918573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:11.476210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271237+0000","last_change":"2024-09-21T06:28:05.267181+0000","last_active":"2024-09-21T06:28:06.271237+0000","last_peered":"2024-09-21T06:28:06.271237+0000","last_clean":"2024-09-21T06:28:06.271237+0000","last_became_active":"2024-09-21T06:28:05.266697+0000","last_became_peered":"2024-09-21T06:28:05.266697+0000","last_unstale":"2024-09-21T06:28:06.271237+0000","last_undegraded":"2024-09-21T06:28:06.271237+0000","last_fullsized":"2024-09-21T06:28:06.271237+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.144842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918246+0000","last_change":"2024-09-21T06:28:00.230877+0000","last_active":"2024-09-21T06:28:06.918246+0000","last_peered":"2024-09-21T06:28:06.918246+0000","last_clean":"2024-09-21T06:28:06.918246+0000","last_became_active":"2024-09-21T06:28:00.230578+0000","last_became_peered":"2024-09-21T06:28:00.230578+0000","last_unstale":"2024-09-21T06:28:06.918246+0000","last_undegraded":"2024-09-21T06:28:06.918246+0000","last_fullsized":"2024-09-21T06:28:06.918246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919137+0000","last_change":"2024-09-21T06:28:01.255720+0000","last_active":"2024-09-21T06:28:06.919137+0000","last_peered":"2024-09-21T06:28:06.919137+0000","last_clean":"2024-09-21T06:28:06.919137+0000","last_became_active":"2024-09-21T06:28:01.254968+0000","last_became_peered":"2024-09-21T06:28:01.254968+0000","last_unstale":"2024-09-21T06:28:06.919137+0000","last_undegraded":"2024-09-21T06:28:06.919137+0000","last_fullsized":"2024-09-21T06:28:06.919137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271315+0000","last_change":"2024-09-21T06:28:05.277963+0000","last_active":"2024-09-21T06:28:06.271315+0000","last_peered":"2024-09-21T06:28:06.271315+0000","last_clean":"2024-09-21T06:28:06.271315+0000","last_became_active":"2024-09-21T06:28:05.277006+0000","last_became_peered":"2024-09-21T06:28:05.277006+0000","last_unstale":"2024-09-21T06:28:06.271315+0000","last_undegraded":"2024-09-21T06:28:06.271315+0000","last_fullsized":"2024-09-21T06:28:06.271315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:35.624735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918079+0000","last_change":"2024-09-21T06:28:00.230976+0000","last_active":"2024-09-21T06:28:06.918079+0000","last_peered":"2024-09-21T06:28:06.918079+0000","last_clean":"2024-09-21T06:28:06.918079+0000","last_became_active":"2024-09-21T06:28:00.230660+0000","last_became_peered":"2024-09-21T06:28:00.230660+0000","last_unstale":"2024-09-21T06:28:06.918079+0000","last_undegraded":"2024-09-21T06:28:06.918079+0000","last_fullsized":"2024-09-21T06:28:06.918079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:59.230778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567463+0000","last_change":"2024-09-21T06:28:01.234299+0000","last_active":"2024-09-21T06:28:06.567463+0000","last_peered":"2024-09-21T06:28:06.567463+0000","last_clean":"2024-09-21T06:28:06.567463+0000","last_became_active":"2024-09-21T06:28:01.233998+0000","last_became_peered":"2024-09-21T06:28:01.233998+0000","last_unstale":"2024-09-21T06:28:06.567463+0000","last_undegraded":"2024-09-21T06:28:06.567463+0000","last_fullsized":"2024-09-21T06:28:06.567463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:08.335780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271460+0000","last_change":"2024-09-21T06:28:05.271851+0000","last_active":"2024-09-21T06:28:06.271460+0000","last_peered":"2024-09-21T06:28:06.271460+0000","last_clean":"2024-09-21T06:28:06.271460+0000","last_became_active":"2024-09-21T06:28:05.265434+0000","last_became_peered":"2024-09-21T06:28:05.265434+0000","last_unstale":"2024-09-21T06:28:06.271460+0000","last_undegraded":"2024-09-21T06:28:06.271460+0000","last_fullsized":"2024-09-21T06:28:06.271460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:39.030315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566989+0000","last_change":"2024-09-21T06:28:01.226065+0000","last_active":"2024-09-21T06:28:06.566989+0000","last_peered":"2024-09-21T06:28:06.566989+0000","last_clean":"2024-09-21T06:28:06.566989+0000","last_became_active":"2024-09-21T06:28:01.225761+0000","last_became_peered":"2024-09-21T06:28:01.225761+0000","last_unstale":"2024-09-21T06:28:06.566989+0000","last_undegraded":"2024-09-21T06:28:06.566989+0000","last_fullsized":"2024-09-21T06:28:06.566989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:02.936599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567107+0000","last_change":"2024-09-21T06:28:00.220356+0000","last_active":"2024-09-21T06:28:06.567107+0000","last_peered":"2024-09-21T06:28:06.567107+0000","last_clean":"2024-09-21T06:28:06.567107+0000","last_became_active":"2024-09-21T06:28:00.219737+0000","last_became_peered":"2024-09-21T06:28:00.219737+0000","last_unstale":"2024-09-21T06:28:06.567107+0000","last_undegraded":"2024-09-21T06:28:06.567107+0000","last_fullsized":"2024-09-21T06:28:06.567107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:58.072689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567253+0000","last_change":"2024-09-21T06:28:05.264666+0000","last_active":"2024-09-21T06:28:06.567253+0000","last_peered":"2024-09-21T06:28:06.567253+0000","last_clean":"2024-09-21T06:28:06.567253+0000","last_became_active":"2024-09-21T06:28:05.264145+0000","last_became_peered":"2024-09-21T06:28:05.264145+0000","last_unstale":"2024-09-21T06:28:06.567253+0000","last_undegraded":"2024-09-21T06:28:06.567253+0000","last_fullsized":"2024-09-21T06:28:06.567253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.384335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276438+0000","last_change":"2024-09-21T06:28:01.238641+0000","last_active":"2024-09-21T06:28:06.276438+0000","last_peered":"2024-09-21T06:28:06.276438+0000","last_clean":"2024-09-21T06:28:06.276438+0000","last_became_active":"2024-09-21T06:28:01.237104+0000","last_became_peered":"2024-09-21T06:28:01.237104+0000","last_unstale":"2024-09-21T06:28:06.276438+0000","last_undegraded":"2024-09-21T06:28:06.276438+0000","last_fullsized":"2024-09-21T06:28:06.276438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919263+0000","last_change":"2024-09-21T06:28:00.222233+0000","last_active":"2024-09-21T06:28:06.919263+0000","last_peered":"2024-09-21T06:28:06.919263+0000","last_clean":"2024-09-21T06:28:06.919263+0000","last_became_active":"2024-09-21T06:28:00.221686+0000","last_became_peered":"2024-09-21T06:28:00.221686+0000","last_unstale":"2024-09-21T06:28:06.919263+0000","last_undegraded":"2024-09-21T06:28:06.919263+0000","last_fullsized":"2024-09-21T06:28:06.919263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:40.731260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917906+0000","last_change":"2024-09-21T06:28:05.275795+0000","last_active":"2024-09-21T06:28:06.917906+0000","last_peered":"2024-09-21T06:28:06.917906+0000","last_clean":"2024-09-21T06:28:06.917906+0000","last_became_active":"2024-09-21T06:28:05.275276+0000","last_became_peered":"2024-09-21T06:28:05.275276+0000","last_unstale":"2024-09-21T06:28:06.917906+0000","last_undegraded":"2024-09-21T06:28:06.917906+0000","last_fullsized":"2024-09-21T06:28:06.917906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:55.953841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273709+0000","last_change":"2024-09-21T06:28:01.231657+0000","last_active":"2024-09-21T06:28:06.273709+0000","last_peered":"2024-09-21T06:28:06.273709+0000","last_clean":"2024-09-21T06:28:06.273709+0000","last_became_active":"2024-09-21T06:28:01.223226+0000","last_became_peered":"2024-09-21T06:28:01.223226+0000","last_unstale":"2024-09-21T06:28:06.273709+0000","last_undegraded":"2024-09-21T06:28:06.273709+0000","last_fullsized":"2024-09-21T06:28:06.273709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274062+0000","last_change":"2024-09-21T06:28:00.227556+0000","last_active":"2024-09-21T06:28:06.274062+0000","last_peered":"2024-09-21T06:28:06.274062+0000","last_clean":"2024-09-21T06:28:06.274062+0000","last_became_active":"2024-09-21T06:28:00.227125+0000","last_became_peered":"2024-09-21T06:28:00.227125+0000","last_unstale":"2024-09-21T06:28:06.274062+0000","last_undegraded":"2024-09-21T06:28:06.274062+0000","last_fullsized":"2024-09-21T06:28:06.274062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:03.936656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277113+0000","last_change":"2024-09-21T06:28:05.283986+0000","last_active":"2024-09-21T06:28:06.277113+0000","last_peered":"2024-09-21T06:28:06.277113+0000","last_clean":"2024-09-21T06:28:06.277113+0000","last_became_active":"2024-09-21T06:28:05.283382+0000","last_became_peered":"2024-09-21T06:28:05.283382+0000","last_unstale":"2024-09-21T06:28:06.277113+0000","last_undegraded":"2024-09-21T06:28:06.277113+0000","last_fullsized":"2024-09-21T06:28:06.277113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.496971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566814+0000","last_change":"2024-09-21T06:28:01.223370+0000","last_active":"2024-09-21T06:28:06.566814+0000","last_peered":"2024-09-21T06:28:06.566814+0000","last_clean":"2024-09-21T06:28:06.566814+0000","last_became_active":"2024-09-21T06:28:01.223018+0000","last_became_peered":"2024-09-21T06:28:01.223018+0000","last_unstale":"2024-09-21T06:28:06.566814+0000","last_undegraded":"2024-09-21T06:28:06.566814+0000","last_fullsized":"2024-09-21T06:28:06.566814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:57.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917258+0000","last_change":"2024-09-21T06:28:00.225692+0000","last_active":"2024-09-21T06:28:06.917258+0000","last_peered":"2024-09-21T06:28:06.917258+0000","last_clean":"2024-09-21T06:28:06.917258+0000","last_became_active":"2024-09-21T06:28:00.224466+0000","last_became_peered":"2024-09-21T06:28:00.224466+0000","last_unstale":"2024-09-21T06:28:06.917258+0000","last_undegraded":"2024-09-21T06:28:06.917258+0000","last_fullsized":"2024-09-21T06:28:06.917258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.485812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271902+0000","last_change":"2024-09-21T06:28:05.282444+0000","last_active":"2024-09-21T06:28:06.271902+0000","last_peered":"2024-09-21T06:28:06.271902+0000","last_clean":"2024-09-21T06:28:06.271902+0000","last_became_active":"2024-09-21T06:28:05.282050+0000","last_became_peered":"2024-09-21T06:28:05.282050+0000","last_unstale":"2024-09-21T06:28:06.271902+0000","last_undegraded":"2024-09-21T06:28:06.271902+0000","last_fullsized":"2024-09-21T06:28:06.271902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274767+0000","last_change":"2024-09-21T06:28:01.225464+0000","last_active":"2024-09-21T06:28:06.274767+0000","last_peered":"2024-09-21T06:28:06.274767+0000","last_clean":"2024-09-21T06:28:06.274767+0000","last_became_active":"2024-09-21T06:28:01.223116+0000","last_became_peered":"2024-09-21T06:28:01.223116+0000","last_unstale":"2024-09-21T06:28:06.274767+0000","last_undegraded":"2024-09-21T06:28:06.274767+0000","last_fullsized":"2024-09-21T06:28:06.274767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567905+0000","last_change":"2024-09-21T06:28:00.230433+0000","last_active":"2024-09-21T06:28:06.567905+0000","last_peered":"2024-09-21T06:28:06.567905+0000","last_clean":"2024-09-21T06:28:06.567905+0000","last_became_active":"2024-09-21T06:28:00.228324+0000","last_became_peered":"2024-09-21T06:28:00.228324+0000","last_unstale":"2024-09-21T06:28:06.567905+0000","last_undegraded":"2024-09-21T06:28:06.567905+0000","last_fullsized":"2024-09-21T06:28:06.567905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:40.653033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271246+0000","last_change":"2024-09-21T06:28:05.280019+0000","last_active":"2024-09-21T06:28:06.271246+0000","last_peered":"2024-09-21T06:28:06.271246+0000","last_clean":"2024-09-21T06:28:06.271246+0000","last_became_active":"2024-09-21T06:28:05.279566+0000","last_became_peered":"2024-09-21T06:28:05.279566+0000","last_unstale":"2024-09-21T06:28:06.271246+0000","last_undegraded":"2024-09-21T06:28:06.271246+0000","last_fullsized":"2024-09-21T06:28:06.271246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:46.736598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275157+0000","last_change":"2024-09-21T06:28:01.227702+0000","last_active":"2024-09-21T06:28:06.275157+0000","last_peered":"2024-09-21T06:28:06.275157+0000","last_clean":"2024-09-21T06:28:06.275157+0000","last_became_active":"2024-09-21T06:28:01.227351+0000","last_became_peered":"2024-09-21T06:28:01.227351+0000","last_unstale":"2024-09-21T06:28:06.275157+0000","last_undegraded":"2024-09-21T06:28:06.275157+0000","last_fullsized":"2024-09-21T06:28:06.275157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275087+0000","last_change":"2024-09-21T06:28:00.227291+0000","last_active":"2024-09-21T06:28:06.275087+0000","last_peered":"2024-09-21T06:28:06.275087+0000","last_clean":"2024-09-21T06:28:06.275087+0000","last_became_active":"2024-09-21T06:28:00.226969+0000","last_became_peered":"2024-09-21T06:28:00.226969+0000","last_unstale":"2024-09-21T06:28:06.275087+0000","last_undegraded":"2024-09-21T06:28:06.275087+0000","last_fullsized":"2024-09-21T06:28:06.275087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:31.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273969+0000","last_change":"2024-09-21T06:28:05.277580+0000","last_active":"2024-09-21T06:28:06.273969+0000","last_peered":"2024-09-21T06:28:06.273969+0000","last_clean":"2024-09-21T06:28:06.273969+0000","last_became_active":"2024-09-21T06:28:05.277134+0000","last_became_peered":"2024-09-21T06:28:05.277134+0000","last_unstale":"2024-09-21T06:28:06.273969+0000","last_undegraded":"2024-09-21T06:28:06.273969+0000","last_fullsized":"2024-09-21T06:28:06.273969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:38.152610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919851+0000","last_change":"2024-09-21T06:28:01.235542+0000","last_active":"2024-09-21T06:28:06.919851+0000","last_peered":"2024-09-21T06:28:06.919851+0000","last_clean":"2024-09-21T06:28:06.919851+0000","last_became_active":"2024-09-21T06:28:01.235235+0000","last_became_peered":"2024-09-21T06:28:01.235235+0000","last_unstale":"2024-09-21T06:28:06.919851+0000","last_undegraded":"2024-09-21T06:28:06.919851+0000","last_fullsized":"2024-09-21T06:28:06.919851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:26.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567780+0000","last_change":"2024-09-21T06:28:00.219554+0000","last_active":"2024-09-21T06:28:06.567780+0000","last_peered":"2024-09-21T06:28:06.567780+0000","last_clean":"2024-09-21T06:28:06.567780+0000","last_became_active":"2024-09-21T06:28:00.219180+0000","last_became_peered":"2024-09-21T06:28:00.219180+0000","last_unstale":"2024-09-21T06:28:06.567780+0000","last_undegraded":"2024-09-21T06:28:06.567780+0000","last_fullsized":"2024-09-21T06:28:06.567780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.267700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919430+0000","last_change":"2024-09-21T06:28:05.268962+0000","last_active":"2024-09-21T06:28:06.919430+0000","last_peered":"2024-09-21T06:28:06.919430+0000","last_clean":"2024-09-21T06:28:06.919430+0000","last_became_active":"2024-09-21T06:28:05.268415+0000","last_became_peered":"2024-09-21T06:28:05.268415+0000","last_unstale":"2024-09-21T06:28:06.919430+0000","last_undegraded":"2024-09-21T06:28:06.919430+0000","last_fullsized":"2024-09-21T06:28:06.919430+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.107198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271569+0000","last_change":"2024-09-21T06:28:01.230048+0000","last_active":"2024-09-21T06:28:06.271569+0000","last_peered":"2024-09-21T06:28:06.271569+0000","last_clean":"2024-09-21T06:28:06.271569+0000","last_became_active":"2024-09-21T06:28:01.229747+0000","last_became_peered":"2024-09-21T06:28:01.229747+0000","last_unstale":"2024-09-21T06:28:06.271569+0000","last_undegraded":"2024-09-21T06:28:06.271569+0000","last_fullsized":"2024-09-21T06:28:06.271569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:35.860406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918630+0000","last_change":"2024-09-21T06:28:00.221155+0000","last_active":"2024-09-21T06:28:06.918630+0000","last_peered":"2024-09-21T06:28:06.918630+0000","last_clean":"2024-09-21T06:28:06.918630+0000","last_became_active":"2024-09-21T06:28:00.220728+0000","last_became_peered":"2024-09-21T06:28:00.220728+0000","last_unstale":"2024-09-21T06:28:06.918630+0000","last_undegraded":"2024-09-21T06:28:06.918630+0000","last_fullsized":"2024-09-21T06:28:06.918630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.642076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919099+0000","last_change":"2024-09-21T06:28:05.276248+0000","last_active":"2024-09-21T06:28:06.919099+0000","last_peered":"2024-09-21T06:28:06.919099+0000","last_clean":"2024-09-21T06:28:06.919099+0000","last_became_active":"2024-09-21T06:28:05.271420+0000","last_became_peered":"2024-09-21T06:28:05.271420+0000","last_unstale":"2024-09-21T06:28:06.919099+0000","last_undegraded":"2024-09-21T06:28:06.919099+0000","last_fullsized":"2024-09-21T06:28:06.919099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:03.357365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917672+0000","last_change":"2024-09-21T06:28:01.240349+0000","last_active":"2024-09-21T06:28:06.917672+0000","last_peered":"2024-09-21T06:28:06.917672+0000","last_clean":"2024-09-21T06:28:06.917672+0000","last_became_active":"2024-09-21T06:28:01.239325+0000","last_became_peered":"2024-09-21T06:28:01.239325+0000","last_unstale":"2024-09-21T06:28:06.917672+0000","last_undegraded":"2024-09-21T06:28:06.917672+0000","last_fullsized":"2024-09-21T06:28:06.917672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:42.691839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567402+0000","last_change":"2024-09-21T06:28:00.225175+0000","last_active":"2024-09-21T06:28:06.567402+0000","last_peered":"2024-09-21T06:28:06.567402+0000","last_clean":"2024-09-21T06:28:06.567402+0000","last_became_active":"2024-09-21T06:28:00.224858+0000","last_became_peered":"2024-09-21T06:28:00.224858+0000","last_unstale":"2024-09-21T06:28:06.567402+0000","last_undegraded":"2024-09-21T06:28:06.567402+0000","last_fullsized":"2024-09-21T06:28:06.567402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:41.165675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919693+0000","last_change":"2024-09-21T06:28:05.283982+0000","last_active":"2024-09-21T06:28:06.919693+0000","last_peered":"2024-09-21T06:28:06.919693+0000","last_clean":"2024-09-21T06:28:06.919693+0000","last_became_active":"2024-09-21T06:28:05.283527+0000","last_became_peered":"2024-09-21T06:28:05.283527+0000","last_unstale":"2024-09-21T06:28:06.919693+0000","last_undegraded":"2024-09-21T06:28:06.919693+0000","last_fullsized":"2024-09-21T06:28:06.919693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:17.117103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917636+0000","last_change":"2024-09-21T06:28:01.224161+0000","last_active":"2024-09-21T06:28:06.917636+0000","last_peered":"2024-09-21T06:28:06.917636+0000","last_clean":"2024-09-21T06:28:06.917636+0000","last_became_active":"2024-09-21T06:28:01.223787+0000","last_became_peered":"2024-09-21T06:28:01.223787+0000","last_unstale":"2024-09-21T06:28:06.917636+0000","last_undegraded":"2024-09-21T06:28:06.917636+0000","last_fullsized":"2024-09-21T06:28:06.917636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:03.351441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276110+0000","last_change":"2024-09-21T06:28:00.218460+0000","last_active":"2024-09-21T06:28:06.276110+0000","last_peered":"2024-09-21T06:28:06.276110+0000","last_clean":"2024-09-21T06:28:06.276110+0000","last_became_active":"2024-09-21T06:28:00.218046+0000","last_became_peered":"2024-09-21T06:28:00.218046+0000","last_unstale":"2024-09-21T06:28:06.276110+0000","last_undegraded":"2024-09-21T06:28:06.276110+0000","last_fullsized":"2024-09-21T06:28:06.276110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919620+0000","last_change":"2024-09-21T06:28:05.274088+0000","last_active":"2024-09-21T06:28:06.919620+0000","last_peered":"2024-09-21T06:28:06.919620+0000","last_clean":"2024-09-21T06:28:06.919620+0000","last_became_active":"2024-09-21T06:28:05.273184+0000","last_became_peered":"2024-09-21T06:28:05.273184+0000","last_unstale":"2024-09-21T06:28:06.919620+0000","last_undegraded":"2024-09-21T06:28:06.919620+0000","last_fullsized":"2024-09-21T06:28:06.919620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:17.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918180+0000","last_change":"2024-09-21T06:28:01.230531+0000","last_active":"2024-09-21T06:28:06.918180+0000","last_peered":"2024-09-21T06:28:06.918180+0000","last_clean":"2024-09-21T06:28:06.918180+0000","last_became_active":"2024-09-21T06:28:01.230236+0000","last_became_peered":"2024-09-21T06:28:01.230236+0000","last_unstale":"2024-09-21T06:28:06.918180+0000","last_undegraded":"2024-09-21T06:28:06.918180+0000","last_fullsized":"2024-09-21T06:28:06.918180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.172198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567087+0000","last_change":"2024-09-21T06:28:00.233247+0000","last_active":"2024-09-21T06:28:06.567087+0000","last_peered":"2024-09-21T06:28:06.567087+0000","last_clean":"2024-09-21T06:28:06.567087+0000","last_became_active":"2024-09-21T06:28:00.232905+0000","last_became_peered":"2024-09-21T06:28:00.232905+0000","last_unstale":"2024-09-21T06:28:06.567087+0000","last_undegraded":"2024-09-21T06:28:06.567087+0000","last_fullsized":"2024-09-21T06:28:06.567087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918161+0000","last_change":"2024-09-21T06:28:05.266249+0000","last_active":"2024-09-21T06:28:06.918161+0000","last_peered":"2024-09-21T06:28:06.918161+0000","last_clean":"2024-09-21T06:28:06.918161+0000","last_became_active":"2024-09-21T06:28:05.265779+0000","last_became_peered":"2024-09-21T06:28:05.265779+0000","last_unstale":"2024-09-21T06:28:06.918161+0000","last_undegraded":"2024-09-21T06:28:06.918161+0000","last_fullsized":"2024-09-21T06:28:06.918161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:58.459609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274361+0000","last_change":"2024-09-21T06:28:01.236228+0000","last_active":"2024-09-21T06:28:06.274361+0000","last_peered":"2024-09-21T06:28:06.274361+0000","last_clean":"2024-09-21T06:28:06.274361+0000","last_became_active":"2024-09-21T06:28:01.229874+0000","last_became_peered":"2024-09-21T06:28:01.229874+0000","last_unstale":"2024-09-21T06:28:06.274361+0000","last_undegraded":"2024-09-21T06:28:06.274361+0000","last_fullsized":"2024-09-21T06:28:06.274361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:08.119892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567235+0000","last_change":"2024-09-21T06:28:00.219928+0000","last_active":"2024-09-21T06:28:06.567235+0000","last_peered":"2024-09-21T06:28:06.567235+0000","last_clean":"2024-09-21T06:28:06.567235+0000","last_became_active":"2024-09-21T06:28:00.219561+0000","last_became_peered":"2024-09-21T06:28:00.219561+0000","last_unstale":"2024-09-21T06:28:06.567235+0000","last_undegraded":"2024-09-21T06:28:06.567235+0000","last_fullsized":"2024-09-21T06:28:06.567235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:22.801415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918345+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:28:06.918345+0000","last_peered":"2024-09-21T06:28:06.918345+0000","last_clean":"2024-09-21T06:28:06.918345+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:28:06.918345+0000","last_undegraded":"2024-09-21T06:28:06.918345+0000","last_fullsized":"2024-09-21T06:28:06.918345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567826+0000","last_change":"2024-09-21T06:28:05.274952+0000","last_active":"2024-09-21T06:28:06.567826+0000","last_peered":"2024-09-21T06:28:06.567826+0000","last_clean":"2024-09-21T06:28:06.567826+0000","last_became_active":"2024-09-21T06:28:05.274514+0000","last_became_peered":"2024-09-21T06:28:05.274514+0000","last_unstale":"2024-09-21T06:28:06.567826+0000","last_undegraded":"2024-09-21T06:28:06.567826+0000","last_fullsized":"2024-09-21T06:28:06.567826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:04.636189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567574+0000","last_change":"2024-09-21T06:28:01.224594+0000","last_active":"2024-09-21T06:28:06.567574+0000","last_peered":"2024-09-21T06:28:06.567574+0000","last_clean":"2024-09-21T06:28:06.567574+0000","last_became_active":"2024-09-21T06:28:01.224267+0000","last_became_peered":"2024-09-21T06:28:01.224267+0000","last_unstale":"2024-09-21T06:28:06.567574+0000","last_undegraded":"2024-09-21T06:28:06.567574+0000","last_fullsized":"2024-09-21T06:28:06.567574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:57.968897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919963+0000","last_change":"2024-09-21T06:28:00.226006+0000","last_active":"2024-09-21T06:28:06.919963+0000","last_peered":"2024-09-21T06:28:06.919963+0000","last_clean":"2024-09-21T06:28:06.919963+0000","last_became_active":"2024-09-21T06:28:00.225678+0000","last_became_peered":"2024-09-21T06:28:00.225678+0000","last_unstale":"2024-09-21T06:28:06.919963+0000","last_undegraded":"2024-09-21T06:28:06.919963+0000","last_fullsized":"2024-09-21T06:28:06.919963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:54.460846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271274+0000","last_change":"2024-09-21T06:28:05.280689+0000","last_active":"2024-09-21T06:28:06.271274+0000","last_peered":"2024-09-21T06:28:06.271274+0000","last_clean":"2024-09-21T06:28:06.271274+0000","last_became_active":"2024-09-21T06:28:05.272594+0000","last_became_peered":"2024-09-21T06:28:05.272594+0000","last_unstale":"2024-09-21T06:28:06.271274+0000","last_undegraded":"2024-09-21T06:28:06.271274+0000","last_fullsized":"2024-09-21T06:28:06.271274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:55.258366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917995+0000","last_change":"2024-09-21T06:28:01.230915+0000","last_active":"2024-09-21T06:28:06.917995+0000","last_peered":"2024-09-21T06:28:06.917995+0000","last_clean":"2024-09-21T06:28:06.917995+0000","last_became_active":"2024-09-21T06:28:01.230702+0000","last_became_peered":"2024-09-21T06:28:01.230702+0000","last_unstale":"2024-09-21T06:28:06.917995+0000","last_undegraded":"2024-09-21T06:28:06.917995+0000","last_fullsized":"2024-09-21T06:28:06.917995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:34.107851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918893+0000","last_change":"2024-09-21T06:28:00.214857+0000","last_active":"2024-09-21T06:28:06.918893+0000","last_peered":"2024-09-21T06:28:06.918893+0000","last_clean":"2024-09-21T06:28:06.918893+0000","last_became_active":"2024-09-21T06:28:00.214582+0000","last_became_peered":"2024-09-21T06:28:00.214582+0000","last_unstale":"2024-09-21T06:28:06.918893+0000","last_undegraded":"2024-09-21T06:28:06.918893+0000","last_fullsized":"2024-09-21T06:28:06.918893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:49.842050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918122+0000","last_change":"2024-09-21T06:28:05.273006+0000","last_active":"2024-09-21T06:28:06.918122+0000","last_peered":"2024-09-21T06:28:06.918122+0000","last_clean":"2024-09-21T06:28:06.918122+0000","last_became_active":"2024-09-21T06:28:05.272504+0000","last_became_peered":"2024-09-21T06:28:05.272504+0000","last_unstale":"2024-09-21T06:28:06.918122+0000","last_undegraded":"2024-09-21T06:28:06.918122+0000","last_fullsized":"2024-09-21T06:28:06.918122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:52.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.247357+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.242943+0000","last_became_peered":"2024-09-21T06:28:01.242943+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274916+0000","last_change":"2024-09-21T06:28:00.219919+0000","last_active":"2024-09-21T06:28:06.274916+0000","last_peered":"2024-09-21T06:28:06.274916+0000","last_clean":"2024-09-21T06:28:06.274916+0000","last_became_active":"2024-09-21T06:28:00.219544+0000","last_became_peered":"2024-09-21T06:28:00.219544+0000","last_unstale":"2024-09-21T06:28:06.274916+0000","last_undegraded":"2024-09-21T06:28:06.274916+0000","last_fullsized":"2024-09-21T06:28:06.274916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:21.858722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567045+0000","last_change":"2024-09-21T06:28:05.258119+0000","last_active":"2024-09-21T06:28:06.567045+0000","last_peered":"2024-09-21T06:28:06.567045+0000","last_clean":"2024-09-21T06:28:06.567045+0000","last_became_active":"2024-09-21T06:28:05.257394+0000","last_became_peered":"2024-09-21T06:28:05.257394+0000","last_unstale":"2024-09-21T06:28:06.567045+0000","last_undegraded":"2024-09-21T06:28:06.567045+0000","last_fullsized":"2024-09-21T06:28:06.567045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:13.794574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271877+0000","last_change":"2024-09-21T06:28:01.239234+0000","last_active":"2024-09-21T06:28:06.271877+0000","last_peered":"2024-09-21T06:28:06.271877+0000","last_clean":"2024-09-21T06:28:06.271877+0000","last_became_active":"2024-09-21T06:28:01.238978+0000","last_became_peered":"2024-09-21T06:28:01.238978+0000","last_unstale":"2024-09-21T06:28:06.271877+0000","last_undegraded":"2024-09-21T06:28:06.271877+0000","last_fullsized":"2024-09-21T06:28:06.271877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918626+0000","last_change":"2024-09-21T06:28:00.224968+0000","last_active":"2024-09-21T06:28:06.918626+0000","last_peered":"2024-09-21T06:28:06.918626+0000","last_clean":"2024-09-21T06:28:06.918626+0000","last_became_active":"2024-09-21T06:28:00.224751+0000","last_became_peered":"2024-09-21T06:28:00.224751+0000","last_unstale":"2024-09-21T06:28:06.918626+0000","last_undegraded":"2024-09-21T06:28:06.918626+0000","last_fullsized":"2024-09-21T06:28:06.918626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919013+0000","last_change":"2024-09-21T06:28:05.268891+0000","last_active":"2024-09-21T06:28:06.919013+0000","last_peered":"2024-09-21T06:28:06.919013+0000","last_clean":"2024-09-21T06:28:06.919013+0000","last_became_active":"2024-09-21T06:28:05.268010+0000","last_became_peered":"2024-09-21T06:28:05.268010+0000","last_unstale":"2024-09-21T06:28:06.919013+0000","last_undegraded":"2024-09-21T06:28:06.919013+0000","last_fullsized":"2024-09-21T06:28:06.919013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:04.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566758+0000","last_change":"2024-09-21T06:28:01.225615+0000","last_active":"2024-09-21T06:28:06.566758+0000","last_peered":"2024-09-21T06:28:06.566758+0000","last_clean":"2024-09-21T06:28:06.566758+0000","last_became_active":"2024-09-21T06:28:01.225277+0000","last_became_peered":"2024-09-21T06:28:01.225277+0000","last_unstale":"2024-09-21T06:28:06.566758+0000","last_undegraded":"2024-09-21T06:28:06.566758+0000","last_fullsized":"2024-09-21T06:28:06.566758+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.531813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273831+0000","last_change":"2024-09-21T06:28:00.215632+0000","last_active":"2024-09-21T06:28:06.273831+0000","last_peered":"2024-09-21T06:28:06.273831+0000","last_clean":"2024-09-21T06:28:06.273831+0000","last_became_active":"2024-09-21T06:28:00.215285+0000","last_became_peered":"2024-09-21T06:28:00.215285+0000","last_unstale":"2024-09-21T06:28:06.273831+0000","last_undegraded":"2024-09-21T06:28:06.273831+0000","last_fullsized":"2024-09-21T06:28:06.273831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918621+0000","last_change":"2024-09-21T06:28:05.274989+0000","last_active":"2024-09-21T06:28:06.918621+0000","last_peered":"2024-09-21T06:28:06.918621+0000","last_clean":"2024-09-21T06:28:06.918621+0000","last_became_active":"2024-09-21T06:28:05.274517+0000","last_became_peered":"2024-09-21T06:28:05.274517+0000","last_unstale":"2024-09-21T06:28:06.918621+0000","last_undegraded":"2024-09-21T06:28:06.918621+0000","last_fullsized":"2024-09-21T06:28:06.918621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:09.260032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918268+0000","last_change":"2024-09-21T06:28:01.221685+0000","last_active":"2024-09-21T06:28:06.918268+0000","last_peered":"2024-09-21T06:28:06.918268+0000","last_clean":"2024-09-21T06:28:06.918268+0000","last_became_active":"2024-09-21T06:28:01.221360+0000","last_became_peered":"2024-09-21T06:28:01.221360+0000","last_unstale":"2024-09-21T06:28:06.918268+0000","last_undegraded":"2024-09-21T06:28:06.918268+0000","last_fullsized":"2024-09-21T06:28:06.918268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917445+0000","last_change":"2024-09-21T06:28:00.235459+0000","last_active":"2024-09-21T06:28:06.917445+0000","last_peered":"2024-09-21T06:28:06.917445+0000","last_clean":"2024-09-21T06:28:06.917445+0000","last_became_active":"2024-09-21T06:28:00.235195+0000","last_became_peered":"2024-09-21T06:28:00.235195+0000","last_unstale":"2024-09-21T06:28:06.917445+0000","last_undegraded":"2024-09-21T06:28:06.917445+0000","last_fullsized":"2024-09-21T06:28:06.917445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918627+0000","last_change":"2024-09-21T06:28:05.271996+0000","last_active":"2024-09-21T06:28:06.918627+0000","last_peered":"2024-09-21T06:28:06.918627+0000","last_clean":"2024-09-21T06:28:06.918627+0000","last_became_active":"2024-09-21T06:28:05.270956+0000","last_became_peered":"2024-09-21T06:28:05.270956+0000","last_unstale":"2024-09-21T06:28:06.918627+0000","last_undegraded":"2024-09-21T06:28:06.918627+0000","last_fullsized":"2024-09-21T06:28:06.918627+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:21.181116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566881+0000","last_change":"2024-09-21T06:28:01.237451+0000","last_active":"2024-09-21T06:28:06.566881+0000","last_peered":"2024-09-21T06:28:06.566881+0000","last_clean":"2024-09-21T06:28:06.566881+0000","last_became_active":"2024-09-21T06:28:01.237064+0000","last_became_peered":"2024-09-21T06:28:01.237064+0000","last_unstale":"2024-09-21T06:28:06.566881+0000","last_undegraded":"2024-09-21T06:28:06.566881+0000","last_fullsized":"2024-09-21T06:28:06.566881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.164946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567273+0000","last_change":"2024-09-21T06:28:00.224624+0000","last_active":"2024-09-21T06:28:06.567273+0000","last_peered":"2024-09-21T06:28:06.567273+0000","last_clean":"2024-09-21T06:28:06.567273+0000","last_became_active":"2024-09-21T06:28:00.224259+0000","last_became_peered":"2024-09-21T06:28:00.224259+0000","last_unstale":"2024-09-21T06:28:06.567273+0000","last_undegraded":"2024-09-21T06:28:06.567273+0000","last_fullsized":"2024-09-21T06:28:06.567273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919872+0000","last_change":"2024-09-21T06:28:05.283392+0000","last_active":"2024-09-21T06:28:06.919872+0000","last_peered":"2024-09-21T06:28:06.919872+0000","last_clean":"2024-09-21T06:28:06.919872+0000","last_became_active":"2024-09-21T06:28:05.271401+0000","last_became_peered":"2024-09-21T06:28:05.271401+0000","last_unstale":"2024-09-21T06:28:06.919872+0000","last_undegraded":"2024-09-21T06:28:06.919872+0000","last_fullsized":"2024-09-21T06:28:06.919872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:21.192646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919152+0000","last_change":"2024-09-21T06:28:01.224123+0000","last_active":"2024-09-21T06:28:06.919152+0000","last_peered":"2024-09-21T06:28:06.919152+0000","last_clean":"2024-09-21T06:28:06.919152+0000","last_became_active":"2024-09-21T06:28:01.223773+0000","last_became_peered":"2024-09-21T06:28:01.223773+0000","last_unstale":"2024-09-21T06:28:06.919152+0000","last_undegraded":"2024-09-21T06:28:06.919152+0000","last_fullsized":"2024-09-21T06:28:06.919152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:59.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276563+0000","last_change":"2024-09-21T06:28:00.232406+0000","last_active":"2024-09-21T06:28:06.276563+0000","last_peered":"2024-09-21T06:28:06.276563+0000","last_clean":"2024-09-21T06:28:06.276563+0000","last_became_active":"2024-09-21T06:28:00.232143+0000","last_became_peered":"2024-09-21T06:28:00.232143+0000","last_unstale":"2024-09-21T06:28:06.276563+0000","last_undegraded":"2024-09-21T06:28:06.276563+0000","last_fullsized":"2024-09-21T06:28:06.276563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:33.343595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567590+0000","last_change":"2024-09-21T06:28:05.274346+0000","last_active":"2024-09-21T06:28:06.567590+0000","last_peered":"2024-09-21T06:28:06.567590+0000","last_clean":"2024-09-21T06:28:06.567590+0000","last_became_active":"2024-09-21T06:28:05.273714+0000","last_became_peered":"2024-09-21T06:28:05.273714+0000","last_unstale":"2024-09-21T06:28:06.567590+0000","last_undegraded":"2024-09-21T06:28:06.567590+0000","last_fullsized":"2024-09-21T06:28:06.567590+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:09.113671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918127+0000","last_change":"2024-09-21T06:28:01.238637+0000","last_active":"2024-09-21T06:28:06.918127+0000","last_peered":"2024-09-21T06:28:06.918127+0000","last_clean":"2024-09-21T06:28:06.918127+0000","last_became_active":"2024-09-21T06:28:01.236539+0000","last_became_peered":"2024-09-21T06:28:01.236539+0000","last_unstale":"2024-09-21T06:28:06.918127+0000","last_undegraded":"2024-09-21T06:28:06.918127+0000","last_fullsized":"2024-09-21T06:28:06.918127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.692846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:14.350747+0000","last_change":"2024-09-21T06:28:00.217119+0000","last_active":"2024-09-21T06:28:14.350747+0000","last_peered":"2024-09-21T06:28:14.350747+0000","last_clean":"2024-09-21T06:28:14.350747+0000","last_became_active":"2024-09-21T06:28:00.216806+0000","last_became_peered":"2024-09-21T06:28:00.216806+0000","last_unstale":"2024-09-21T06:28:14.350747+0000","last_undegraded":"2024-09-21T06:28:14.350747+0000","last_fullsized":"2024-09-21T06:28:14.350747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:17.032531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274402+0000","last_change":"2024-09-21T06:28:05.273619+0000","last_active":"2024-09-21T06:28:06.274402+0000","last_peered":"2024-09-21T06:28:06.274402+0000","last_clean":"2024-09-21T06:28:06.274402+0000","last_became_active":"2024-09-21T06:28:05.273193+0000","last_became_peered":"2024-09-21T06:28:05.273193+0000","last_unstale":"2024-09-21T06:28:06.274402+0000","last_undegraded":"2024-09-21T06:28:06.274402+0000","last_fullsized":"2024-09-21T06:28:06.274402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:31.630020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567019+0000","last_change":"2024-09-21T06:28:01.233057+0000","last_active":"2024-09-21T06:28:06.567019+0000","last_peered":"2024-09-21T06:28:06.567019+0000","last_clean":"2024-09-21T06:28:06.567019+0000","last_became_active":"2024-09-21T06:28:01.232602+0000","last_became_peered":"2024-09-21T06:28:01.232602+0000","last_unstale":"2024-09-21T06:28:06.567019+0000","last_undegraded":"2024-09-21T06:28:06.567019+0000","last_fullsized":"2024-09-21T06:28:06.567019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:34.157716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276884+0000","last_change":"2024-09-21T06:28:00.218229+0000","last_active":"2024-09-21T06:28:06.276884+0000","last_peered":"2024-09-21T06:28:06.276884+0000","last_clean":"2024-09-21T06:28:06.276884+0000","last_became_active":"2024-09-21T06:28:00.217635+0000","last_became_peered":"2024-09-21T06:28:00.217635+0000","last_unstale":"2024-09-21T06:28:06.276884+0000","last_undegraded":"2024-09-21T06:28:06.276884+0000","last_fullsized":"2024-09-21T06:28:06.276884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:52.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271664+0000","last_change":"2024-09-21T06:28:05.277550+0000","last_active":"2024-09-21T06:28:06.271664+0000","last_peered":"2024-09-21T06:28:06.271664+0000","last_clean":"2024-09-21T06:28:06.271664+0000","last_became_active":"2024-09-21T06:28:05.276077+0000","last_became_peered":"2024-09-21T06:28:05.276077+0000","last_unstale":"2024-09-21T06:28:06.271664+0000","last_undegraded":"2024-09-21T06:28:06.271664+0000","last_fullsized":"2024-09-21T06:28:06.271664+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.300870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918857+0000","last_change":"2024-09-21T06:28:01.232773+0000","last_active":"2024-09-21T06:28:06.918857+0000","last_peered":"2024-09-21T06:28:06.918857+0000","last_clean":"2024-09-21T06:28:06.918857+0000","last_became_active":"2024-09-21T06:28:01.232081+0000","last_became_peered":"2024-09-21T06:28:01.232081+0000","last_unstale":"2024-09-21T06:28:06.918857+0000","last_undegraded":"2024-09-21T06:28:06.918857+0000","last_fullsized":"2024-09-21T06:28:06.918857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:54.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917778+0000","last_change":"2024-09-21T06:28:00.234206+0000","last_active":"2024-09-21T06:28:06.917778+0000","last_peered":"2024-09-21T06:28:06.917778+0000","last_clean":"2024-09-21T06:28:06.917778+0000","last_became_active":"2024-09-21T06:28:00.233696+0000","last_became_peered":"2024-09-21T06:28:00.233696+0000","last_unstale":"2024-09-21T06:28:06.917778+0000","last_undegraded":"2024-09-21T06:28:06.917778+0000","last_fullsized":"2024-09-21T06:28:06.917778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:40.091754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275902+0000","last_change":"2024-09-21T06:28:05.283000+0000","last_active":"2024-09-21T06:28:06.275902+0000","last_peered":"2024-09-21T06:28:06.275902+0000","last_clean":"2024-09-21T06:28:06.275902+0000","last_became_active":"2024-09-21T06:28:05.282638+0000","last_became_peered":"2024-09-21T06:28:05.282638+0000","last_unstale":"2024-09-21T06:28:06.275902+0000","last_undegraded":"2024-09-21T06:28:06.275902+0000","last_fullsized":"2024-09-21T06:28:06.275902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271814+0000","last_change":"2024-09-21T06:28:01.221987+0000","last_active":"2024-09-21T06:28:06.271814+0000","last_peered":"2024-09-21T06:28:06.271814+0000","last_clean":"2024-09-21T06:28:06.271814+0000","last_became_active":"2024-09-21T06:28:01.221678+0000","last_became_peered":"2024-09-21T06:28:01.221678+0000","last_unstale":"2024-09-21T06:28:06.271814+0000","last_undegraded":"2024-09-21T06:28:06.271814+0000","last_fullsized":"2024-09-21T06:28:06.271814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:20.730550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919718+0000","last_change":"2024-09-21T06:28:00.202531+0000","last_active":"2024-09-21T06:28:06.919718+0000","last_peered":"2024-09-21T06:28:06.919718+0000","last_clean":"2024-09-21T06:28:06.919718+0000","last_became_active":"2024-09-21T06:28:00.202126+0000","last_became_peered":"2024-09-21T06:28:00.202126+0000","last_unstale":"2024-09-21T06:28:06.919718+0000","last_undegraded":"2024-09-21T06:28:06.919718+0000","last_fullsized":"2024-09-21T06:28:06.919718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:33.977242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274058+0000","last_change":"2024-09-21T06:28:05.278157+0000","last_active":"2024-09-21T06:28:06.274058+0000","last_peered":"2024-09-21T06:28:06.274058+0000","last_clean":"2024-09-21T06:28:06.274058+0000","last_became_active":"2024-09-21T06:28:05.277532+0000","last_became_peered":"2024-09-21T06:28:05.277532+0000","last_unstale":"2024-09-21T06:28:06.274058+0000","last_undegraded":"2024-09-21T06:28:06.274058+0000","last_fullsized":"2024-09-21T06:28:06.274058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.737605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274991+0000","last_change":"2024-09-21T06:28:05.278200+0000","last_active":"2024-09-21T06:28:06.274991+0000","last_peered":"2024-09-21T06:28:06.274991+0000","last_clean":"2024-09-21T06:28:06.274991+0000","last_became_active":"2024-09-21T06:28:05.277575+0000","last_became_peered":"2024-09-21T06:28:05.277575+0000","last_unstale":"2024-09-21T06:28:06.274991+0000","last_undegraded":"2024-09-21T06:28:06.274991+0000","last_fullsized":"2024-09-21T06:28:06.274991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:05.311819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.273908+0000","last_change":"2024-09-21T06:28:01.228981+0000","last_active":"2024-09-21T06:28:06.273908+0000","last_peered":"2024-09-21T06:28:06.273908+0000","last_clean":"2024-09-21T06:28:06.273908+0000","last_became_active":"2024-09-21T06:28:01.222592+0000","last_became_peered":"2024-09-21T06:28:01.222592+0000","last_unstale":"2024-09-21T06:28:06.273908+0000","last_undegraded":"2024-09-21T06:28:06.273908+0000","last_fullsized":"2024-09-21T06:28:06.273908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566966+0000","last_change":"2024-09-21T06:28:00.214521+0000","last_active":"2024-09-21T06:28:06.566966+0000","last_peered":"2024-09-21T06:28:06.566966+0000","last_clean":"2024-09-21T06:28:06.566966+0000","last_became_active":"2024-09-21T06:28:00.212408+0000","last_became_peered":"2024-09-21T06:28:00.212408+0000","last_unstale":"2024-09-21T06:28:06.566966+0000","last_undegraded":"2024-09-21T06:28:06.566966+0000","last_fullsized":"2024-09-21T06:28:06.566966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:21.852670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567614+0000","last_change":"2024-09-21T06:28:05.265492+0000","last_active":"2024-09-21T06:28:06.567614+0000","last_peered":"2024-09-21T06:28:06.567614+0000","last_clean":"2024-09-21T06:28:06.567614+0000","last_became_active":"2024-09-21T06:28:05.264988+0000","last_became_peered":"2024-09-21T06:28:05.264988+0000","last_unstale":"2024-09-21T06:28:06.567614+0000","last_undegraded":"2024-09-21T06:28:06.567614+0000","last_fullsized":"2024-09-21T06:28:06.567614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:02.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917735+0000","last_change":"2024-09-21T06:28:01.228392+0000","last_active":"2024-09-21T06:28:06.917735+0000","last_peered":"2024-09-21T06:28:06.917735+0000","last_clean":"2024-09-21T06:28:06.917735+0000","last_became_active":"2024-09-21T06:28:01.228163+0000","last_became_peered":"2024-09-21T06:28:01.228163+0000","last_unstale":"2024-09-21T06:28:06.917735+0000","last_undegraded":"2024-09-21T06:28:06.917735+0000","last_fullsized":"2024-09-21T06:28:06.917735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:24.791034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918738+0000","last_change":"2024-09-21T06:28:00.228862+0000","last_active":"2024-09-21T06:28:06.918738+0000","last_peered":"2024-09-21T06:28:06.918738+0000","last_clean":"2024-09-21T06:28:06.918738+0000","last_became_active":"2024-09-21T06:28:00.228490+0000","last_became_peered":"2024-09-21T06:28:00.228490+0000","last_unstale":"2024-09-21T06:28:06.918738+0000","last_undegraded":"2024-09-21T06:28:06.918738+0000","last_fullsized":"2024-09-21T06:28:06.918738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:09:00.810925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567138+0000","last_change":"2024-09-21T06:28:00.234315+0000","last_active":"2024-09-21T06:28:06.567138+0000","last_peered":"2024-09-21T06:28:06.567138+0000","last_clean":"2024-09-21T06:28:06.567138+0000","last_became_active":"2024-09-21T06:28:00.233567+0000","last_became_peered":"2024-09-21T06:28:00.233567+0000","last_unstale":"2024-09-21T06:28:06.567138+0000","last_undegraded":"2024-09-21T06:28:06.567138+0000","last_fullsized":"2024-09-21T06:28:06.567138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:59.800994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274635+0000","last_change":"2024-09-21T06:28:01.236968+0000","last_active":"2024-09-21T06:28:06.274635+0000","last_peered":"2024-09-21T06:28:06.274635+0000","last_clean":"2024-09-21T06:28:06.274635+0000","last_became_active":"2024-09-21T06:28:01.236577+0000","last_became_peered":"2024-09-21T06:28:01.236577+0000","last_unstale":"2024-09-21T06:28:06.274635+0000","last_undegraded":"2024-09-21T06:28:06.274635+0000","last_fullsized":"2024-09-21T06:28:06.274635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:01.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274854+0000","last_change":"2024-09-21T06:28:05.271567+0000","last_active":"2024-09-21T06:28:06.274854+0000","last_peered":"2024-09-21T06:28:06.274854+0000","last_clean":"2024-09-21T06:28:06.274854+0000","last_became_active":"2024-09-21T06:28:05.271160+0000","last_became_peered":"2024-09-21T06:28:05.271160+0000","last_unstale":"2024-09-21T06:28:06.274854+0000","last_undegraded":"2024-09-21T06:28:06.274854+0000","last_fullsized":"2024-09-21T06:28:06.274854+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:02.495413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918355+0000","last_change":"2024-09-21T06:28:00.215612+0000","last_active":"2024-09-21T06:28:06.918355+0000","last_peered":"2024-09-21T06:28:06.918355+0000","last_clean":"2024-09-21T06:28:06.918355+0000","last_became_active":"2024-09-21T06:28:00.214929+0000","last_became_peered":"2024-09-21T06:28:00.214929+0000","last_unstale":"2024-09-21T06:28:06.918355+0000","last_undegraded":"2024-09-21T06:28:06.918355+0000","last_fullsized":"2024-09-21T06:28:06.918355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:59.180913+0000","last_clean_scrub_stamp":"2024-09-21T06:27:59.180913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:41.140041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918598+0000","last_change":"2024-09-21T06:28:01.234866+0000","last_active":"2024-09-21T06:28:06.918598+0000","last_peered":"2024-09-21T06:28:06.918598+0000","last_clean":"2024-09-21T06:28:06.918598+0000","last_became_active":"2024-09-21T06:28:01.234568+0000","last_became_peered":"2024-09-21T06:28:01.234568+0000","last_unstale":"2024-09-21T06:28:06.918598+0000","last_undegraded":"2024-09-21T06:28:06.918598+0000","last_fullsized":"2024-09-21T06:28:06.918598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:00.807517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.276818+0000","last_change":"2024-09-21T06:28:05.287452+0000","last_active":"2024-09-21T06:28:06.276818+0000","last_peered":"2024-09-21T06:28:06.276818+0000","last_clean":"2024-09-21T06:28:06.276818+0000","last_became_active":"2024-09-21T06:28:05.286963+0000","last_became_peered":"2024-09-21T06:28:05.286963+0000","last_unstale":"2024-09-21T06:28:06.276818+0000","last_undegraded":"2024-09-21T06:28:06.276818+0000","last_fullsized":"2024-09-21T06:28:06.276818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:04.232196+0000","last_clean_scrub_stamp":"2024-09-21T06:28:04.232196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:03.321463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271084+0000","last_change":"2024-09-21T06:28:01.232705+0000","last_active":"2024-09-21T06:28:06.271084+0000","last_peered":"2024-09-21T06:28:06.271084+0000","last_clean":"2024-09-21T06:28:06.271084+0000","last_became_active":"2024-09-21T06:28:01.232305+0000","last_became_peered":"2024-09-21T06:28:01.232305+0000","last_unstale":"2024-09-21T06:28:06.271084+0000","last_undegraded":"2024-09-21T06:28:06.271084+0000","last_fullsized":"2024-09-21T06:28:06.271084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.275922+0000","last_change":"2024-09-21T06:28:01.235529+0000","last_active":"2024-09-21T06:28:06.275922+0000","last_peered":"2024-09-21T06:28:06.275922+0000","last_clean":"2024-09-21T06:28:06.275922+0000","last_became_active":"2024-09-21T06:28:01.235170+0000","last_became_peered":"2024-09-21T06:28:01.235170+0000","last_unstale":"2024-09-21T06:28:06.275922+0000","last_undegraded":"2024-09-21T06:28:06.275922+0000","last_fullsized":"2024-09-21T06:28:06.275922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.262700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566964+0000","last_change":"2024-09-21T06:28:01.242450+0000","last_active":"2024-09-21T06:28:06.566964+0000","last_peered":"2024-09-21T06:28:06.566964+0000","last_clean":"2024-09-21T06:28:06.566964+0000","last_became_active":"2024-09-21T06:28:01.242078+0000","last_became_peered":"2024-09-21T06:28:01.242078+0000","last_unstale":"2024-09-21T06:28:06.566964+0000","last_undegraded":"2024-09-21T06:28:06.566964+0000","last_fullsized":"2024-09-21T06:28:06.566964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:58.450653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918263+0000","last_change":"2024-09-21T06:28:01.230631+0000","last_active":"2024-09-21T06:28:06.918263+0000","last_peered":"2024-09-21T06:28:06.918263+0000","last_clean":"2024-09-21T06:28:06.918263+0000","last_became_active":"2024-09-21T06:28:01.223844+0000","last_became_peered":"2024-09-21T06:28:01.223844+0000","last_unstale":"2024-09-21T06:28:06.918263+0000","last_undegraded":"2024-09-21T06:28:06.918263+0000","last_fullsized":"2024-09-21T06:28:06.918263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:28.649005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919766+0000","last_change":"2024-09-21T06:28:01.248984+0000","last_active":"2024-09-21T06:28:06.919766+0000","last_peered":"2024-09-21T06:28:06.919766+0000","last_clean":"2024-09-21T06:28:06.919766+0000","last_became_active":"2024-09-21T06:28:01.248639+0000","last_became_peered":"2024-09-21T06:28:01.248639+0000","last_unstale":"2024-09-21T06:28:06.919766+0000","last_undegraded":"2024-09-21T06:28:06.919766+0000","last_fullsized":"2024-09-21T06:28:06.919766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.659556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919689+0000","last_change":"2024-09-21T06:28:01.226847+0000","last_active":"2024-09-21T06:28:06.919689+0000","last_peered":"2024-09-21T06:28:06.919689+0000","last_clean":"2024-09-21T06:28:06.919689+0000","last_became_active":"2024-09-21T06:28:01.226536+0000","last_became_peered":"2024-09-21T06:28:01.226536+0000","last_unstale":"2024-09-21T06:28:06.919689+0000","last_undegraded":"2024-09-21T06:28:06.919689+0000","last_fullsized":"2024-09-21T06:28:06.919689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:59.276703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271732+0000","last_change":"2024-09-21T06:28:01.223972+0000","last_active":"2024-09-21T06:28:06.271732+0000","last_peered":"2024-09-21T06:28:06.271732+0000","last_clean":"2024-09-21T06:28:06.271732+0000","last_became_active":"2024-09-21T06:28:01.223601+0000","last_became_peered":"2024-09-21T06:28:01.223601+0000","last_unstale":"2024-09-21T06:28:06.271732+0000","last_undegraded":"2024-09-21T06:28:06.271732+0000","last_fullsized":"2024-09-21T06:28:06.271732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918404+0000","last_change":"2024-09-21T06:28:01.233118+0000","last_active":"2024-09-21T06:28:06.918404+0000","last_peered":"2024-09-21T06:28:06.918404+0000","last_clean":"2024-09-21T06:28:06.918404+0000","last_became_active":"2024-09-21T06:28:01.232482+0000","last_became_peered":"2024-09-21T06:28:01.232482+0000","last_unstale":"2024-09-21T06:28:06.918404+0000","last_undegraded":"2024-09-21T06:28:06.918404+0000","last_fullsized":"2024-09-21T06:28:06.918404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:39.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566851+0000","last_change":"2024-09-21T06:28:01.243008+0000","last_active":"2024-09-21T06:28:06.566851+0000","last_peered":"2024-09-21T06:28:06.566851+0000","last_clean":"2024-09-21T06:28:06.566851+0000","last_became_active":"2024-09-21T06:28:01.242713+0000","last_became_peered":"2024-09-21T06:28:01.242713+0000","last_unstale":"2024-09-21T06:28:06.566851+0000","last_undegraded":"2024-09-21T06:28:06.566851+0000","last_fullsized":"2024-09-21T06:28:06.566851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:17.147886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917756+0000","last_change":"2024-09-21T06:28:01.216205+0000","last_active":"2024-09-21T06:28:06.917756+0000","last_peered":"2024-09-21T06:28:06.917756+0000","last_clean":"2024-09-21T06:28:06.917756+0000","last_became_active":"2024-09-21T06:28:01.215869+0000","last_became_peered":"2024-09-21T06:28:01.215869+0000","last_unstale":"2024-09-21T06:28:06.917756+0000","last_undegraded":"2024-09-21T06:28:06.917756+0000","last_fullsized":"2024-09-21T06:28:06.917756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.725175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918494+0000","last_change":"2024-09-21T06:28:01.246071+0000","last_active":"2024-09-21T06:28:06.918494+0000","last_peered":"2024-09-21T06:28:06.918494+0000","last_clean":"2024-09-21T06:28:06.918494+0000","last_became_active":"2024-09-21T06:28:01.245739+0000","last_became_peered":"2024-09-21T06:28:01.245739+0000","last_unstale":"2024-09-21T06:28:06.918494+0000","last_undegraded":"2024-09-21T06:28:06.918494+0000","last_fullsized":"2024-09-21T06:28:06.918494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.274517+0000","last_change":"2024-09-21T06:28:01.232435+0000","last_active":"2024-09-21T06:28:06.274517+0000","last_peered":"2024-09-21T06:28:06.274517+0000","last_clean":"2024-09-21T06:28:06.274517+0000","last_became_active":"2024-09-21T06:28:01.229739+0000","last_became_peered":"2024-09-21T06:28:01.229739+0000","last_unstale":"2024-09-21T06:28:06.274517+0000","last_undegraded":"2024-09-21T06:28:06.274517+0000","last_fullsized":"2024-09-21T06:28:06.274517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:59.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917974+0000","last_change":"2024-09-21T06:28:01.242252+0000","last_active":"2024-09-21T06:28:06.917974+0000","last_peered":"2024-09-21T06:28:06.917974+0000","last_clean":"2024-09-21T06:28:06.917974+0000","last_became_active":"2024-09-21T06:28:01.241926+0000","last_became_peered":"2024-09-21T06:28:01.241926+0000","last_unstale":"2024-09-21T06:28:06.917974+0000","last_undegraded":"2024-09-21T06:28:06.917974+0000","last_fullsized":"2024-09-21T06:28:06.917974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:31.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567677+0000","last_change":"2024-09-21T06:28:01.232426+0000","last_active":"2024-09-21T06:28:06.567677+0000","last_peered":"2024-09-21T06:28:06.567677+0000","last_clean":"2024-09-21T06:28:06.567677+0000","last_became_active":"2024-09-21T06:28:01.232067+0000","last_became_peered":"2024-09-21T06:28:01.232067+0000","last_unstale":"2024-09-21T06:28:06.567677+0000","last_undegraded":"2024-09-21T06:28:06.567677+0000","last_fullsized":"2024-09-21T06:28:06.567677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:28.274577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.277142+0000","last_change":"2024-09-21T06:28:01.247978+0000","last_active":"2024-09-21T06:28:06.277142+0000","last_peered":"2024-09-21T06:28:06.277142+0000","last_clean":"2024-09-21T06:28:06.277142+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.277142+0000","last_undegraded":"2024-09-21T06:28:06.277142+0000","last_fullsized":"2024-09-21T06:28:06.277142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.567819+0000","last_change":"2024-09-21T06:28:01.236566+0000","last_active":"2024-09-21T06:28:06.567819+0000","last_peered":"2024-09-21T06:28:06.567819+0000","last_clean":"2024-09-21T06:28:06.567819+0000","last_became_active":"2024-09-21T06:28:01.236209+0000","last_became_peered":"2024-09-21T06:28:01.236209+0000","last_unstale":"2024-09-21T06:28:06.567819+0000","last_undegraded":"2024-09-21T06:28:06.567819+0000","last_fullsized":"2024-09-21T06:28:06.567819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:55.410838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.566732+0000","last_change":"2024-09-21T06:28:01.227863+0000","last_active":"2024-09-21T06:28:06.566732+0000","last_peered":"2024-09-21T06:28:06.566732+0000","last_clean":"2024-09-21T06:28:06.566732+0000","last_became_active":"2024-09-21T06:28:01.227553+0000","last_became_peered":"2024-09-21T06:28:01.227553+0000","last_unstale":"2024-09-21T06:28:06.566732+0000","last_undegraded":"2024-09-21T06:28:06.566732+0000","last_fullsized":"2024-09-21T06:28:06.566732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:51.034026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917669+0000","last_change":"2024-09-21T06:28:01.242950+0000","last_active":"2024-09-21T06:28:06.917669+0000","last_peered":"2024-09-21T06:28:06.917669+0000","last_clean":"2024-09-21T06:28:06.917669+0000","last_became_active":"2024-09-21T06:28:01.242715+0000","last_became_peered":"2024-09-21T06:28:01.242715+0000","last_unstale":"2024-09-21T06:28:06.917669+0000","last_undegraded":"2024-09-21T06:28:06.917669+0000","last_fullsized":"2024-09-21T06:28:06.917669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:12.827134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919061+0000","last_change":"2024-09-21T06:28:01.231009+0000","last_active":"2024-09-21T06:28:06.919061+0000","last_peered":"2024-09-21T06:28:06.919061+0000","last_clean":"2024-09-21T06:28:06.919061+0000","last_became_active":"2024-09-21T06:28:01.230706+0000","last_became_peered":"2024-09-21T06:28:01.230706+0000","last_unstale":"2024-09-21T06:28:06.919061+0000","last_undegraded":"2024-09-21T06:28:06.919061+0000","last_fullsized":"2024-09-21T06:28:06.919061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:38.274224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919349+0000","last_change":"2024-09-21T06:28:01.235786+0000","last_active":"2024-09-21T06:28:06.919349+0000","last_peered":"2024-09-21T06:28:06.919349+0000","last_clean":"2024-09-21T06:28:06.919349+0000","last_became_active":"2024-09-21T06:28:01.235551+0000","last_became_peered":"2024-09-21T06:28:01.235551+0000","last_unstale":"2024-09-21T06:28:06.919349+0000","last_undegraded":"2024-09-21T06:28:06.919349+0000","last_fullsized":"2024-09-21T06:28:06.919349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:00.237301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917934+0000","last_change":"2024-09-21T06:28:01.242543+0000","last_active":"2024-09-21T06:28:06.917934+0000","last_peered":"2024-09-21T06:28:06.917934+0000","last_clean":"2024-09-21T06:28:06.917934+0000","last_became_active":"2024-09-21T06:28:01.242225+0000","last_became_peered":"2024-09-21T06:28:01.242225+0000","last_unstale":"2024-09-21T06:28:06.917934+0000","last_undegraded":"2024-09-21T06:28:06.917934+0000","last_fullsized":"2024-09-21T06:28:06.917934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:39.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918078+0000","last_change":"2024-09-21T06:28:01.237214+0000","last_active":"2024-09-21T06:28:06.918078+0000","last_peered":"2024-09-21T06:28:06.918078+0000","last_clean":"2024-09-21T06:28:06.918078+0000","last_became_active":"2024-09-21T06:28:01.235938+0000","last_became_peered":"2024-09-21T06:28:01.235938+0000","last_unstale":"2024-09-21T06:28:06.918078+0000","last_undegraded":"2024-09-21T06:28:06.918078+0000","last_fullsized":"2024-09-21T06:28:06.918078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.564737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917663+0000","last_change":"2024-09-21T06:28:01.251436+0000","last_active":"2024-09-21T06:28:06.917663+0000","last_peered":"2024-09-21T06:28:06.917663+0000","last_clean":"2024-09-21T06:28:06.917663+0000","last_became_active":"2024-09-21T06:28:01.250562+0000","last_became_peered":"2024-09-21T06:28:01.250562+0000","last_unstale":"2024-09-21T06:28:06.917663+0000","last_undegraded":"2024-09-21T06:28:06.917663+0000","last_fullsized":"2024-09-21T06:28:06.917663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:58.692504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.919694+0000","last_change":"2024-09-21T06:28:01.226763+0000","last_active":"2024-09-21T06:28:06.919694+0000","last_peered":"2024-09-21T06:28:06.919694+0000","last_clean":"2024-09-21T06:28:06.919694+0000","last_became_active":"2024-09-21T06:28:01.226438+0000","last_became_peered":"2024-09-21T06:28:01.226438+0000","last_unstale":"2024-09-21T06:28:06.919694+0000","last_undegraded":"2024-09-21T06:28:06.919694+0000","last_fullsized":"2024-09-21T06:28:06.919694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.381108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917570+0000","last_change":"2024-09-21T06:28:01.242596+0000","last_active":"2024-09-21T06:28:06.917570+0000","last_peered":"2024-09-21T06:28:06.917570+0000","last_clean":"2024-09-21T06:28:06.917570+0000","last_became_active":"2024-09-21T06:28:01.242205+0000","last_became_peered":"2024-09-21T06:28:01.242205+0000","last_unstale":"2024-09-21T06:28:06.917570+0000","last_undegraded":"2024-09-21T06:28:06.917570+0000","last_fullsized":"2024-09-21T06:28:06.917570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:44.145821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917382+0000","last_change":"2024-09-21T06:28:01.249034+0000","last_active":"2024-09-21T06:28:06.917382+0000","last_peered":"2024-09-21T06:28:06.917382+0000","last_clean":"2024-09-21T06:28:06.917382+0000","last_became_active":"2024-09-21T06:28:01.248727+0000","last_became_peered":"2024-09-21T06:28:01.248727+0000","last_unstale":"2024-09-21T06:28:06.917382+0000","last_undegraded":"2024-09-21T06:28:06.917382+0000","last_fullsized":"2024-09-21T06:28:06.917382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:06.428324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271467+0000","last_change":"2024-09-21T06:28:01.215764+0000","last_active":"2024-09-21T06:28:06.271467+0000","last_peered":"2024-09-21T06:28:06.271467+0000","last_clean":"2024-09-21T06:28:06.271467+0000","last_became_active":"2024-09-21T06:28:01.215412+0000","last_became_peered":"2024-09-21T06:28:01.215412+0000","last_unstale":"2024-09-21T06:28:06.271467+0000","last_undegraded":"2024-09-21T06:28:06.271467+0000","last_fullsized":"2024-09-21T06:28:06.271467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:22.300961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.917795+0000","last_change":"2024-09-21T06:28:01.253480+0000","last_active":"2024-09-21T06:28:06.917795+0000","last_peered":"2024-09-21T06:28:06.917795+0000","last_clean":"2024-09-21T06:28:06.917795+0000","last_became_active":"2024-09-21T06:28:01.247649+0000","last_became_peered":"2024-09-21T06:28:01.247649+0000","last_unstale":"2024-09-21T06:28:06.917795+0000","last_undegraded":"2024-09-21T06:28:06.917795+0000","last_fullsized":"2024-09-21T06:28:06.917795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:51.213607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.271436+0000","last_change":"2024-09-21T06:28:01.231505+0000","last_active":"2024-09-21T06:28:06.271436+0000","last_peered":"2024-09-21T06:28:06.271436+0000","last_clean":"2024-09-21T06:28:06.271436+0000","last_became_active":"2024-09-21T06:28:01.223703+0000","last_became_peered":"2024-09-21T06:28:01.223703+0000","last_unstale":"2024-09-21T06:28:06.271436+0000","last_undegraded":"2024-09-21T06:28:06.271436+0000","last_fullsized":"2024-09-21T06:28:06.271436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.164731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918618+0000","last_change":"2024-09-21T06:28:01.236472+0000","last_active":"2024-09-21T06:28:06.918618+0000","last_peered":"2024-09-21T06:28:06.918618+0000","last_clean":"2024-09-21T06:28:06.918618+0000","last_became_active":"2024-09-21T06:28:01.221503+0000","last_became_peered":"2024-09-21T06:28:01.221503+0000","last_unstale":"2024-09-21T06:28:06.918618+0000","last_undegraded":"2024-09-21T06:28:06.918618+0000","last_fullsized":"2024-09-21T06:28:06.918618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:35.021658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918538+0000","last_change":"2024-09-21T06:28:01.227458+0000","last_active":"2024-09-21T06:28:06.918538+0000","last_peered":"2024-09-21T06:28:06.918538+0000","last_clean":"2024-09-21T06:28:06.918538+0000","last_became_active":"2024-09-21T06:28:01.227145+0000","last_became_peered":"2024-09-21T06:28:01.227145+0000","last_unstale":"2024-09-21T06:28:06.918538+0000","last_undegraded":"2024-09-21T06:28:06.918538+0000","last_fullsized":"2024-09-21T06:28:06.918538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:24.909945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T06:28:06.918495+0000","last_change":"2024-09-21T06:28:01.233932+0000","last_active":"2024-09-21T06:28:06.918495+0000","last_peered":"2024-09-21T06:28:06.918495+0000","last_clean":"2024-09-21T06:28:06.918495+0000","last_became_active":"2024-09-21T06:28:01.233617+0000","last_became_peered":"2024-09-21T06:28:01.233617+0000","last_unstale":"2024-09-21T06:28:06.918495+0000","last_undegraded":"2024-09-21T06:28:06.918495+0000","last_fullsized":"2024-09-21T06:28:06.918495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:28:00.187436+0000","last_clean_scrub_stamp":"2024-09-21T06:28:00.187436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:27.585260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:28:27.474 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:28:27.475 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T06:28:27.475 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T06:28:27.475 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T06:28:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 -- 172.21.15.37:0/446493228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608004800 msgr2=0x7f2608004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 --2- 172.21.15.37:0/446493228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608004800 0x7f2608004c10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f26240cd8c0 tx=0x7f2608005830 comp rx=0 tx=0).stop 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 -- 172.21.15.37:0/446493228 shutdown_connections 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 --2- 172.21.15.37:0/446493228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608004800 0x7f2608004c10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 -- 172.21.15.37:0/446493228 >> 172.21.15.37:0/446493228 conn(0x7f26240cca40 msgr2=0x7f26240cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 -- 172.21.15.37:0/446493228 shutdown_connections 2024-09-21T06:28:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.732+0000 7f262aa00640 1 -- 172.21.15.37:0/446493228 wait complete. 2024-09-21T06:28:27.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.734+0000 7f262aa00640 1 Processor -- start 2024-09-21T06:28:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.735+0000 7f262aa00640 1 -- start start 2024-09-21T06:28:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2608004800 0x7f26240b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26241565b0 0x7f262409f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f262406e180 con 0x7f262409fae0 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f262406e000 con 0x7f2608004800 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f262aa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f26240cbc10 con 0x7f26241565b0 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f2628a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f2628a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40438/0 (socket says 172.21.15.37:40438) 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f2628a00640 1 -- 172.21.15.37:0/3388667442 learned_addr learned my addr 172.21.15.37:0/3388667442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f2623e00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2608004800 0x7f26240b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.736+0000 7f261be00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26241565b0 0x7f262409f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 -- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26241565b0 msgr2=0x7f262409f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26241565b0 0x7f262409f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 -- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2608004800 msgr2=0x7f26240b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2608004800 0x7f26240b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 -- 172.21.15.37:0/3388667442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240a0430 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2628a00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f262415dcb0 tx=0x7f261c0064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c011900 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f261c011aa0 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00f430 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.737+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2624156d20 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.738+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241572e0 con 0x7f262409fae0 2024-09-21T06:28:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.739+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2624156a90 con 0x7f262409fae0 2024-09-21T06:28:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.740+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f261c00f5d0 con 0x7f262409fae0 2024-09-21T06:28:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.741+0000 7f2620c00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 0x7f260007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.741+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f261c09b7b0 con 0x7f262409fae0 2024-09-21T06:28:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.741+0000 7f2623e00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 0x7f260007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:27.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.742+0000 7f2623e00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 0x7f260007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2614001050 tx=0x7f26140011e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.745+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f262406e740 con 0x7f262409fae0 2024-09-21T06:28:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.982+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f262409f290 con 0x7f262409fae0 2024-09-21T06:28:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.982+0000 7f2620c00640 1 -- 172.21.15.37:0/3388667442 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f261c0631c0 con 0x7f262409fae0 2024-09-21T06:28:27.983 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:27.984 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T06:28:27.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 msgr2=0x7f260007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 0x7f260007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2614001050 tx=0x7f26140011e0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 msgr2=0x7f262409fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f262415dcb0 tx=0x7f261c0064f0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 shutdown_connections 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2600078bc0 0x7f260007b0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262409fae0 0x7f262409fef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26241565b0 0x7f262409f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 --2- 172.21.15.37:0/3388667442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2608004800 0x7f26240b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.984+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 >> 172.21.15.37:0/3388667442 conn(0x7f26240cca40 msgr2=0x7f26240c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.985+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 shutdown_connections 2024-09-21T06:28:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:27.985+0000 7f262aa00640 1 -- 172.21.15.37:0/3388667442 wait complete. 2024-09-21T06:28:28.008 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T06:28:28.008 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 -- 172.21.15.37:0/2576861066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2480b9e40 msgr2=0x7fb2481622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 --2- 172.21.15.37:0/2576861066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2480b9e40 0x7fb2481622b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005fd0 tx=0x7fb23c02a670 comp rx=0 tx=0).stop 2024-09-21T06:28:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 -- 172.21.15.37:0/2576861066 shutdown_connections 2024-09-21T06:28:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 --2- 172.21.15.37:0/2576861066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2480b9e40 0x7fb2481622b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 -- 172.21.15.37:0/2576861066 >> 172.21.15.37:0/2576861066 conn(0x7fb2480d1930 msgr2=0x7fb2480d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 -- 172.21.15.37:0/2576861066 shutdown_connections 2024-09-21T06:28:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.235+0000 7fb24f000640 1 -- 172.21.15.37:0/2576861066 wait complete. 2024-09-21T06:28:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.237+0000 7fb24f000640 1 Processor -- start 2024-09-21T06:28:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.237+0000 7fb24f000640 1 -- start start 2024-09-21T06:28:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb24806d9c0 0x7fb2480ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 0x7fb248144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481555b0 con 0x7fb2481627f0 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2480d37e0 con 0x7fb2480b9e40 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24f000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb248155730 con 0x7fb24806d9c0 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40460/0 (socket says 172.21.15.37:40460) 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 -- 172.21.15.37:0/1535683248 learned_addr learned my addr 172.21.15.37:0/1535683248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24d200640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 0x7fb248144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24dc00640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb24806d9c0 0x7fb2480ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 -- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb24806d9c0 msgr2=0x7fb2480ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb24806d9c0 0x7fb2480ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 -- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 msgr2=0x7fb248144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.239+0000 7fb24e600640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 0x7fb248144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb24e600640 1 -- 172.21.15.37:0/1535683248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2480ce420 con 0x7fb2481627f0 2024-09-21T06:28:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb24d200640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 0x7fb248144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb24e600640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c700 tx=0x7fb24000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240003b70 con 0x7fb2481627f0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb240003d10 con 0x7fb2481627f0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24000aab0 con 0x7fb2481627f0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2481595a0 con 0x7fb2481627f0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.240+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24815ac20 con 0x7fb2481627f0 2024-09-21T06:28:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.241+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2481622b0 con 0x7fb2481627f0 2024-09-21T06:28:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.243+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb24000ac50 con 0x7fb2481627f0 2024-09-21T06:28:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.244+0000 7fb245600640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 0x7fb22807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.244+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb24009af70 con 0x7fb2481627f0 2024-09-21T06:28:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.244+0000 7fb24dc00640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 0x7fb22807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.245+0000 7fb24dc00640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 0x7fb22807b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb23c033000 tx=0x7fb23c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.248+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb248162c00 con 0x7fb2481627f0 2024-09-21T06:28:28.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.480+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb24815aa10 con 0x7fb2481627f0 2024-09-21T06:28:28.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.481+0000 7fb245600640 1 -- 172.21.15.37:0/1535683248 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7fb24815aa10 con 0x7fb2481627f0 2024-09-21T06:28:28.482 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:28.482 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:28.482 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:28.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 msgr2=0x7fb22807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 0x7fb22807b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb23c033000 tx=0x7fb23c004730 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 msgr2=0x7fb248145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c700 tx=0x7fb24000cbc0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 shutdown_connections 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb228078b90 0x7fb22807b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2481627f0 0x7fb248145350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2480b9e40 0x7fb248144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 --2- 172.21.15.37:0/1535683248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb24806d9c0 0x7fb2480ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 >> 172.21.15.37:0/1535683248 conn(0x7fb2480d1930 msgr2=0x7fb2480d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 shutdown_connections 2024-09-21T06:28:28.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.483+0000 7fb24f000640 1 -- 172.21.15.37:0/1535683248 wait complete. 2024-09-21T06:28:28.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:28.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.734+0000 7f3605600640 1 -- 172.21.15.37:0/3185876790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 msgr2=0x7f360006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.734+0000 7f3605600640 1 --2- 172.21.15.37:0/3185876790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f360006ddf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00a970 tx=0x7f35ec02b8d0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.735+0000 7f3605600640 1 -- 172.21.15.37:0/3185876790 shutdown_connections 2024-09-21T06:28:28.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.735+0000 7f3605600640 1 --2- 172.21.15.37:0/3185876790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f360006ddf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.735+0000 7f3605600640 1 -- 172.21.15.37:0/3185876790 >> 172.21.15.37:0/3185876790 conn(0x7f36000cca40 msgr2=0x7f36000cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:28.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.735+0000 7f3605600640 1 -- 172.21.15.37:0/3185876790 shutdown_connections 2024-09-21T06:28:28.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.735+0000 7f3605600640 1 -- 172.21.15.37:0/3185876790 wait complete. 2024-09-21T06:28:28.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.737+0000 7f3605600640 1 Processor -- start 2024-09-21T06:28:28.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.737+0000 7f3605600640 1 -- start start 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360006e330 0x7f36000b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f36000b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3600071000 con 0x7f3600074d90 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3600070e80 con 0x7f360006e330 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.738+0000 7f3605600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f36000ce2a0 con 0x7f3600070030 2024-09-21T06:28:28.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40468/0 (socket says 172.21.15.37:40468) 2024-09-21T06:28:28.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 -- 172.21.15.37:0/1772908804 learned_addr learned my addr 172.21.15.37:0/1772908804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:28.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35fea00640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360006e330 0x7f36000b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35fe000640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f36000b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 -- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 msgr2=0x7f36000b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f36000b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 -- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360006e330 msgr2=0x7f36000b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360006e330 0x7f36000b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.739+0000 7f35ff400640 1 -- 172.21.15.37:0/1772908804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36001f5af0 con 0x7f3600074d90 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f35ff400640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35f800ca50 tx=0x7f35f800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8016020 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35f800aca0 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8012680 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36000b53f0 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.740+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36000b59b0 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.741+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36000ce750 con 0x7f3600074d90 2024-09-21T06:28:28.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.742+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35f8019020 con 0x7f3600074d90 2024-09-21T06:28:28.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.743+0000 7f35f6a00640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 0x7f35e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.743+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f35f809b4a0 con 0x7f3600074d90 2024-09-21T06:28:28.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.744+0000 7f35fea00640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 0x7f35e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.744+0000 7f35fea00640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 0x7f35e007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35f0000c00 tx=0x7f35f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.748+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f35f8020020 con 0x7f3600074d90 2024-09-21T06:28:28.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.983+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36000b5180 con 0x7f3600074d90 2024-09-21T06:28:28.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.984+0000 7f35f6a00640 1 -- 172.21.15.37:0/1772908804 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7f35f8062f40 con 0x7f3600074d90 2024-09-21T06:28:28.985 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:28.985 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:28.985 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 msgr2=0x7f35e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 0x7f35e007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35f0000c00 tx=0x7f35f0009040 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 msgr2=0x7f3600153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35f800ca50 tx=0x7f35f800cf10 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 shutdown_connections 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f35e0078b90 0x7f35e007b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600074d90 0x7f3600153980 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3600070030 0x7f36000b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 --2- 172.21.15.37:0/1772908804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360006e330 0x7f36000b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.986+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 >> 172.21.15.37:0/1772908804 conn(0x7f36000cca40 msgr2=0x7f36000730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.987+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 shutdown_connections 2024-09-21T06:28:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:28.987+0000 7f3605600640 1 -- 172.21.15.37:0/1772908804 wait complete. 2024-09-21T06:28:29.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.239+0000 7f0d2ae00640 1 -- 172.21.15.37:0/577629285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 msgr2=0x7f0d08004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/577629285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d08004c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c310 tx=0x7f0d08005830 comp rx=0 tx=0).stop 2024-09-21T06:28:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 -- 172.21.15.37:0/577629285 shutdown_connections 2024-09-21T06:28:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/577629285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d08004c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 -- 172.21.15.37:0/577629285 >> 172.21.15.37:0/577629285 conn(0x7f0d24072990 msgr2=0x7f0d24072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 -- 172.21.15.37:0/577629285 shutdown_connections 2024-09-21T06:28:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.240+0000 7f0d2ae00640 1 -- 172.21.15.37:0/577629285 wait complete. 2024-09-21T06:28:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.242+0000 7f0d2ae00640 1 Processor -- start 2024-09-21T06:28:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.242+0000 7f0d2ae00640 1 -- start start 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 0x7f0d240b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 0x7f0d240cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2415de80 con 0x7f0d08004800 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2415dd00 con 0x7f0d24157810 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d2ae00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2415e230 con 0x7f0d240b6a40 2024-09-21T06:28:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40486/0 (socket says 172.21.15.37:40486) 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 -- 172.21.15.37:0/4151051102 learned_addr learned my addr 172.21.15.37:0/4151051102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d1be00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 0x7f0d240b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d28e00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 0x7f0d240cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 -- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 msgr2=0x7f0d240cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 0x7f0d240cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 -- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 msgr2=0x7f0d240b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 0x7f0d240b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d23e00640 1 -- 172.21.15.37:0/4151051102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d240b6e50 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d1be00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 0x7f0d240b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.244+0000 7f0d28e00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 0x7f0d240cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d23e00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d14011a10 tx=0x7f0d14011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1401e070 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d14004460 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d241f9e50 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400fc00 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.245+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241f9b70 con 0x7f0d08004800 2024-09-21T06:28:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.246+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2415e790 con 0x7f0d08004800 2024-09-21T06:28:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.247+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d14021070 con 0x7f0d08004800 2024-09-21T06:28:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.248+0000 7f0d20c00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 0x7f0d0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.248+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0d1409bb50 con 0x7f0d08004800 2024-09-21T06:28:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.249+0000 7f0d1be00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 0x7f0d0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.249+0000 7f0d1be00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 0x7f0d0007b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c033000 tx=0x7f0d0c0019f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:29.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.253+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d240b6e50 con 0x7f0d08004800 2024-09-21T06:28:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.457+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d2415ec00 con 0x7f0d08004800 2024-09-21T06:28:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.459+0000 7f0d20c00640 1 -- 172.21.15.37:0/4151051102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18614 (secure 0 0 0) 0x7f0d14063560 con 0x7f0d08004800 2024-09-21T06:28:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:29.461 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:05.777737+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 msgr2=0x7f0d0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 0x7f0d0007b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c033000 tx=0x7f0d0c0019f0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 msgr2=0x7f0d241514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d14011a10 tx=0x7f0d14011ed0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 shutdown_connections 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d00078b90 0x7f0d0007b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d240b6a40 0x7f0d240cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d24157810 0x7f0d240b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 --2- 172.21.15.37:0/4151051102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d08004800 0x7f0d241514c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 >> 172.21.15.37:0/4151051102 conn(0x7f0d24072990 msgr2=0x7f0d24068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:29.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 shutdown_connections 2024-09-21T06:28:29.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.461+0000 7f0d2ae00640 1 -- 172.21.15.37:0/4151051102 wait complete. 2024-09-21T06:28:29.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 -- 172.21.15.37:0/3069447788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740ce860 msgr2=0x7f717415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 --2- 172.21.15.37:0/3069447788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740ce860 0x7f717415f5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7174072560 tx=0x7f716c02aa30 comp rx=0 tx=0).stop 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 -- 172.21.15.37:0/3069447788 shutdown_connections 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 --2- 172.21.15.37:0/3069447788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740ce860 0x7f717415f5a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 --2- 172.21.15.37:0/3069447788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740cefa0 0x7f71740ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 --2- 172.21.15.37:0/3069447788 >> v2:172.21.15.150:3300/0 conn(0x7f717415b9f0 0x7f71740cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 -- 172.21.15.37:0/3069447788 >> 172.21.15.37:0/3069447788 conn(0x7f71740cca40 msgr2=0x7f71740cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:29.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.713+0000 7f717b000640 1 -- 172.21.15.37:0/3069447788 shutdown_connections 2024-09-21T06:28:29.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.714+0000 7f717b000640 1 -- 172.21.15.37:0/3069447788 wait complete. 2024-09-21T06:28:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.716+0000 7f717b000640 1 Processor -- start 2024-09-21T06:28:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.716+0000 7f717b000640 1 -- start start 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740ce860 0x7f7174151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f717415b9f0 0x7f71740a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f717415fe70 con 0x7f71740cefa0 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f717415fcf0 con 0x7f717415b9f0 2024-09-21T06:28:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f717b000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f717406ddf0 con 0x7f71740ce860 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.717+0000 7f716be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40502/0 (socket says 172.21.15.37:40502) 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 -- 172.21.15.37:0/3574020525 learned_addr learned my addr 172.21.15.37:0/3574020525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f7173e00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740ce860 0x7f7174151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f7179000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f717415b9f0 0x7f71740a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 -- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740ce860 msgr2=0x7f7174151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740ce860 0x7f7174151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 -- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f717415b9f0 msgr2=0x7f71740a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f717415b9f0 0x7f71740a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 -- 172.21.15.37:0/3574020525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71740c1c10 con 0x7f71740cefa0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.718+0000 7f716be00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d6d0 tx=0x7f715c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.719+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c017070 con 0x7f71740cefa0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.719+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f715c009b10 con 0x7f71740cefa0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.719+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c010460 con 0x7f71740cefa0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.719+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71740c1770 con 0x7f71740cefa0 2024-09-21T06:28:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.719+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71740c2180 con 0x7f71740cefa0 2024-09-21T06:28:29.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.720+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7174070160 con 0x7f71740cefa0 2024-09-21T06:28:29.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.721+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f715c01a070 con 0x7f71740cefa0 2024-09-21T06:28:29.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.722+0000 7f7170c00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 0x7f715007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:29.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.722+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f715c09b260 con 0x7f71740cefa0 2024-09-21T06:28:29.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.723+0000 7f7173e00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 0x7f715007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:29.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.723+0000 7f7173e00640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 0x7f715007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f716400bad0 tx=0x7f716400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.727+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f715c062d90 con 0x7f71740cefa0 2024-09-21T06:28:29.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.963+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71740c1940 con 0x7f71740cefa0 2024-09-21T06:28:29.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.963+0000 7f7170c00640 1 -- 172.21.15.37:0/3574020525 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7f715c0677c0 con 0x7f71740cefa0 2024-09-21T06:28:29.965 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:29.965 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:29.965 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 msgr2=0x7f715007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 0x7f715007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f716400bad0 tx=0x7f716400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 msgr2=0x7f71740a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d6d0 tx=0x7f715c00db90 comp rx=0 tx=0).stop 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 shutdown_connections 2024-09-21T06:28:29.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7150078b90 0x7f715007b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71740cefa0 0x7f71740a03e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f717415b9f0 0x7f71740a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 --2- 172.21.15.37:0/3574020525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71740ce860 0x7f7174151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 >> 172.21.15.37:0/3574020525 conn(0x7f71740cca40 msgr2=0x7f71740aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 shutdown_connections 2024-09-21T06:28:29.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:29.966+0000 7f717b000640 1 -- 172.21.15.37:0/3574020525 wait complete. 2024-09-21T06:28:30.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:28:00.716481+0000', 'modified': '2024-09-21T06:28:18.477528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4399': {'gid': 4399, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/2075485628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 2075485628}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2075485628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4399, 'qdb_cluster': [4399]} 2024-09-21T06:28:30.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:28:30.394 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:28:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 -- 172.21.15.37:0/3797502656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0d2ba0 msgr2=0x7f38bc0b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 --2- 172.21.15.37:0/3797502656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0d2ba0 0x7f38bc0b9dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005fd0 tx=0x7f38ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 -- 172.21.15.37:0/3797502656 shutdown_connections 2024-09-21T06:28:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 --2- 172.21.15.37:0/3797502656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0cf7e0 0x7f38bc162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 --2- 172.21.15.37:0/3797502656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0d2ba0 0x7f38bc0b9dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 -- 172.21.15.37:0/3797502656 >> 172.21.15.37:0/3797502656 conn(0x7f38bc0d42e0 msgr2=0x7f38bc0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 -- 172.21.15.37:0/3797502656 shutdown_connections 2024-09-21T06:28:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.622+0000 7f38bb200640 1 -- 172.21.15.37:0/3797502656 wait complete. 2024-09-21T06:28:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.624+0000 7f38bb200640 1 Processor -- start 2024-09-21T06:28:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.624+0000 7f38bb200640 1 -- start start 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 0x7f38bc15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0d2ba0 0x7f38bc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc155730 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1555b0 con 0x7f38bc0cfbf0 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38bb200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc1558b0 con 0x7f38bc0d2ba0 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40532/0 (socket says 172.21.15.37:40532) 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38b9e00640 1 -- 172.21.15.37:0/1392975576 learned_addr learned my addr 172.21.15.37:0/1392975576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38ba800640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0d2ba0 0x7f38bc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.626+0000 7f38b1400640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 0x7f38bc15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 -- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0d2ba0 msgr2=0x7f38bc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0d2ba0 0x7f38bc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 -- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 msgr2=0x7f38bc15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 0x7f38bc15ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 -- 172.21.15.37:0/1392975576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1595e0 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b1400640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 0x7f38bc15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38ac034040 tx=0x7f38ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b2a00640 1 -- 172.21.15.37:0/1392975576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac03e030 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b2a00640 1 -- 172.21.15.37:0/1392975576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38ac003ca0 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38b2a00640 1 -- 172.21.15.37:0/1392975576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac035040 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc15a370 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.627+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc0cc480 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.630+0000 7f38b2a00640 1 -- 172.21.15.37:0/1392975576 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38ac041030 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.631+0000 7f38b2a00640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 0x7f389807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.631+0000 7f38b2a00640 1 -- 172.21.15.37:0/1392975576 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f38ac0bb540 con 0x7f38bc0cf7e0 2024-09-21T06:28:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.631+0000 7f38b1400640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 0x7f389807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.631+0000 7f38c7e59740 1 Processor -- start 2024-09-21T06:28:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.631+0000 7f38c7e59740 1 -- start start 2024-09-21T06:28:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.632+0000 7f38b1400640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 0x7f389807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0cb130 tx=0x7f38a4005ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a7473b0 0x55602ab410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602ab11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55602aa9a430 con 0x55602a770270 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55602aa8a5d0 con 0x55602a7473b0 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55602aa79a00 con 0x55602a74f590 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40544/0 (socket says 172.21.15.37:40544) 2024-09-21T06:28:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38b9e00640 1 -- 172.21.15.37:0/2709072427 learned_addr learned my addr 172.21.15.37:0/2709072427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38b1400640 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a7473b0 0x55602ab410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.633+0000 7f38ba800640 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602ab11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 -- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 msgr2=0x55602ab11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602ab11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 -- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a7473b0 msgr2=0x55602ab410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a7473b0 0x55602ab410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 -- 172.21.15.37:0/2709072427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602aa42520 con 0x55602a770270 2024-09-21T06:28:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38b9e00640 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38ac082ce0 tx=0x7f38ac082db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4902fcc722383803 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38aaa00640 1 -- 172.21.15.37:0/2709072427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac048040 con 0x55602a770270 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38aaa00640 1 -- 172.21.15.37:0/2709072427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38ac005310 con 0x55602a770270 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.634+0000 7f38aaa00640 1 -- 172.21.15.37:0/2709072427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac082af0 con 0x55602a770270 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 -- 172.21.15.37:0/2709072427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 msgr2=0x55602ab40b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38ac082ce0 tx=0x7f38ac082db0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 -- 172.21.15.37:0/2709072427 shutdown_connections 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602ab11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a7473b0 0x55602ab410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2709072427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab40b60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 -- 172.21.15.37:0/2709072427 >> 172.21.15.37:0/2709072427 conn(0x55602a7643c0 msgr2=0x55602ab16b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 -- 172.21.15.37:0/2709072427 shutdown_connections 2024-09-21T06:28:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.635+0000 7f38c7e59740 1 -- 172.21.15.37:0/2709072427 wait complete. 2024-09-21T06:28:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.636+0000 7f38c7e59740 1 -- start start 2024-09-21T06:28:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.636+0000 7f38c7e59740 1 Processor -- start 2024-09-21T06:28:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602ab3d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55602aa20330 con 0x55602a74f590 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55602aa7d810 con 0x55602a770270 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55602a7ea620 con 0x55602a7473b0 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38c7e59740 10 client.0 _ll_register_callbacks cb 0x55602aa25420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38b1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38b1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40548/0 (socket says 172.21.15.37:40548) 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38b1400640 1 -- 172.21.15.37:0/3262851650 learned_addr learned my addr 172.21.15.37:0/3262851650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38ba800640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602ab3d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.638+0000 7f38b9e00640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 -- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 msgr2=0x55602ab371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 -- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 msgr2=0x55602ab3d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602ab3d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 -- 172.21.15.37:0/3262851650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602a7db850 con 0x55602a74f590 2024-09-21T06:28:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38b1400640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38a40116f0 tx=0x7f38a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a404420398c40b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f389f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:28:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a401f900 con 0x55602a74f590 2024-09-21T06:28:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38ac005310 con 0x55602a74f590 2024-09-21T06:28:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.639+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f38a401faa0 con 0x55602a74f590 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f38a4030050 con 0x55602a74f590 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 10 client.4761 fetch_fsmap learned FSMap version 8 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55602a8526b0 con 0x55602a74f590 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38ac005310 con 0x55602a74f590 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f38a4028470 con 0x55602a74f590 2024-09-21T06:28:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 10 client.4761 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 10 client.4761 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 20 client.4761 populate_metadata read hostname 'smithi037' 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602a72ee40 0x55602ab6b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 4 client.4761 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 --> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55602aa8c520 con 0x55602a72ee40 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.640+0000 7f38b9e00640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602a72ee40 0x55602ab6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.641+0000 7f38b9e00640 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602a72ee40 0x55602ab6b180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4b373e9f055111 server_cookie=4b8e7a34bfb984c6 in_seq=0 out_seq=0 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.641+0000 7f389f400640 10 client.4761 ms_handle_connect on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f389f400640 1 -- 172.21.15.37:0/3262851650 <== mds.0 v2:172.21.15.150:6835/2075485628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55602aa8c520 con 0x55602a72ee40 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f389f400640 10 client.4761 handle_command_reply: tid=0 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 client.4761 shutdown 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 msgr2=0x55602ab3d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38a40116f0 tx=0x7f38a40046c0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 shutdown_connections 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602a72ee40 0x55602ab6b180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a74f590 0x55602ab3d180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602ab3d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 --2- 172.21.15.37:0/3262851650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 >> 172.21.15.37:0/3262851650 conn(0x55602a7643c0 msgr2=0x55602ab12430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f389f400640 0 client.4761 ms_handle_reset on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.642+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 shutdown_connections 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.643+0000 7f38c7e59740 1 -- 172.21.15.37:0/3262851650 wait complete. 2024-09-21T06:28:30.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.643+0000 7f38c7e59740 20 client.4761 trim_cache size 0 max 16384 2024-09-21T06:28:30.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.662+0000 7f38c7e59740 1 Processor -- start 2024-09-21T06:28:30.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.662+0000 7f38c7e59740 1 -- start start 2024-09-21T06:28:30.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 0x55602ab6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc2023c0 con 0x55602a770270 2024-09-21T06:28:30.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc201720 con 0x55602a74f590 2024-09-21T06:28:30.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.663+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc201d70 con 0x55602a7473b0 2024-09-21T06:28:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40550/0 (socket says 172.21.15.37:40550) 2024-09-21T06:28:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 -- 172.21.15.37:0/2160432591 learned_addr learned my addr 172.21.15.37:0/2160432591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38b1400640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 0x55602ab6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38b9e00640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 -- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 msgr2=0x55602ab3d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab3d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 -- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 msgr2=0x55602ab6a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 0x55602ab6a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38ba800640 1 -- 172.21.15.37:0/2160432591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602ab5cbb0 con 0x55602a770270 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38b1400640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 0x55602ab6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.664+0000 7f38b9e00640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38ba800640 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001fb0 tx=0x7f38b4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ef0331e3bd22c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38aa000640 1 -- 172.21.15.37:0/2160432591 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4003970 con 0x55602a770270 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38aa000640 1 -- 172.21.15.37:0/2160432591 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b4003b10 con 0x55602a770270 2024-09-21T06:28:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38aa000640 1 -- 172.21.15.37:0/2160432591 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4005110 con 0x55602a770270 2024-09-21T06:28:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38c7e59740 1 -- 172.21.15.37:0/2160432591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 msgr2=0x55602ab6a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.665+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001fb0 tx=0x7f38b4001630 comp rx=0 tx=0).stop 2024-09-21T06:28:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 -- 172.21.15.37:0/2160432591 shutdown_connections 2024-09-21T06:28:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a770270 0x55602ab6a600 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a74f590 0x55602ab6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 --2- 172.21.15.37:0/2160432591 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a7473b0 0x55602ab3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 -- 172.21.15.37:0/2160432591 >> 172.21.15.37:0/2160432591 conn(0x55602a7643c0 msgr2=0x55602ab3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 -- 172.21.15.37:0/2160432591 shutdown_connections 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.666+0000 7f38c7e59740 1 -- 172.21.15.37:0/2160432591 wait complete. 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.667+0000 7f38c7e59740 1 -- start start 2024-09-21T06:28:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.667+0000 7f38c7e59740 1 Processor -- start 2024-09-21T06:28:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602abdeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602abdf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55602ab55f30 con 0x55602a7473b0 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55602ab3a140 con 0x55602a770270 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55602ab6eb90 con 0x55602a74f590 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.668+0000 7f38c7e59740 10 client.0 _ll_register_callbacks cb 0x7f38a400fdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40564/0 (socket says 172.21.15.37:40564) 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 -- 172.21.15.37:0/1581384363 learned_addr learned my addr 172.21.15.37:0/1581384363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b1400640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602abdeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38ba800640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602abdf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 -- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 msgr2=0x55602abdeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602abdeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 -- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 msgr2=0x55602abdf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:28:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602abdf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 -- 172.21.15.37:0/1581384363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602ab5bf30 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38ba800640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602abdf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b1400640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602abdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f38b9e00640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38ac082ce0 tx=0x7f38ac02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da85725cf41c1685 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f389f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.669+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac048040 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f38ac0bb0b0 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4003b10 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f38ac005310 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 10 client.4767 fetch_fsmap learned FSMap version 8 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55602ab6bef0 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b4003b10 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f38ac08c430 con 0x55602a7473b0 2024-09-21T06:28:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 10 client.4767 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 10 client.4767 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 20 client.4767 populate_metadata read hostname 'smithi037' 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602abdb190 0x55602abdb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 4 client.4767 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 --> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] -- command(tid 0: {"prefix": "status"}) -- 0x55602ab17f90 con 0x55602abdb190 2024-09-21T06:28:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.670+0000 7f38b1400640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602abdb190 0x55602abdb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.671+0000 7f38b1400640 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602abdb190 0x55602abdb5a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43313cdb1e60f3d3 server_cookie=beefd95bed91b9b in_seq=0 out_seq=0 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.671+0000 7f389f400640 10 client.4767 ms_handle_connect on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.671+0000 7f389f400640 1 -- 172.21.15.37:0/1581384363 <== mds.0 v2:172.21.15.150:6835/2075485628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55602ab17f90 con 0x55602abdb190 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.671+0000 7f389f400640 10 client.4767 handle_command_reply: tid=0 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 client.4767 shutdown 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 msgr2=0x55602ab6b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38ac082ce0 tx=0x7f38ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 shutdown_connections 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:6835/2075485628,v1:172.21.15.150:6837/2075485628] conn(0x55602abdb190 0x55602abdb5a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602a770270 0x55602abdf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55602a74f590 0x55602abdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 --2- 172.21.15.37:0/1581384363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55602a7473b0 0x55602ab6b8f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f389f400640 0 client.4767 ms_handle_reset on v2:172.21.15.150:6835/2075485628 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 >> 172.21.15.37:0/1581384363 conn(0x55602a7643c0 msgr2=0x55602ab3ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 shutdown_connections 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 1 -- 172.21.15.37:0/1581384363 wait complete. 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.672+0000 7f38c7e59740 20 client.4767 trim_cache size 0 max 16384 2024-09-21T06:28:30.676 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4399, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 29.446640036000002, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 8, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 27, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 27, 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 35.866307382000002 2024-09-21T06:28:30.677 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:28:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 msgr2=0x7f389807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 0x7f389807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0cb130 tx=0x7f38a4005ef0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 msgr2=0x7f38bc1519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38ac034040 tx=0x7f38ac0038a0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 shutdown_connections 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3898078b90 0x7f389807b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38bc0d2ba0 0x7f38bc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38bc0cfbf0 0x7f38bc15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 --2- 172.21.15.37:0/1392975576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38bc0cf7e0 0x7f38bc1519c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 >> 172.21.15.37:0/1392975576 conn(0x7f38bc0d42e0 msgr2=0x7f38bc160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 shutdown_connections 2024-09-21T06:28:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:30.673+0000 7f38bb200640 1 -- 172.21.15.37:0/1392975576 wait complete. 2024-09-21T06:28:30.696 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T06:28:30.708 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T06:28:30.708 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T06:28:30.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:30.708 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:28:30.743 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T06:28:30.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:30.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:28:30.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:30.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-21T06:28:31.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.149 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:28:31.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-21T06:28:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-09-21T06:28:31.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:28:31.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.287 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:28:31.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-09-21T06:28:31.429 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-09-21T06:28:31.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:28:31.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.433 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:28:31.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:31.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-09-21T06:28:31.568 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-09-21T06:28:31.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:28:31.571 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T06:28:31.571 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:28:31.571 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:28:31.571 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 -- 172.21.15.37:0/487108718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898073dd0 msgr2=0x7f989806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 --2- 172.21.15.37:0/487108718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898073dd0 0x7f989806ddf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9884009830 tx=0x7f988402a5b0 comp rx=0 tx=0).stop 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 -- 172.21.15.37:0/487108718 shutdown_connections 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 --2- 172.21.15.37:0/487108718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898073dd0 0x7f989806ddf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 -- 172.21.15.37:0/487108718 >> 172.21.15.37:0/487108718 conn(0x7f98980cca40 msgr2=0x7f98980cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.798+0000 7f989ea00640 1 -- 172.21.15.37:0/487108718 shutdown_connections 2024-09-21T06:28:31.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.799+0000 7f989ea00640 1 -- 172.21.15.37:0/487108718 wait complete. 2024-09-21T06:28:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.801+0000 7f989ea00640 1 Processor -- start 2024-09-21T06:28:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.801+0000 7f989ea00640 1 -- start start 2024-09-21T06:28:31.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.802+0000 7f989ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f989806e330 0x7f98980b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9898073dd0 0x7f98980b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f989806f670 con 0x7f9898074cb0 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f989806f4f0 con 0x7f9898073dd0 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f989806f7f0 con 0x7f989806e330 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40576/0 (socket says 172.21.15.37:40576) 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 -- 172.21.15.37:0/2116410416 learned_addr learned my addr 172.21.15.37:0/2116410416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f9897400640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9898073dd0 0x7f98980b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f9897e00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f989806e330 0x7f98980b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 -- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f989806e330 msgr2=0x7f98980b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f989806e330 0x7f98980b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:31.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 -- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9898073dd0 msgr2=0x7f98980b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:31.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9898073dd0 0x7f98980b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:31.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.803+0000 7f989ca00640 1 -- 172.21.15.37:0/2116410416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98980b18b0 con 0x7f9898074cb0 2024-09-21T06:28:31.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f989ca00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f989000cb10 tx=0x7f989000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890016020 con 0x7f9898074cb0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f989000ad00 con 0x7f9898074cb0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890012680 con 0x7f9898074cb0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98981e6910 con 0x7f9898074cb0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.804+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981e6ed0 con 0x7f9898074cb0 2024-09-21T06:28:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.806+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98980ca290 con 0x7f9898074cb0 2024-09-21T06:28:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.807+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9890019020 con 0x7f9898074cb0 2024-09-21T06:28:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.808+0000 7f988fe00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 0x7f987807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.808+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f989009b1d0 con 0x7f9898074cb0 2024-09-21T06:28:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.808+0000 7f9897e00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 0x7f987807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.809+0000 7f9897e00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 0x7f987807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f989815a730 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:31.812+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9890020030 con 0x7f9898074cb0 2024-09-21T06:28:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.052+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98981e66a0 con 0x7f9898074cb0 2024-09-21T06:28:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.053+0000 7f988fe00640 1 -- 172.21.15.37:0/2116410416 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f9890062c90 con 0x7f9898074cb0 2024-09-21T06:28:32.054 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:28:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 msgr2=0x7f987807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 0x7f987807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f989815a730 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 msgr2=0x7f98981f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f989000cb10 tx=0x7f989000a060 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 shutdown_connections 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9878078b90 0x7f987807b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9898074cb0 0x7f98981f9200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9898073dd0 0x7f98980b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 --2- 172.21.15.37:0/2116410416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f989806e330 0x7f98980b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 >> 172.21.15.37:0/2116410416 conn(0x7f98980cca40 msgr2=0x7f989815f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 shutdown_connections 2024-09-21T06:28:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32.055+0000 7f989ea00640 1 -- 172.21.15.37:0/2116410416 wait complete. 2024-09-21T06:28:32.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:28:32.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:28:32.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:28:32.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:28:32.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:28:32.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:28:32.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:28:32.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:28:32.079 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:28:32.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:32.079 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:28:32.114 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:28:32.115 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42679sec preferred_lft 42679sec 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:32.116 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:28:32.116 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:28:32.116 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:28:32.117 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:32.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:32.343 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:28:32.397 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:28:32.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:32.400 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:28:32.436 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:28:32.436 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:28:32.436 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:28:32.436 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:28:32.436 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:28:32.436 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:32.436 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:32.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:28:32.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:28:32.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:28:32.437 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:32.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:32.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:32.634 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:28:32.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:32.670 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-21T06:28:32.744 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:32.745 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:32.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:28:32.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T06:28:32.745 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:32.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:32.922 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T06:28:32.922 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T06:28:32.923 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:33.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:33.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:33.141 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:33.141 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:33.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:28:33.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:28:33.141 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:33.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:33.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:33.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:33.311 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:28:33.347 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:28:33.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:33.347 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:28:33.421 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:28:33.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:33.421 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:28:33.620 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:28:33.620 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:28:33.620 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:34.247 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:28:34.248 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:28:34.248 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:34.248 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:34.248 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:34.285 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:28:34.285 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:28:34.285 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:34.285 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:34.286 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:34.286 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:28:34.286 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:28:34.286 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:28:34.287 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:28:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:28:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:28:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:28:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:28:34.291 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:28:34.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:34.291 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:28:34.348 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:34.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:34.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:34.439 INFO:teuthology.orchestra.run.smithi037.stdout:client4779 2024-09-21T06:28:34.440 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:34.440 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:34.475 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:34.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:34.605 INFO:teuthology.orchestra.run.smithi037.stdout:client4779 2024-09-21T06:28:34.605 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:34.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:34.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:34.695 INFO:teuthology.orchestra.run.smithi037.stdout:client4779 2024-09-21T06:28:34.695 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/13e9e1b2-9d52-4cf2-a477-95a0639530e0.client4779/status') 2024-09-21T06:28:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:34.795 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.4779 (3)192.168.0.1:0/746206315 2024-09-21T06:28:34.795 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-09-21T06:28:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-09-21T06:28:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-09-21T06:28:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:68 bytes copied, 5.7151e-05 s, 1.2 MB/s 2024-09-21T06:28:34.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:34.806 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T06:28:34.806 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:28:34.806 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T06:28:34.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:28:35.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 -- 172.21.15.37:0/321135989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781406e500 msgr2=0x7f781406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 --2- 172.21.15.37:0/321135989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781406e500 0x7f781406ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f780c005fd0 tx=0x7f780c02ab30 comp rx=0 tx=0).stop 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 -- 172.21.15.37:0/321135989 shutdown_connections 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 --2- 172.21.15.37:0/321135989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781406e500 0x7f781406ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 -- 172.21.15.37:0/321135989 >> 172.21.15.37:0/321135989 conn(0x7f78140cca40 msgr2=0x7f78140cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 -- 172.21.15.37:0/321135989 shutdown_connections 2024-09-21T06:28:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.032+0000 7f781b200640 1 -- 172.21.15.37:0/321135989 wait complete. 2024-09-21T06:28:35.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.035+0000 7f781b200640 1 Processor -- start 2024-09-21T06:28:35.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.035+0000 7f781b200640 1 -- start start 2024-09-21T06:28:35.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7814073dd0 0x7f78141578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:35.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:35.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78141584f0 0x7f78140a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:35.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f781406f6b0 con 0x7f7814074cb0 2024-09-21T06:28:35.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f781406f530 con 0x7f7814073dd0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f781b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f781406f830 con 0x7f78141584f0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.036+0000 7f780be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46892/0 (socket says 172.21.15.37:46892) 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 -- 172.21.15.37:0/1843166189 learned_addr learned my addr 172.21.15.37:0/1843166189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f7819200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78141584f0 0x7f78140a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f7813e00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7814073dd0 0x7f78141578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 -- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78141584f0 msgr2=0x7f78140a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78141584f0 0x7f78140a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 -- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7814073dd0 msgr2=0x7f78141578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7814073dd0 0x7f78141578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 -- 172.21.15.37:0/1843166189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78141e8870 con 0x7f7814074cb0 2024-09-21T06:28:35.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f780be00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c3a0 tx=0x7f77fc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.037+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc017070 con 0x7f7814074cb0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.038+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77fc0053c0 con 0x7f7814074cb0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.038+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc003850 con 0x7f7814074cb0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.038+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78141e8b50 con 0x7f7814074cb0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.038+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78140a0520 con 0x7f7814074cb0 2024-09-21T06:28:35.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.039+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781406ede0 con 0x7f7814074cb0 2024-09-21T06:28:35.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.040+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77fc01a070 con 0x7f7814074cb0 2024-09-21T06:28:35.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.041+0000 7f7810c00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 0x7f77f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:35.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.041+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f77fc09a3e0 con 0x7f7814074cb0 2024-09-21T06:28:35.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.042+0000 7f7813e00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 0x7f77f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:35.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.042+0000 7f7813e00640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 0x7f77f007b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f781415a730 tx=0x7f7804009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:35.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.046+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77fc061e80 con 0x7f7814074cb0 2024-09-21T06:28:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.286+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f781415c020 con 0x7f7814074cb0 2024-09-21T06:28:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.286+0000 7f7810c00640 1 -- 172.21.15.37:0/1843166189 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f77fc0668b0 con 0x7f7814074cb0 2024-09-21T06:28:35.287 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:28:35.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 msgr2=0x7f77f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:35.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 0x7f77f007b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f781415a730 tx=0x7f7804009040 comp rx=0 tx=0).stop 2024-09-21T06:28:35.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 msgr2=0x7f78141e5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:35.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c3a0 tx=0x7f77fc00c860 comp rx=0 tx=0).stop 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 shutdown_connections 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f77f0078b90 0x7f77f007b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7814074cb0 0x7f78141e5040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78141584f0 0x7f78140a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 --2- 172.21.15.37:0/1843166189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7814073dd0 0x7f78141578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 >> 172.21.15.37:0/1843166189 conn(0x7f78140cca40 msgr2=0x7f781406dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 shutdown_connections 2024-09-21T06:28:35.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35.289+0000 7f781b200640 1 -- 172.21.15.37:0/1843166189 wait complete. 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:28:35.313 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T06:28:35.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.313 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:28:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42676sec preferred_lft 42676sec 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:35.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.352 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:28:35.443 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:28:35.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.446 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:28:35.482 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:28:35.482 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:28:35.483 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:28:35.483 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:28:35.483 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:28:35.483 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:35.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:35.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.682 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:28:35.717 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:28:35.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.718 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-21T06:28:35.792 INFO:teuthology.orchestra.run.smithi037.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T06:28:35.793 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:35.793 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:35.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:28:35.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T06:28:35.794 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:35.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:35.972 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T06:28:36.028 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:36.028 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:36.028 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:36.028 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 7a:bf:cd:75:d9:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::78bf:cdff:fe75:d92d/64 scope link 2024-09-21T06:28:36.029 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:36.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 7a:bf:cd:75:d9:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::78bf:cdff:fe75:d92d/64 scope link 2024-09-21T06:28:36.088 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:36.090 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T06:28:36.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T06:28:36.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T06:28:36.091 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:36.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:36.309 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:36.309 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:28:36.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.1 up 2024-09-21T06:28:36.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T06:28:36.309 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:28:36.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:36.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:36.476 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T06:28:36.511 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T06:28:36.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:36.511 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T06:28:36.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:28:36.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:36.587 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T06:28:36.784 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T06:28:36.784 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T06:28:36.784 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:36.805 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T06:28:36.805 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T06:28:36.805 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:36.805 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:36.805 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:36.835 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T06:28:36.835 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T06:28:36.835 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:28:36.835 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:36.835 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:36.836 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:28:36.836 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:28:36.836 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:28:36.837 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T06:28:36.837 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:28:36.837 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:28:36.837 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:28:36.837 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:28:36.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:28:36.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:36.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T06:28:36.895 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:36.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:36.986 INFO:teuthology.orchestra.run.smithi037.stdout:client4673 2024-09-21T06:28:36.986 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:36.986 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:37.022 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:37.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:37.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:37.119 INFO:teuthology.orchestra.run.smithi037.stdout:client4673 2024-09-21T06:28:37.119 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:37.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:client4673 2024-09-21T06:28:37.211 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/13e9e1b2-9d52-4cf2-a477-95a0639530e0.client4673/status') 2024-09-21T06:28:37.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:37.316 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.4673 (3)172.21.15.37:0/813615054 2024-09-21T06:28:37.317 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-09-21T06:28:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-09-21T06:28:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-09-21T06:28:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:69 bytes copied, 5.7293e-05 s, 1.2 MB/s 2024-09-21T06:28:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:37.327 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T06:28:37.327 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:28:37.327 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T06:28:37.328 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 -- 172.21.15.37:0/3929314071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928074d90 msgr2=0x7fa92806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 --2- 172.21.15.37:0/3929314071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928074d90 0x7fa92806f710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a940 tx=0x7fa91c02b8f0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 -- 172.21.15.37:0/3929314071 shutdown_connections 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 --2- 172.21.15.37:0/3929314071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928074d90 0x7fa92806f710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 -- 172.21.15.37:0/3929314071 >> 172.21.15.37:0/3929314071 conn(0x7fa9280cca40 msgr2=0x7fa9280cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 -- 172.21.15.37:0/3929314071 shutdown_connections 2024-09-21T06:28:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.553+0000 7fa930000640 1 -- 172.21.15.37:0/3929314071 wait complete. 2024-09-21T06:28:37.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.556+0000 7fa930000640 1 Processor -- start 2024-09-21T06:28:37.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.556+0000 7fa930000640 1 -- start start 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 0x7fa9281582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa92809b3f0 0x7fa9280b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa928071000 con 0x7fa928070030 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa928070e80 con 0x7fa928074d90 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa930000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280ce2a0 con 0x7fa92809b3f0 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa92d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa92d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46920/0 (socket says 172.21.15.37:46920) 2024-09-21T06:28:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.557+0000 7fa92d600640 1 -- 172.21.15.37:0/3293697709 learned_addr learned my addr 172.21.15.37:0/3293697709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92cc00640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 0x7fa9281582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92e000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa92809b3f0 0x7fa9280b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 -- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa92809b3f0 msgr2=0x7fa9280b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa92809b3f0 0x7fa9280b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 -- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 msgr2=0x7fa9281582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 0x7fa9281582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 -- 172.21.15.37:0/3293697709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92809d250 con 0x7fa928070030 2024-09-21T06:28:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92cc00640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 0x7fa9281582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa92d600640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00aa60 tx=0x7fa91c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.558+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c005220 con 0x7fa928070030 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.559+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa91c02e070 con 0x7fa928070030 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.559+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92809d530 con 0x7fa928070030 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.559+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0047a0 con 0x7fa928070030 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.559+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9280b2ff0 con 0x7fa928070030 2024-09-21T06:28:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.560+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9280b2e10 con 0x7fa928070030 2024-09-21T06:28:37.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.561+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa91c004940 con 0x7fa928070030 2024-09-21T06:28:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.562+0000 7fa925600640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.562+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7fa91c0bca50 con 0x7fa928070030 2024-09-21T06:28:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.562+0000 7fa92cc00640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:37.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.563+0000 7fa92cc00640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 0x7fa90807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa92809b2b0 tx=0x7fa914009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:37.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.567+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9280b2e10 con 0x7fa928070030 2024-09-21T06:28:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.799+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa92806f710 con 0x7fa928070030 2024-09-21T06:28:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.800+0000 7fa925600640 1 -- 172.21.15.37:0/3293697709 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fa91c084390 con 0x7fa928070030 2024-09-21T06:28:37.801 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:28:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 msgr2=0x7fa90807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 0x7fa90807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa92809b2b0 tx=0x7fa914009040 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 msgr2=0x7fa9280b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00aa60 tx=0x7fa91c004270 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 shutdown_connections 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa92809b3f0 0x7fa9280b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa928074d90 0x7fa9281582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 --2- 172.21.15.37:0/3293697709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa928070030 0x7fa9280b3fe0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 >> 172.21.15.37:0/3293697709 conn(0x7fa9280cca40 msgr2=0x7fa9280b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 shutdown_connections 2024-09-21T06:28:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:37.802+0000 7fa930000640 1 -- 172.21.15.37:0/3293697709 wait complete. 2024-09-21T06:28:37.827 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:28:37.828 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T06:28:37.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:37.828 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-09-21T06:28:37.868 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:37.868 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:28:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:37.870 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:28:37.870 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42647sec preferred_lft 42647sec 2024-09-21T06:28:37.870 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link noprefixroute 2024-09-21T06:28:37.870 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:37.870 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:28:37.870 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:37.870 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:37.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:28:37.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-09-21T06:28:37.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-09-21T06:28:37.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:28:37.871 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:37.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:38.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:38.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:38.294 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:28:38.350 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-09-21T06:28:38.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:38.353 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-09-21T06:28:38.402 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-09-21T06:28:38.402 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:28:38.402 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:28:38.402 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:28:38.402 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:28:38.403 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:38.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:38.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:38.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:38.982 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:28:39.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:39.017 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-09-21T06:28:39.093 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:39.093 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:39.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T06:28:39.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T06:28:39.094 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:39.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:39.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:39.272 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T06:28:39.272 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T06:28:39.273 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:39.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:39.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:39.512 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:39.512 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:39.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-09-21T06:28:39.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:28:39.512 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:39.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:39.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:39.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:39.679 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T06:28:39.714 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T06:28:39.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:39.715 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T06:28:39.789 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:28:39.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:39.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T06:28:40.003 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T06:28:40.003 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T06:28:40.003 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:40.607 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T06:28:40.608 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T06:28:40.608 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:40.608 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:40.608 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:40.642 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T06:28:40.642 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:28:40.643 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:28:40.644 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T06:28:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:28:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:28:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:28:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:28:40.647 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:28:40.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:40.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T06:28:40.704 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:40.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:40.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:40.795 INFO:teuthology.orchestra.run.smithi150.stdout:client4807 2024-09-21T06:28:40.795 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:40.795 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:40.831 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:40.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:40.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:40.959 INFO:teuthology.orchestra.run.smithi150.stdout:client4807 2024-09-21T06:28:40.959 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:41.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:41.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:41.049 INFO:teuthology.orchestra.run.smithi150.stdout:client4807 2024-09-21T06:28:41.049 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/13e9e1b2-9d52-4cf2-a477-95a0639530e0.client4807/status') 2024-09-21T06:28:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:41.149 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4807 (3)192.168.0.1:0/3694485366 2024-09-21T06:28:41.149 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-09-21T06:28:41.149 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-09-21T06:28:41.149 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-09-21T06:28:41.149 INFO:teuthology.orchestra.run.smithi150.stderr:69 bytes copied, 5.5772e-05 s, 1.2 MB/s 2024-09-21T06:28:41.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:41.160 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T06:28:41.160 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:28:41.160 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T06:28:41.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:28:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 -- 172.21.15.37:0/217960505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be406e500 msgr2=0x7f3be406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 --2- 172.21.15.37:0/217960505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be406e500 0x7f3be406ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005fd0 tx=0x7f3bdc02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 -- 172.21.15.37:0/217960505 shutdown_connections 2024-09-21T06:28:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 --2- 172.21.15.37:0/217960505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be406e500 0x7f3be406ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 -- 172.21.15.37:0/217960505 >> 172.21.15.37:0/217960505 conn(0x7f3be40cca40 msgr2=0x7f3be40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 -- 172.21.15.37:0/217960505 shutdown_connections 2024-09-21T06:28:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.402+0000 7f3beb000640 1 -- 172.21.15.37:0/217960505 wait complete. 2024-09-21T06:28:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.404+0000 7f3beb000640 1 Processor -- start 2024-09-21T06:28:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.404+0000 7f3beb000640 1 -- start start 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4074cb0 0x7f3be41f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3be4156880 0x7f3be41f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3be406f6b0 con 0x7f3be4073dd0 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3be406f530 con 0x7f3be4074cb0 2024-09-21T06:28:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3beb000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3be406f830 con 0x7f3be4156880 2024-09-21T06:28:41.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:41.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46934/0 (socket says 172.21.15.37:46934) 2024-09-21T06:28:41.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 -- 172.21.15.37:0/3878074701 learned_addr learned my addr 172.21.15.37:0/3878074701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3400640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4074cb0 0x7f3be41f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be9000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3be4156880 0x7f3be41f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 -- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3be4156880 msgr2=0x7f3be41f6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3be4156880 0x7f3be41f6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 -- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4074cb0 msgr2=0x7f3be41f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4074cb0 0x7f3be41f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.406+0000 7f3be3e00640 1 -- 172.21.15.37:0/3878074701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41e78f0 con 0x7f3be4073dd0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3be3e00640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c400 tx=0x7f3bd400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4017070 con 0x7f3be4073dd0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd40053c0 con 0x7f3be4073dd0 2024-09-21T06:28:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4003850 con 0x7f3be4073dd0 2024-09-21T06:28:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be4157ed0 con 0x7f3be4073dd0 2024-09-21T06:28:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.407+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be409cca0 con 0x7f3be4073dd0 2024-09-21T06:28:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.408+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be409ba20 con 0x7f3be4073dd0 2024-09-21T06:28:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.410+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3bd401a070 con 0x7f3be4073dd0 2024-09-21T06:28:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.410+0000 7f3bdbe00640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 0x7f3bc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.411+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f3bd409a580 con 0x7f3be4073dd0 2024-09-21T06:28:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.411+0000 7f3be3400640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 0x7f3bc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.412+0000 7f3be3400640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 0x7f3bc407b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00046b0 tx=0x7f3bd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:41.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.415+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3be409ba20 con 0x7f3be4073dd0 2024-09-21T06:28:41.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.651+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3be415c2f0 con 0x7f3be4073dd0 2024-09-21T06:28:41.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.651+0000 7f3bdbe00640 1 -- 172.21.15.37:0/3878074701 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f3bd4062590 con 0x7f3be4073dd0 2024-09-21T06:28:41.653 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:28:41.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 msgr2=0x7f3bc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:41.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 0x7f3bc407b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00046b0 tx=0x7f3bd0009040 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 msgr2=0x7f3be40b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c400 tx=0x7f3bd400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 shutdown_connections 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3bc4078b90 0x7f3bc407b0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3be4156880 0x7f3be41f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4074cb0 0x7f3be41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 --2- 172.21.15.37:0/3878074701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be4073dd0 0x7f3be40b5700 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 >> 172.21.15.37:0/3878074701 conn(0x7f3be40cca40 msgr2=0x7f3be40a0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 shutdown_connections 2024-09-21T06:28:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:41.654+0000 7f3beb000640 1 -- 172.21.15.37:0/3878074701 wait complete. 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:28:41.678 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T06:28:41.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:41.679 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-09-21T06:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42644sec preferred_lft 42644sec 2024-09-21T06:28:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link noprefixroute 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether ba:fa:4a:56:2c:ba brd ff:ff:ff:ff:ff:ff 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::fc86:91ff:feab:1207/64 scope link 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether ba:fa:4a:56:2c:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::b8fa:4aff:fe56:2cba/64 scope link 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:41.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:41.718 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:28:41.809 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-09-21T06:28:41.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:41.812 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-09-21T06:28:41.848 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-09-21T06:28:41.848 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:28:41.849 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:28:41.849 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:28:41.849 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:28:41.849 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:41.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:42.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:42.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:42.046 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:28:42.081 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T06:28:42.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:42.082 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-09-21T06:28:42.159 INFO:teuthology.orchestra.run.smithi150.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T06:28:42.160 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:42.160 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:42.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T06:28:42.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T06:28:42.160 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:42.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:42.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:42.338 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T06:28:42.394 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.395 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether e2:97:dd:32:8f:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T06:28:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T06:28:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::e097:ddff:fe32:8feb/64 scope link 2024-09-21T06:28:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:42.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.454 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:28:42.455 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether e2:97:dd:32:8f:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T06:28:42.455 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T06:28:42.455 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.455 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::e097:ddff:fe32:8feb/64 scope link 2024-09-21T06:28:42.455 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:28:42.457 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T06:28:42.457 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:42.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:42.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:42.674 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:28:42.674 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-09-21T06:28:42.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2024-09-21T06:28:42.675 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T06:28:42.675 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-09-21T06:28:42.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:42.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:42.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:42.842 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T06:28:42.876 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T06:28:42.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:28:42.877 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T06:28:42.952 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:28:42.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:42.952 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T06:28:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T06:28:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T06:28:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T06:28:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T06:28:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T06:28:43.206 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:28:43.207 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:28:43.207 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:28:43.207 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:28:43.208 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:28:43.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:43.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T06:28:43.265 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:43.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:43.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:43.354 INFO:teuthology.orchestra.run.smithi150.stdout:client4819 2024-09-21T06:28:43.354 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:43.354 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:43.390 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:43.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:43.518 INFO:teuthology.orchestra.run.smithi150.stdout:client4819 2024-09-21T06:28:43.518 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:28:43.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:28:43.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:43.607 INFO:teuthology.orchestra.run.smithi150.stdout:client4819 2024-09-21T06:28:43.607 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/13e9e1b2-9d52-4cf2-a477-95a0639530e0.client4819/status') 2024-09-21T06:28:43.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:28:43.705 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4819 (3)192.168.0.2:0/228668091 2024-09-21T06:28:43.706 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-09-21T06:28:43.706 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-09-21T06:28:43.706 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-09-21T06:28:43.706 INFO:teuthology.orchestra.run.smithi150.stderr:68 bytes copied, 6.9684e-05 s, 976 kB/s 2024-09-21T06:28:43.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T06:28:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:28:43.716 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T06:28:45.007 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:28:45.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.247+0000 7f38aec00640 1 -- 172.21.15.37:0/2664670807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 msgr2=0x7f38a80ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.247+0000 7f38aec00640 1 --2- 172.21.15.37:0/2664670807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80ae630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38a80b6290 tx=0x7f389402a540 comp rx=0 tx=0).stop 2024-09-21T06:28:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 -- 172.21.15.37:0/2664670807 shutdown_connections 2024-09-21T06:28:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 --2- 172.21.15.37:0/2664670807 >> v2:172.21.15.150:3300/0 conn(0x7f38a80aeb70 0x7f38a80d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 --2- 172.21.15.37:0/2664670807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80ae630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 -- 172.21.15.37:0/2664670807 >> 172.21.15.37:0/2664670807 conn(0x7f38a80b5140 msgr2=0x7f38a80b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 -- 172.21.15.37:0/2664670807 shutdown_connections 2024-09-21T06:28:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.248+0000 7f38aec00640 1 -- 172.21.15.37:0/2664670807 wait complete. 2024-09-21T06:28:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.250+0000 7f38aec00640 1 Processor -- start 2024-09-21T06:28:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.250+0000 7f38aec00640 1 -- start start 2024-09-21T06:28:45.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.251+0000 7f38aec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a80acfd0 0x7f38a81f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38aec00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 0x7f38a8154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38aec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38aec00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80d0ae0 con 0x7f38a80af210 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38aec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80d0960 con 0x7f38a80acfd0 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38aec00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d0c60 con 0x7f38a80aeb70 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48416/0 (socket says 172.21.15.37:48416) 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 -- 172.21.15.37:0/2433748095 learned_addr learned my addr 172.21.15.37:0/2433748095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ad800640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a80acfd0 0x7f38a81f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ace00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 0x7f38a8154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 -- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 msgr2=0x7f38a8154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 0x7f38a8154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 -- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a80acfd0 msgr2=0x7f38a81f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a80acfd0 0x7f38a81f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ae200640 1 -- 172.21.15.37:0/2433748095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81f93a0 con 0x7f38a80af210 2024-09-21T06:28:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.252+0000 7f38ace00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 0x7f38a8154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38ae200640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d960 tx=0x7f38a000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017070 con 0x7f38a80af210 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38a000be20 con 0x7f38a80af210 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00106a0 con 0x7f38a80af210 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a81f9680 con 0x7f38a80af210 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.253+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80ce0b0 con 0x7f38a80af210 2024-09-21T06:28:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.254+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a81f9110 con 0x7f38a80af210 2024-09-21T06:28:45.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.255+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38a001a070 con 0x7f38a80af210 2024-09-21T06:28:45.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.256+0000 7f38a5600640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 0x7f388807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.256+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f38a009b540 con 0x7f38a80af210 2024-09-21T06:28:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.257+0000 7f38ad800640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 0x7f388807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.258+0000 7f38ad800640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 0x7f388807b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38a80c3db0 tx=0x7f389c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:45.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.261+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38a0062f90 con 0x7f38a80af210 2024-09-21T06:28:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.462+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38a80d0420 con 0x7f38a80af210 2024-09-21T06:28:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.464+0000 7f38a5600640 1 -- 172.21.15.37:0/2433748095 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18756 (secure 0 0 0) 0x7f38a00679c0 con 0x7f38a80af210 2024-09-21T06:28:45.465 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:45.465 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:38.200188+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.24","mappings":[{"from":0,"to":7}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 msgr2=0x7f388807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 0x7f388807b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38a80c3db0 tx=0x7f389c009040 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 msgr2=0x7f38a80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d960 tx=0x7f38a000de20 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 shutdown_connections 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3888078b90 0x7f388807b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a80af210 0x7f38a80cd0b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a80aeb70 0x7f38a8154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 --2- 172.21.15.37:0/2433748095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a80acfd0 0x7f38a81f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 >> 172.21.15.37:0/2433748095 conn(0x7f38a80b5140 msgr2=0x7f38a806f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 shutdown_connections 2024-09-21T06:28:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.466+0000 7f38aec00640 1 -- 172.21.15.37:0/2433748095 wait complete. 2024-09-21T06:28:45.492 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:27:58.550218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:27:59.683562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T06:28:03.246648+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T06:28:45.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:28:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 -- 172.21.15.37:0/3755054478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980b7c60 msgr2=0x7f7e980d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 --2- 172.21.15.37:0/3755054478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980b7c60 0x7f7e980d3420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009830 tx=0x7f7e8002a480 comp rx=0 tx=0).stop 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 -- 172.21.15.37:0/3755054478 shutdown_connections 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 --2- 172.21.15.37:0/3755054478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980b7c60 0x7f7e980d3420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 --2- 172.21.15.37:0/3755054478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e9815e4d0 0x7f7e980b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 -- 172.21.15.37:0/3755054478 >> 172.21.15.37:0/3755054478 conn(0x7f7e980d1980 msgr2=0x7f7e980d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 -- 172.21.15.37:0/3755054478 shutdown_connections 2024-09-21T06:28:45.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.721+0000 7f7e97e00640 1 -- 172.21.15.37:0/3755054478 wait complete. 2024-09-21T06:28:45.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.723+0000 7f7e97e00640 1 Processor -- start 2024-09-21T06:28:45.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.724+0000 7f7e97e00640 1 -- start start 2024-09-21T06:28:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 0x7f7e980cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 0x7f7e980b9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9806de10 con 0x7f7e980d3960 2024-09-21T06:28:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9806dc90 con 0x7f7e9815e4d0 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e97e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9806df90 con 0x7f7e980b7c60 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e8e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e8e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48424/0 (socket says 172.21.15.37:48424) 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.725+0000 7f7e8e000640 1 -- 172.21.15.37:0/62886229 learned_addr learned my addr 172.21.15.37:0/62886229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e96a00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 0x7f7e980cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e97400640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 0x7f7e980b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 -- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 msgr2=0x7f7e980cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 0x7f7e980cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 -- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 msgr2=0x7f7e980b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 0x7f7e980b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 -- 172.21.15.37:0/62886229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e980cceb0 con 0x7f7e980d3960 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e96a00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 0x7f7e980cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e97400640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 0x7f7e980b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8e000640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e80033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:45.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.726+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80005c90 con 0x7f7e980d3960 2024-09-21T06:28:45.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.727+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e80031e50 con 0x7f7e980d3960 2024-09-21T06:28:45.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.727+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80033290 con 0x7f7e980d3960 2024-09-21T06:28:45.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.727+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e980cd190 con 0x7f7e980d3960 2024-09-21T06:28:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.727+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e980cd750 con 0x7f7e980d3960 2024-09-21T06:28:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.728+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9806e520 con 0x7f7e980d3960 2024-09-21T06:28:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.729+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e80033430 con 0x7f7e980d3960 2024-09-21T06:28:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.730+0000 7f7e8f400640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 0x7f7e7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.730+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f7e800bc5d0 con 0x7f7e980d3960 2024-09-21T06:28:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.730+0000 7f7e96a00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 0x7f7e7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.731+0000 7f7e96a00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 0x7f7e7407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005fd0 tx=0x7f7e88009480 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.735+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e8003f030 con 0x7f7e980d3960 2024-09-21T06:28:45.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.930+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e980ccc40 con 0x7f7e980d3960 2024-09-21T06:28:45.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.931+0000 7f7e8f400640 1 -- 172.21.15.37:0/62886229 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f7e80084670 con 0x7f7e980d3960 2024-09-21T06:28:45.933 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 msgr2=0x7f7e7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 0x7f7e7407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005fd0 tx=0x7f7e88009480 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 msgr2=0x7f7e9814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e80033c90 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 shutdown_connections 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e74078b90 0x7f7e7407b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e980d3960 0x7f7e9814c5e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e9815e4d0 0x7f7e980b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 --2- 172.21.15.37:0/62886229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e980b7c60 0x7f7e980cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 >> 172.21.15.37:0/62886229 conn(0x7f7e980d1980 msgr2=0x7f7e98161f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 shutdown_connections 2024-09-21T06:28:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:45.933+0000 7f7e97e00640 1 -- 172.21.15.37:0/62886229 wait complete. 2024-09-21T06:28:45.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.184+0000 7fde29400640 1 -- 172.21.15.37:0/60174268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde241609e0 msgr2=0x7fde24152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.184+0000 7fde29400640 1 --2- 172.21.15.37:0/60174268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde241609e0 0x7fde24152c30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde240b1920 tx=0x7fde1c02a980 comp rx=0 tx=0).stop 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 -- 172.21.15.37:0/60174268 shutdown_connections 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 --2- 172.21.15.37:0/60174268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde241609e0 0x7fde24152c30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 --2- 172.21.15.37:0/60174268 >> v2:172.21.15.150:3300/0 conn(0x7fde240ccdd0 0x7fde241604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 -- 172.21.15.37:0/60174268 >> 172.21.15.37:0/60174268 conn(0x7fde240ceaf0 msgr2=0x7fde240cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 -- 172.21.15.37:0/60174268 shutdown_connections 2024-09-21T06:28:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.185+0000 7fde29400640 1 -- 172.21.15.37:0/60174268 wait complete. 2024-09-21T06:28:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.187+0000 7fde29400640 1 Processor -- start 2024-09-21T06:28:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.187+0000 7fde29400640 1 -- start start 2024-09-21T06:28:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde24071c10 0x7fde240b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde241609e0 0x7fde241f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde24153580 con 0x7fde24158980 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde24153400 con 0x7fde241609e0 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde29400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fde24153700 con 0x7fde24071c10 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde28a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde28a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48446/0 (socket says 172.21.15.37:48446) 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde28a00640 1 -- 172.21.15.37:0/2967744389 learned_addr learned my addr 172.21.15.37:0/2967744389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde23e00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde24071c10 0x7fde240b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.190+0000 7fde1be00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde241609e0 0x7fde241f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 -- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde24071c10 msgr2=0x7fde240b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde24071c10 0x7fde240b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 -- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde241609e0 msgr2=0x7fde241f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde241609e0 0x7fde241f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 -- 172.21.15.37:0/2967744389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2415a110 con 0x7fde24158980 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde28a00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1c033040 tx=0x7fde1c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c034040 con 0x7fde24158980 2024-09-21T06:28:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde1c004640 con 0x7fde24158980 2024-09-21T06:28:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c045480 con 0x7fde24158980 2024-09-21T06:28:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.191+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde240b54c0 con 0x7fde24158980 2024-09-21T06:28:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.192+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde24159910 con 0x7fde24158980 2024-09-21T06:28:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.193+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde240b5230 con 0x7fde24158980 2024-09-21T06:28:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.194+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde1c037020 con 0x7fde24158980 2024-09-21T06:28:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.195+0000 7fde20c00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 0x7fde0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.195+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7fde1c0bc880 con 0x7fde24158980 2024-09-21T06:28:46.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.196+0000 7fde23e00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 0x7fde0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.196+0000 7fde23e00640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 0x7fde0007b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde14006fd0 tx=0x7fde14009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:46.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.200+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde2415caa0 con 0x7fde24158980 2024-09-21T06:28:46.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.394+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde2415c8c0 con 0x7fde24158980 2024-09-21T06:28:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.395+0000 7fde20c00640 1 -- 172.21.15.37:0/2967744389 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fde2415c8c0 con 0x7fde24158980 2024-09-21T06:28:46.397 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-09-21T06:28:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 msgr2=0x7fde0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 0x7fde0007b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde14006fd0 tx=0x7fde14009060 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 msgr2=0x7fde24158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1c033040 tx=0x7fde1c003ef0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 shutdown_connections 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fde00078bc0 0x7fde0007b0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde24158980 0x7fde24158d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde241609e0 0x7fde241f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 --2- 172.21.15.37:0/2967744389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde24071c10 0x7fde240b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 >> 172.21.15.37:0/2967744389 conn(0x7fde240ceaf0 msgr2=0x7fde2415fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 shutdown_connections 2024-09-21T06:28:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.397+0000 7fde29400640 1 -- 172.21.15.37:0/2967744389 wait complete. 2024-09-21T06:28:46.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:28:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 -- 172.21.15.37:0/2787674780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab015b900 msgr2=0x7f3ab006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/2787674780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab015b900 0x7f3ab006e720 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0073490 tx=0x7f3a9802a4d0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 -- 172.21.15.37:0/2787674780 shutdown_connections 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/2787674780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab015b900 0x7f3ab006e720 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/2787674780 >> v2:172.21.15.150:3300/0 conn(0x7f3ab0072320 0x7f3ab0152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 -- 172.21.15.37:0/2787674780 >> 172.21.15.37:0/2787674780 conn(0x7f3ab00d1020 msgr2=0x7f3ab00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 -- 172.21.15.37:0/2787674780 shutdown_connections 2024-09-21T06:28:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.648+0000 7f3ab5c00640 1 -- 172.21.15.37:0/2787674780 wait complete. 2024-09-21T06:28:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.651+0000 7f3ab5c00640 1 Processor -- start 2024-09-21T06:28:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.651+0000 7f3ab5c00640 1 -- start start 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 0x7f3ab01f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab015b900 0x7f3ab01f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00b40d0 con 0x7f3ab0071160 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00b3f50 con 0x7f3ab0072320 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3ab5c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab00b4250 con 0x7f3ab015b900 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.652+0000 7f3aafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48464/0 (socket says 172.21.15.37:48464) 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 -- 172.21.15.37:0/1585688746 learned_addr learned my addr 172.21.15.37:0/1585688746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aa7e00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 0x7f3ab01f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3ab5200640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab015b900 0x7f3ab01f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 -- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab015b900 msgr2=0x7f3ab01f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab015b900 0x7f3ab01f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 -- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 msgr2=0x7f3ab01f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 0x7f3ab01f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 -- 172.21.15.37:0/1585688746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab01f9280 con 0x7f3ab0071160 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aa7e00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 0x7f3ab01f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.653+0000 7f3aafe00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d7a0 tx=0x7f3aa000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.654+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab0071160 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.654+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aa0009d20 con 0x7f3ab0071160 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.654+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00106c0 con 0x7f3ab0071160 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.654+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab01e7e40 con 0x7f3ab0071160 2024-09-21T06:28:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.654+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab01e8400 con 0x7f3ab0071160 2024-09-21T06:28:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.655+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab01e7c70 con 0x7f3ab0071160 2024-09-21T06:28:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.656+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3aa001a070 con 0x7f3ab0071160 2024-09-21T06:28:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.657+0000 7f3aacc00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 0x7f3a8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.657+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f3aa009b130 con 0x7f3ab0071160 2024-09-21T06:28:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.658+0000 7f3aa7e00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 0x7f3a8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.658+0000 7f3aa7e00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 0x7f3a8c07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a98033000 tx=0x7f3a98004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:46.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.661+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3aa0062c10 con 0x7f3ab0071160 2024-09-21T06:28:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.855+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ab00b4770 con 0x7f3ab0071160 2024-09-21T06:28:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.855+0000 7f3aacc00640 1 -- 172.21.15.37:0/1585688746 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f3aa0067640 con 0x7f3ab0071160 2024-09-21T06:28:46.858 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-09-21T06:28:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.857+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 msgr2=0x7f3a8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.857+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 0x7f3a8c07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a98033000 tx=0x7f3a98004730 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.857+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 msgr2=0x7f3ab0146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.857+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d7a0 tx=0x7f3aa000dc60 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 shutdown_connections 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3a8c078b90 0x7f3a8c07b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab015b900 0x7f3ab01f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0072320 0x7f3ab01f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 --2- 172.21.15.37:0/1585688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0071160 0x7f3ab0146a80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 >> 172.21.15.37:0/1585688746 conn(0x7f3ab00d1020 msgr2=0x7f3ab00b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 shutdown_connections 2024-09-21T06:28:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:46.858+0000 7f3ab5c00640 1 -- 172.21.15.37:0/1585688746 wait complete. 2024-09-21T06:28:46.883 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 -- 172.21.15.37:0/2941653863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde0004e60 msgr2=0x7fbde00052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 --2- 172.21.15.37:0/2941653863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde0004e60 0x7fbde00052b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0005fd0 tx=0x7fbde0005830 comp rx=0 tx=0).stop 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfd600640 1 -- 172.21.15.37:0/2941653863 reap_dead start 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 -- 172.21.15.37:0/2941653863 shutdown_connections 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 -- 172.21.15.37:0/2941653863 >> 172.21.15.37:0/2941653863 conn(0x7fbdf806f920 msgr2=0x7fbdf80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 -- 172.21.15.37:0/2941653863 shutdown_connections 2024-09-21T06:28:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.113+0000 7fbdfe000640 1 -- 172.21.15.37:0/2941653863 wait complete. 2024-09-21T06:28:47.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.115+0000 7fbdfe000640 1 Processor -- start 2024-09-21T06:28:47.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.116+0000 7fbdfe000640 1 -- start start 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbde0004e60 0x7fbdf814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf815e470 0x7fbdf815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cdf40 con 0x7fbdf81f3f30 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cddc0 con 0x7fbdf815e470 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfe000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf80ce0c0 con 0x7fbde0004e60 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48486/0 (socket says 172.21.15.37:48486) 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdf7e00640 1 -- 172.21.15.37:0/46894936 learned_addr learned my addr 172.21.15.37:0/46894936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.117+0000 7fbdfcc00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbde0004e60 0x7fbdf814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdfd600640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf815e470 0x7fbdf815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 -- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbde0004e60 msgr2=0x7fbdf814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbde0004e60 0x7fbdf814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 -- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf815e470 msgr2=0x7fbdf815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf815e470 0x7fbdf815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 -- 172.21.15.37:0/46894936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf815edc0 con 0x7fbdf81f3f30 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf7e00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbde4003ec0 tx=0x7fbde400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4015070 con 0x7fbdf81f3f30 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbde400fe90 con 0x7fbdf81f3f30 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.118+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4011b00 con 0x7fbdf81f3f30 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.119+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf81f7150 con 0x7fbdf81f3f30 2024-09-21T06:28:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.119+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf8150500 con 0x7fbdf81f3f30 2024-09-21T06:28:47.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.120+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf81f6ea0 con 0x7fbdf81f3f30 2024-09-21T06:28:47.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.121+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbde4011ca0 con 0x7fbdf81f3f30 2024-09-21T06:28:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.123+0000 7fbdf4c00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.123+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7fbde409bb80 con 0x7fbdf81f3f30 2024-09-21T06:28:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.123+0000 7fbdfcc00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.124+0000 7fbdfcc00640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 0x7fbdd807b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbdf806eb80 tx=0x7fbdec006720 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.127+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbdf815edc0 con 0x7fbdf81f3f30 2024-09-21T06:28:47.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.324+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbdf806d990 con 0x7fbdf81f3f30 2024-09-21T06:28:47.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.324+0000 7fbdf4c00640 1 -- 172.21.15.37:0/46894936 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fbde40635d0 con 0x7fbdf81f3f30 2024-09-21T06:28:47.342 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-09-21T06:28:47.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 msgr2=0x7fbdd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 0x7fbdd807b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbdf806eb80 tx=0x7fbdec006720 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 msgr2=0x7fbdf8150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbde4003ec0 tx=0x7fbde400f720 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 shutdown_connections 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdf81f3f30 0x7fbdf8150bf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf815e470 0x7fbdf815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 --2- 172.21.15.37:0/46894936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbde0004e60 0x7fbdf814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 >> 172.21.15.37:0/46894936 conn(0x7fbdf806f920 msgr2=0x7fbdf80b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 shutdown_connections 2024-09-21T06:28:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.327+0000 7fbdfe000640 1 -- 172.21.15.37:0/46894936 wait complete. 2024-09-21T06:28:47.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-21T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 -- 172.21.15.37:0/4156384966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 msgr2=0x7f335c0b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 --2- 172.21.15.37:0/4156384966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c0b1dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3348009830 tx=0x7f334802a510 comp rx=0 tx=0).stop 2024-09-21T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 -- 172.21.15.37:0/4156384966 shutdown_connections 2024-09-21T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 --2- 172.21.15.37:0/4156384966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c0b1dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 --2- 172.21.15.37:0/4156384966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c15a280 0x7f335c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.581+0000 7f3360e00640 1 -- 172.21.15.37:0/4156384966 >> 172.21.15.37:0/4156384966 conn(0x7f335c0ccf00 msgr2=0x7f335c0cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.582+0000 7f3360e00640 1 -- 172.21.15.37:0/4156384966 shutdown_connections 2024-09-21T06:28:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.582+0000 7f3360e00640 1 -- 172.21.15.37:0/4156384966 wait complete. 2024-09-21T06:28:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.584+0000 7f3360e00640 1 Processor -- start 2024-09-21T06:28:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.584+0000 7f3360e00640 1 -- start start 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c0b2300 0x7f335c0c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f335c15a280 0x7f335c0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f335c160860 con 0x7f335c0ace60 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1609e0 con 0x7f335c15a280 2024-09-21T06:28:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3360e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f335c160b60 con 0x7f335c0b2300 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.585+0000 7f3359e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48494/0 (socket says 172.21.15.37:48494) 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 -- 172.21.15.37:0/2960522981 learned_addr learned my addr 172.21.15.37:0/2960522981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359400640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c0b2300 0x7f335c0c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f335a800640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f335c15a280 0x7f335c0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 -- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c0b2300 msgr2=0x7f335c0c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c0b2300 0x7f335c0c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 -- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f335c15a280 msgr2=0x7f335c0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f335c15a280 0x7f335c0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:47.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 -- 172.21.15.37:0/2960522981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c0c5200 con 0x7f335c0ace60 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3359e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c9a0 tx=0x7f334c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.586+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c007bb0 con 0x7f335c0ace60 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.587+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f334c007e40 con 0x7f335c0ace60 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.587+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c005290 con 0x7f335c0ace60 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.587+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c0c54e0 con 0x7f335c0ace60 2024-09-21T06:28:47.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.587+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c0c5aa0 con 0x7f335c0ace60 2024-09-21T06:28:47.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.588+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335c0b1dc0 con 0x7f335c0ace60 2024-09-21T06:28:47.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.589+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f334c005430 con 0x7f335c0ace60 2024-09-21T06:28:47.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.591+0000 7f3352000640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 0x7f333c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:47.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.591+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f334c09b600 con 0x7f335c0ace60 2024-09-21T06:28:47.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.591+0000 7f3359400640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 0x7f333c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:47.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.592+0000 7f3359400640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 0x7f333c07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3348034000 tx=0x7f3348004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:47.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.595+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f334c014070 con 0x7f335c0ace60 2024-09-21T06:28:47.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:47.870+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7f335c0696c0 con 0x7f335c0ace60 2024-09-21T06:28:48.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.196+0000 7f3352000640 1 -- 172.21.15.37:0/2960522981 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v70) ==== 139+0+0 (secure 0 0 0) 0x7f334c063610 con 0x7f335c0ace60 2024-09-21T06:28:48.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 msgr2=0x7f333c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 0x7f333c07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3348034000 tx=0x7f3348004730 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 msgr2=0x7f335c1f3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c9a0 tx=0x7f334c00ce60 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 shutdown_connections 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f333c078bc0 0x7f333c07b0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f335c15a280 0x7f335c0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f335c0b2300 0x7f335c0c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 --2- 172.21.15.37:0/2960522981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f335c0ace60 0x7f335c1f3d20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 >> 172.21.15.37:0/2960522981 conn(0x7f335c0ccf00 msgr2=0x7f335c0b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 shutdown_connections 2024-09-21T06:28:48.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.198+0000 7f3360e00640 1 -- 172.21.15.37:0/2960522981 wait complete. 2024-09-21T06:28:48.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:28:48.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 -- 172.21.15.37:0/3880617652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c0afb80 msgr2=0x7f484c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 --2- 172.21.15.37:0/3880617652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c0afb80 0x7f484c06ddf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4834005fd0 tx=0x7f483402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 -- 172.21.15.37:0/3880617652 shutdown_connections 2024-09-21T06:28:48.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 --2- 172.21.15.37:0/3880617652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c0afb80 0x7f484c06ddf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 --2- 172.21.15.37:0/3880617652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c073f20 0x7f484c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 -- 172.21.15.37:0/3880617652 >> 172.21.15.37:0/3880617652 conn(0x7f484c073400 msgr2=0x7f484c073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:48.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 -- 172.21.15.37:0/3880617652 shutdown_connections 2024-09-21T06:28:48.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.450+0000 7f4850e00640 1 -- 172.21.15.37:0/3880617652 wait complete. 2024-09-21T06:28:48.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.452+0000 7f4850e00640 1 Processor -- start 2024-09-21T06:28:48.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.452+0000 7f4850e00640 1 -- start start 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f484c06e330 0x7f484c148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c0afb80 0x7f484c1e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f484c160700 con 0x7f484c073f20 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f484c0cfb80 con 0x7f484c06e330 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4850e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f484c160880 con 0x7f484c0afb80 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48506/0 (socket says 172.21.15.37:48506) 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 -- 172.21.15.37:0/2502909753 learned_addr learned my addr 172.21.15.37:0/2502909753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:48.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f484b400640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f484c06e330 0x7f484c148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f484be00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c0afb80 0x7f484c1e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 -- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c0afb80 msgr2=0x7f484c1e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c0afb80 0x7f484c1e5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 -- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f484c06e330 msgr2=0x7f484c148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f484c06e330 0x7f484c148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.454+0000 7f4843e00640 1 -- 172.21.15.37:0/2502909753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c1e6200 con 0x7f484c073f20 2024-09-21T06:28:48.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4843e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f48340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483403e030 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4834003ca0 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834035040 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c14b0c0 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.455+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c14b680 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.456+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484c14ae30 con 0x7f484c073f20 2024-09-21T06:28:48.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.457+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4834041030 con 0x7f484c073f20 2024-09-21T06:28:48.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.458+0000 7f4843400640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 0x7f482807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.459+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f48340bb6f0 con 0x7f484c073f20 2024-09-21T06:28:48.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.459+0000 7f484b400640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 0x7f482807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.460+0000 7f484b400640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 0x7f482807b0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f483c00bb50 tx=0x7f483c00b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.463+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f484c1e6200 con 0x7f484c073f20 2024-09-21T06:28:48.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.662+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f484c160d30 con 0x7f484c073f20 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.663+0000 7f4843400640 1 -- 172.21.15.37:0/2502909753 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f48340830b0 con 0x7f484c073f20 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:28:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:28:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:28:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 msgr2=0x7f482807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 0x7f482807b0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f483c00bb50 tx=0x7f483c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 msgr2=0x7f484c148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f48340038a0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 shutdown_connections 2024-09-21T06:28:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4828078bc0 0x7f482807b0d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f484c073f20 0x7f484c148610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f484c0afb80 0x7f484c1e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 --2- 172.21.15.37:0/2502909753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f484c06e330 0x7f484c148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.665+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 >> 172.21.15.37:0/2502909753 conn(0x7f484c073400 msgr2=0x7f484c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.666+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 shutdown_connections 2024-09-21T06:28:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.666+0000 7f4850e00640 1 -- 172.21.15.37:0/2502909753 wait complete. 2024-09-21T06:28:48.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:28:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 -- 172.21.15.37:0/3435365569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 msgr2=0x7f355400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 --2- 172.21.15.37:0/3435365569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f355400b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3568005fd0 tx=0x7f3554005830 comp rx=0 tx=0).stop 2024-09-21T06:28:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 -- 172.21.15.37:0/3435365569 shutdown_connections 2024-09-21T06:28:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 --2- 172.21.15.37:0/3435365569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f355400b9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 -- 172.21.15.37:0/3435365569 >> 172.21.15.37:0/3435365569 conn(0x7f356c0b1b90 msgr2=0x7f356c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 -- 172.21.15.37:0/3435365569 shutdown_connections 2024-09-21T06:28:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.921+0000 7f3574800640 1 -- 172.21.15.37:0/3435365569 wait complete. 2024-09-21T06:28:48.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.923+0000 7f3574800640 1 Processor -- start 2024-09-21T06:28:48.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.923+0000 7f3574800640 1 -- start start 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 0x7f356c15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 0x7f356c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f356c06e590 con 0x7f35540052a0 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f356c06e410 con 0x7f356c0c5d40 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3574800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f356c06e710 con 0x7f356c15ac60 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48534/0 (socket says 172.21.15.37:48534) 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 -- 172.21.15.37:0/2910168245 learned_addr learned my addr 172.21.15.37:0/2910168245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571400640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 0x7f356c15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3572800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 0x7f356c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 -- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 msgr2=0x7f356c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 0x7f356c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 -- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 msgr2=0x7f356c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 0x7f356c15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571e00640 1 -- 172.21.15.37:0/2910168245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1554c0 con 0x7f35540052a0 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3571400640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 0x7f356c15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.925+0000 7f3572800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 0x7f356c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:48.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3571e00640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3560003d60 tx=0x7f3560001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560011640 con 0x7f35540052a0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3560016020 con 0x7f35540052a0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c15b070 con 0x7f35540052a0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560011a60 con 0x7f35540052a0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.926+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c0b5c50 con 0x7f35540052a0 2024-09-21T06:28:48.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.927+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c0cfc30 con 0x7f35540052a0 2024-09-21T06:28:48.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.929+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3560011c00 con 0x7f35540052a0 2024-09-21T06:28:48.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.930+0000 7f3566000640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 0x7f355007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:48.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.930+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f356009bd50 con 0x7f35540052a0 2024-09-21T06:28:48.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.931+0000 7f3571400640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 0x7f355007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:48.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.931+0000 7f3571400640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 0x7f355007b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f355c001860 tx=0x7f355c000fd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:48.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:48.935+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3560020020 con 0x7f35540052a0 2024-09-21T06:28:49.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:28:49.124 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:28:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.135+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f356c0748a0 con 0x7f35540052a0 2024-09-21T06:28:49.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.140+0000 7f3566000640 1 -- 172.21.15.37:0/2910168245 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f356c15b070 con 0x7f35540052a0 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 msgr2=0x7f355007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 0x7f355007b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f355c001860 tx=0x7f355c000fd0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 msgr2=0x7f356c154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3560003d60 tx=0x7f3560001800 comp rx=0 tx=0).stop 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 shutdown_connections 2024-09-21T06:28:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3550078b90 0x7f355007b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c15ac60 0x7f356c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c0c5d40 0x7f356c15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 --2- 172.21.15.37:0/2910168245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35540052a0 0x7f356c154f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.143+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 >> 172.21.15.37:0/2910168245 conn(0x7f356c0b1b90 msgr2=0x7f356c156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.144+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 shutdown_connections 2024-09-21T06:28:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.144+0000 7f3574800640 1 -- 172.21.15.37:0/2910168245 wait complete. 2024-09-21T06:28:49.173 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:28:49.173 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:28:49.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:28:49.402 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:28:49.402 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:49.403 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:49.438 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:49.438 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:49.515 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:49.515 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:49.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 -- 172.21.15.37:0/2347425924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7314072d90 msgr2=0x7f73140731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 --2- 172.21.15.37:0/2347425924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7314072d90 0x7f73140731a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f731415cda0 tx=0x7f730c02a920 comp rx=0 tx=0).stop 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 -- 172.21.15.37:0/2347425924 shutdown_connections 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 --2- 172.21.15.37:0/2347425924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7314072d90 0x7f73140731a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 -- 172.21.15.37:0/2347425924 >> 172.21.15.37:0/2347425924 conn(0x7f73140b1b90 msgr2=0x7f73140b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 -- 172.21.15.37:0/2347425924 shutdown_connections 2024-09-21T06:28:49.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.864+0000 7f731c600640 1 -- 172.21.15.37:0/2347425924 wait complete. 2024-09-21T06:28:49.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.866+0000 7f731c600640 1 Processor -- start 2024-09-21T06:28:49.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.867+0000 7f731c600640 1 -- start start 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 0x7f73140cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314156a50 0x7f73141f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7314073860 con 0x7f7314072d90 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73140736e0 con 0x7f731406ddf0 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f731c600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f73140739e0 con 0x7f7314156a50 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f7319200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f7319200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48550/0 (socket says 172.21.15.37:48550) 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.868+0000 7f7319200640 1 -- 172.21.15.37:0/919380172 learned_addr learned my addr 172.21.15.37:0/919380172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 0x7f73140cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:49.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f731a600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314156a50 0x7f73141f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:49.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 -- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314156a50 msgr2=0x7f73141f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:49.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314156a50 0x7f73141f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 -- 172.21.15.37:0/919380172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 msgr2=0x7f73141f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319c00640 1 -- 172.21.15.37:0/919380172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73141f9680 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.869+0000 7f7319200640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f7319c00640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 0x7f73140cb8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f730800c3c0 tx=0x7f730800c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308011790 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7308017070 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308011cc0 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73140cc070 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.870+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141f8940 con 0x7f731406ddf0 2024-09-21T06:28:49.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.871+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73140731a0 con 0x7f731406ddf0 2024-09-21T06:28:49.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.873+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f730801c460 con 0x7f731406ddf0 2024-09-21T06:28:49.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.873+0000 7f7311600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 0x7f72f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:49.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.874+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f730809af80 con 0x7f731406ddf0 2024-09-21T06:28:49.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.874+0000 7f7319200640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 0x7f72f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:49.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.875+0000 7f7319200640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 0x7f72f407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7314070b30 tx=0x7f7300009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:49.877+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7308062b10 con 0x7f731406ddf0 2024-09-21T06:28:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.088+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7314070b60 con 0x7f731406ddf0 2024-09-21T06:28:50.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.089+0000 7f7311600640 1 -- 172.21.15.37:0/919380172 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f7308067540 con 0x7f731406ddf0 2024-09-21T06:28:50.091 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.091+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 msgr2=0x7f72f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.091+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 0x7f72f407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7314070b30 tx=0x7f7300009040 comp rx=0 tx=0).stop 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.091+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 msgr2=0x7f73140cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.091+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 0x7f73140cb8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f730800c3c0 tx=0x7f730800c7c0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 shutdown_connections 2024-09-21T06:28:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f4078b90 0x7f72f407b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7314072d90 0x7f73141f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314156a50 0x7f73141f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 --2- 172.21.15.37:0/919380172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f731406ddf0 0x7f73140cb8d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 >> 172.21.15.37:0/919380172 conn(0x7f73140b1b90 msgr2=0x7f731409fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 shutdown_connections 2024-09-21T06:28:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.092+0000 7f731c600640 1 -- 172.21.15.37:0/919380172 wait complete. 2024-09-21T06:28:50.117 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:28:50.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:50.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T06:28:50.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:28:50.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:50.182 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:28:50.219 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:50.220 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:50.261 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:50.261 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:50.338 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:50.338 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:50.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.640+0000 7f2c84600640 1 -- 172.21.15.37:0/1092410438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c06e500 msgr2=0x7f2c7c0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.640+0000 7f2c84600640 1 --2- 172.21.15.37:0/1092410438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c06e500 0x7f2c7c0cef00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0b2c70 tx=0x7f2c7402aac0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.640+0000 7f2c84600640 1 -- 172.21.15.37:0/1092410438 shutdown_connections 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.640+0000 7f2c84600640 1 --2- 172.21.15.37:0/1092410438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c06e500 0x7f2c7c0cef00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.640+0000 7f2c84600640 1 --2- 172.21.15.37:0/1092410438 >> v2:172.21.15.150:3300/0 conn(0x7f2c7c156a50 0x7f2c7c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.641+0000 7f2c84600640 1 -- 172.21.15.37:0/1092410438 >> 172.21.15.37:0/1092410438 conn(0x7f2c7c0b1b90 msgr2=0x7f2c7c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.641+0000 7f2c84600640 1 -- 172.21.15.37:0/1092410438 shutdown_connections 2024-09-21T06:28:50.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.641+0000 7f2c84600640 1 -- 172.21.15.37:0/1092410438 wait complete. 2024-09-21T06:28:50.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.643+0000 7f2c84600640 1 Processor -- start 2024-09-21T06:28:50.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.643+0000 7f2c84600640 1 -- start start 2024-09-21T06:28:50.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.644+0000 7f2c84600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c7c074b60 0x7f2c7c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:50.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.644+0000 7f2c84600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 0x7f2c7c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:50.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c84600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c84600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0cf7d0 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c84600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0cf650 con 0x7f2c7c156a50 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c84600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7c0cf950 con 0x7f2c7c074b60 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48580/0 (socket says 172.21.15.37:48580) 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 -- 172.21.15.37:0/994277910 learned_addr learned my addr 172.21.15.37:0/994277910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c81c00640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c7c074b60 0x7f2c7c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c81200640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 0x7f2c7c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 -- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c7c074b60 msgr2=0x7f2c7c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c7c074b60 0x7f2c7c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 -- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 msgr2=0x7f2c7c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 0x7f2c7c157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c82600640 1 -- 172.21.15.37:0/994277910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c15c500 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.645+0000 7f2c81200640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 0x7f2c7c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c82600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d60 tx=0x7f2c74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74044070 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c74003e00 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c740044b0 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1f51c0 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.646+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1f5780 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.647+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c157260 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.648+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2c74004650 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.649+0000 7f2c79600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 0x7f2c5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.649+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f2c740bb930 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.650+0000 7f2c81c00640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 0x7f2c5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.650+0000 7f2c81c00640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 0x7f2c5c07b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009b30 tx=0x7f2c70005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:50.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.654+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c7c15c500 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.863+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c7c0b2320 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.863+0000 7f2c79600640 1 -- 172.21.15.37:0/994277910 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f2c74032030 con 0x7f2c7c15bbb0 2024-09-21T06:28:50.865 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 msgr2=0x7f2c5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 0x7f2c5c07b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009b30 tx=0x7f2c70005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 msgr2=0x7f2c7c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d60 tx=0x7f2c74003c10 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 shutdown_connections 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c5c078b90 0x7f2c5c07b0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7c15bbb0 0x7f2c7c15bfc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c7c156a50 0x7f2c7c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 --2- 172.21.15.37:0/994277910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c7c074b60 0x7f2c7c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 >> 172.21.15.37:0/994277910 conn(0x7f2c7c0b1b90 msgr2=0x7f2c7c0afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 shutdown_connections 2024-09-21T06:28:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:50.866+0000 7f2c84600640 1 -- 172.21.15.37:0/994277910 wait complete. 2024-09-21T06:28:50.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T06:28:50.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:50.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T06:28:50.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:28:50.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:50.956 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T06:28:51.003 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:51.003 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.039 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:51.039 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.115 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:28:51.115 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.456+0000 7f7a15600640 1 -- 172.21.15.37:0/2724292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10145bf0 msgr2=0x7f7a100d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.456+0000 7f7a15600640 1 --2- 172.21.15.37:0/2724292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10145bf0 0x7f7a100d05b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009830 tx=0x7f79f802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.456+0000 7f7a15600640 1 -- 172.21.15.37:0/2724292216 shutdown_connections 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.456+0000 7f7a15600640 1 --2- 172.21.15.37:0/2724292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10145bf0 0x7f7a100d05b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.456+0000 7f7a15600640 1 -- 172.21.15.37:0/2724292216 >> 172.21.15.37:0/2724292216 conn(0x7f7a1006fc80 msgr2=0x7f7a10070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.457+0000 7f7a15600640 1 -- 172.21.15.37:0/2724292216 shutdown_connections 2024-09-21T06:28:51.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.457+0000 7f7a15600640 1 -- 172.21.15.37:0/2724292216 wait complete. 2024-09-21T06:28:51.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.459+0000 7f7a15600640 1 Processor -- start 2024-09-21T06:28:51.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.459+0000 7f7a15600640 1 -- start start 2024-09-21T06:28:51.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a10145bf0 0x7f7a101442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a1015d5e0 0x7f7a10144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1006d260 con 0x7f7a10071160 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1006d0e0 con 0x7f7a10145bf0 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.460+0000 7f7a15600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1006d3e0 con 0x7f7a1015d5e0 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48600/0 (socket says 172.21.15.37:48600) 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 -- 172.21.15.37:0/767263898 learned_addr learned my addr 172.21.15.37:0/767263898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a07e00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a10145bf0 0x7f7a101442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a14c00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a1015d5e0 0x7f7a10144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 -- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a1015d5e0 msgr2=0x7f7a10144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a1015d5e0 0x7f7a10144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 -- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a10145bf0 msgr2=0x7f7a101442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a10145bf0 0x7f7a101442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.461+0000 7f7a0fe00640 1 -- 172.21.15.37:0/767263898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a100b4fb0 con 0x7f7a10071160 2024-09-21T06:28:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a0fe00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c430 tx=0x7f7a0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00017070 con 0x7f7a10071160 2024-09-21T06:28:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a000053e0 con 0x7f7a10071160 2024-09-21T06:28:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00003850 con 0x7f7a10071160 2024-09-21T06:28:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a100b41f0 con 0x7f7a10071160 2024-09-21T06:28:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.462+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a100b47b0 con 0x7f7a10071160 2024-09-21T06:28:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.463+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a100d05b0 con 0x7f7a10071160 2024-09-21T06:28:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.464+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a0001a070 con 0x7f7a10071160 2024-09-21T06:28:51.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.465+0000 7f7a0cc00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 0x7f79ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:51.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.465+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f7a0009a4c0 con 0x7f7a10071160 2024-09-21T06:28:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.466+0000 7f7a07e00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 0x7f79ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.467+0000 7f7a07e00640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 0x7f79ec07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:51.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.470+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a00061e80 con 0x7f7a10071160 2024-09-21T06:28:51.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.678+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a100b08a0 con 0x7f7a10071160 2024-09-21T06:28:51.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.679+0000 7f7a0cc00640 1 -- 172.21.15.37:0/767263898 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f7a000668b0 con 0x7f7a10071160 2024-09-21T06:28:51.681 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 msgr2=0x7f79ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 0x7f79ec07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f8004730 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 msgr2=0x7f7a101f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c430 tx=0x7f7a0000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 shutdown_connections 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79ec078bc0 0x7f79ec07b0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a1015d5e0 0x7f7a10144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a10145bf0 0x7f7a101442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 --2- 172.21.15.37:0/767263898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a10071160 0x7f7a101f8260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 >> 172.21.15.37:0/767263898 conn(0x7f7a1006fc80 msgr2=0x7f7a10142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 shutdown_connections 2024-09-21T06:28:51.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:51.681+0000 7f7a15600640 1 -- 172.21.15.37:0/767263898 wait complete. 2024-09-21T06:28:51.707 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T06:28:51.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:51.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T06:28:51.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:28:51.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:51.767 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T06:28:51.803 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:51.803 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.838 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:51.839 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.914 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:28:51.914 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:28:51.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:28:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 -- 172.21.15.37:0/2800795150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 msgr2=0x7fdfbc0b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/2800795150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc0b9dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc08d460 tx=0x7fdfac02a5f0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 -- 172.21.15.37:0/2800795150 shutdown_connections 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/2800795150 >> v2:172.21.15.150:3300/0 conn(0x7fdfbc162940 0x7fdfbc154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/2800795150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc0b9dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 -- 172.21.15.37:0/2800795150 >> 172.21.15.37:0/2800795150 conn(0x7fdfbc0d42e0 msgr2=0x7fdfbc0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 -- 172.21.15.37:0/2800795150 shutdown_connections 2024-09-21T06:28:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.217+0000 7fdfc0c00640 1 -- 172.21.15.37:0/2800795150 wait complete. 2024-09-21T06:28:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.219+0000 7fdfc0c00640 1 Processor -- start 2024-09-21T06:28:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.219+0000 7fdfc0c00640 1 -- start start 2024-09-21T06:28:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfbc0cf7e0 0x7fdfbc14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc14eab0 0x7fdfbc161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc155770 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1555f0 con 0x7fdfbc14eab0 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfc0c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc1558f0 con 0x7fdfbc0cf7e0 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48612/0 (socket says 172.21.15.37:48612) 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfbaa00640 1 -- 172.21.15.37:0/968988551 learned_addr learned my addr 172.21.15.37:0/968988551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfbb400640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfbc0cf7e0 0x7fdfbc14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.221+0000 7fdfbbe00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc14eab0 0x7fdfbc161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 -- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfbc0cf7e0 msgr2=0x7fdfbc14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfbc0cf7e0 0x7fdfbc14a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 -- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc14eab0 msgr2=0x7fdfbc161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc14eab0 0x7fdfbc161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 -- 172.21.15.37:0/968988551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc14f290 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfbaa00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c340 tx=0x7fdfa800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8017070 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa80053c0 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc14f570 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003850 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.222+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1f6560 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.223+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc0b9dd0 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.225+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfa801a070 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.225+0000 7fdfb3400640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 0x7fdf9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.226+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7fdfa809a370 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.226+0000 7fdfbb400640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 0x7fdf9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.227+0000 7fdfbb400640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 0x7fdf9c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfac033000 tx=0x7fdfac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:52.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.230+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfa8061dc0 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.439+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfbc14efb0 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.439+0000 7fdfb3400640 1 -- 172.21.15.37:0/968988551 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fdfa80667f0 con 0x7fdfbc0d2ba0 2024-09-21T06:28:52.442 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 msgr2=0x7fdf9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 0x7fdf9c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfac033000 tx=0x7fdfac004730 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 msgr2=0x7fdfbc161590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c340 tx=0x7fdfa800c800 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 shutdown_connections 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdf9c078b90 0x7fdf9c07b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfbc0d2ba0 0x7fdfbc161590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc14eab0 0x7fdfbc161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 --2- 172.21.15.37:0/968988551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfbc0cf7e0 0x7fdfbc14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 >> 172.21.15.37:0/968988551 conn(0x7fdfbc0d42e0 msgr2=0x7fdfbc15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:52.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 shutdown_connections 2024-09-21T06:28:52.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.442+0000 7fdfc0c00640 1 -- 172.21.15.37:0/968988551 wait complete. 2024-09-21T06:28:52.469 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T06:28:52.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:52.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T06:28:52.533 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:28:52.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:28:52.533 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T06:28:52.571 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:28:52.571 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 -- 172.21.15.37:0/1563069697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c074cb0 msgr2=0x7f793c073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 --2- 172.21.15.37:0/1563069697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c074cb0 0x7f793c073dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f792c009830 tx=0x7f792c02a520 comp rx=0 tx=0).stop 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 -- 172.21.15.37:0/1563069697 shutdown_connections 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 --2- 172.21.15.37:0/1563069697 >> v2:172.21.15.150:3300/0 conn(0x7f793c0cbc10 0x7f793c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 --2- 172.21.15.37:0/1563069697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c074cb0 0x7f793c073dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 -- 172.21.15.37:0/1563069697 >> 172.21.15.37:0/1563069697 conn(0x7f793c0cca40 msgr2=0x7f793c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 -- 172.21.15.37:0/1563069697 shutdown_connections 2024-09-21T06:28:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.798+0000 7f7942c00640 1 -- 172.21.15.37:0/1563069697 wait complete. 2024-09-21T06:28:52.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.801+0000 7f7942c00640 1 Processor -- start 2024-09-21T06:28:52.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.801+0000 7f7942c00640 1 -- start start 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c06ddf0 0x7f793c1fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f793c0cbc10 0x7f793c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f793c06f360 con 0x7f793c074cb0 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f793c06f1e0 con 0x7f793c0cbc10 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.802+0000 7f7942c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f793c06f4e0 con 0x7f793c06ddf0 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48622/0 (socket says 172.21.15.37:48622) 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 -- 172.21.15.37:0/3705999957 learned_addr learned my addr 172.21.15.37:0/3705999957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793be00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c06ddf0 0x7f793c1fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f7940c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f793c0cbc10 0x7f793c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 -- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c06ddf0 msgr2=0x7f793c1fa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c06ddf0 0x7f793c1fa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 -- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f793c0cbc10 msgr2=0x7f793c152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f793c0cbc10 0x7f793c152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 -- 172.21.15.37:0/3705999957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1fa9e0 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.803+0000 7f793b400640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f792800e9f0 tx=0x7f792800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.804+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792800cd80 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.804+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7928004020 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.804+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c154ce0 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.804+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004470 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.804+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c151400 con 0x7f793c074cb0 2024-09-21T06:28:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.805+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c06fa00 con 0x7f793c074cb0 2024-09-21T06:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.806+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f79280035f0 con 0x7f793c074cb0 2024-09-21T06:28:52.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.807+0000 7f7933e00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 0x7f791c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:52.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.807+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f792809ba40 con 0x7f793c074cb0 2024-09-21T06:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.808+0000 7f793be00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 0x7f791c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.808+0000 7f793be00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 0x7f791c07b0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f792c034000 tx=0x7f792c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:52.812+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7928063380 con 0x7f793c074cb0 2024-09-21T06:28:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.051+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f793c074050 con 0x7f793c074cb0 2024-09-21T06:28:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.052+0000 7f7933e00640 1 -- 172.21.15.37:0/3705999957 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7f7928067db0 con 0x7f793c074cb0 2024-09-21T06:28:53.054 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 msgr2=0x7f791c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 0x7f791c07b0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f792c034000 tx=0x7f792c004730 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 msgr2=0x7f793c1f0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f792800e9f0 tx=0x7f792800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 shutdown_connections 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f791c078bc0 0x7f791c07b0d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f793c074cb0 0x7f793c1f0440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f793c0cbc10 0x7f793c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 --2- 172.21.15.37:0/3705999957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f793c06ddf0 0x7f793c1fa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 >> 172.21.15.37:0/3705999957 conn(0x7f793c0cca40 msgr2=0x7f793c15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 shutdown_connections 2024-09-21T06:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.054+0000 7f7942c00640 1 -- 172.21.15.37:0/3705999957 wait complete. 2024-09-21T06:28:53.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.308+0000 7f20cbc00640 1 -- 172.21.15.37:0/2153590398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c406e330 msgr2=0x7f20c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.308+0000 7f20cbc00640 1 --2- 172.21.15.37:0/2153590398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c406e330 0x7f20c406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005fd0 tx=0x7f20bc02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.308+0000 7f20cbc00640 1 -- 172.21.15.37:0/2153590398 shutdown_connections 2024-09-21T06:28:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.308+0000 7f20cbc00640 1 --2- 172.21.15.37:0/2153590398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 0x7f20c406ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.308+0000 7f20cbc00640 1 --2- 172.21.15.37:0/2153590398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c406e330 0x7f20c406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.309+0000 7f20cbc00640 1 -- 172.21.15.37:0/2153590398 >> 172.21.15.37:0/2153590398 conn(0x7f20c40cca40 msgr2=0x7f20c40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.309+0000 7f20cbc00640 1 -- 172.21.15.37:0/2153590398 shutdown_connections 2024-09-21T06:28:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.309+0000 7f20cbc00640 1 -- 172.21.15.37:0/2153590398 wait complete. 2024-09-21T06:28:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.311+0000 7f20cbc00640 1 Processor -- start 2024-09-21T06:28:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.311+0000 7f20cbc00640 1 -- start start 2024-09-21T06:28:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 0x7f20c40c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4074cb0 0x7f20c40b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20c406f4a0 con 0x7f20c406e330 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20c406f320 con 0x7f20c4074cb0 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.312+0000 7f20cbc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f20c406f620 con 0x7f20c4073dd0 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48638/0 (socket says 172.21.15.37:48638) 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 -- 172.21.15.37:0/561636316 learned_addr learned my addr 172.21.15.37:0/561636316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c3e00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 0x7f20c40c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9c00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4074cb0 0x7f20c40b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 -- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 msgr2=0x7f20c40c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 0x7f20c40c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 -- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4074cb0 msgr2=0x7f20c40b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4074cb0 0x7f20c40b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 -- 172.21.15.37:0/561636316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40c84b0 con 0x7f20c406e330 2024-09-21T06:28:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.313+0000 7f20c9200640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20b800c3f0 tx=0x7f20b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:53.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.314+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8017070 con 0x7f20c406e330 2024-09-21T06:28:53.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.314+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b80053c0 con 0x7f20c406e330 2024-09-21T06:28:53.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.314+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8003850 con 0x7f20c406e330 2024-09-21T06:28:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.314+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c40c8790 con 0x7f20c406e330 2024-09-21T06:28:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.314+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c40c8d50 con 0x7f20c406e330 2024-09-21T06:28:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.315+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c406fb60 con 0x7f20c406e330 2024-09-21T06:28:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.316+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20b801a070 con 0x7f20c406e330 2024-09-21T06:28:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.317+0000 7f20c0c00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.317+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f20b809a480 con 0x7f20c406e330 2024-09-21T06:28:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.318+0000 7f20c3e00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.319+0000 7f20c3e00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 0x7f20a407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20b00096c0 tx=0x7f20b0009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:53.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.322+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20b8061e40 con 0x7f20c406e330 2024-09-21T06:28:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.559+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20c40ce0c0 con 0x7f20c406e330 2024-09-21T06:28:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.559+0000 7f20c0c00640 1 -- 172.21.15.37:0/561636316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7f20b8066870 con 0x7f20c406e330 2024-09-21T06:28:53.562 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:53.562 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 msgr2=0x7f20a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 0x7f20a407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20b00096c0 tx=0x7f20b0009260 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 msgr2=0x7f20c40b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20b800c3f0 tx=0x7f20b800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 shutdown_connections 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4074cb0 0x7f20c40b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4073dd0 0x7f20c40c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 --2- 172.21.15.37:0/561636316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406e330 0x7f20c40b57f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 >> 172.21.15.37:0/561636316 conn(0x7f20c40cca40 msgr2=0x7f20c4159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 shutdown_connections 2024-09-21T06:28:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.562+0000 7f20cbc00640 1 -- 172.21.15.37:0/561636316 wait complete. 2024-09-21T06:28:53.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 -- 172.21.15.37:0/1027041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a24155560 msgr2=0x7f7a240cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 --2- 172.21.15.37:0/1027041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a24155560 0x7f7a240cc8b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7a240723d0 tx=0x7f7a1002a560 comp rx=0 tx=0).stop 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 -- 172.21.15.37:0/1027041231 shutdown_connections 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 --2- 172.21.15.37:0/1027041231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a240ccdf0 0x7f7a240cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 --2- 172.21.15.37:0/1027041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a24155560 0x7f7a240cc8b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 --2- 172.21.15.37:0/1027041231 >> v2:172.21.15.150:3300/0 conn(0x7f7a240af000 0x7f7a2415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.810+0000 7f7a23200640 1 -- 172.21.15.37:0/1027041231 >> 172.21.15.37:0/1027041231 conn(0x7f7a2406f940 msgr2=0x7f7a240b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.811+0000 7f7a23200640 1 -- 172.21.15.37:0/1027041231 shutdown_connections 2024-09-21T06:28:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.811+0000 7f7a23200640 1 -- 172.21.15.37:0/1027041231 wait complete. 2024-09-21T06:28:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.813+0000 7f7a23200640 1 Processor -- start 2024-09-21T06:28:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.813+0000 7f7a23200640 1 -- start start 2024-09-21T06:28:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 0x7f7a241528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a24155560 0x7f7a24155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a240ce890 con 0x7f7a2415bb10 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a240ce710 con 0x7f7a240ccdf0 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.814+0000 7f7a23200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a240cea10 con 0x7f7a24155560 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48652/0 (socket says 172.21.15.37:48652) 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 -- 172.21.15.37:0/3701631769 learned_addr learned my addr 172.21.15.37:0/3701631769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a21e00640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 0x7f7a241528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a21400640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a24155560 0x7f7a24155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 -- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a24155560 msgr2=0x7f7a24155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a24155560 0x7f7a24155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 -- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 msgr2=0x7f7a241528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 0x7f7a241528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 -- 172.21.15.37:0/3701631769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a241568c0 con 0x7f7a2415bb10 2024-09-21T06:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a21e00640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 0x7f7a241528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.815+0000 7f7a22800640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00eab0 tx=0x7f7a1c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.816+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00cea0 con 0x7f7a2415bb10 2024-09-21T06:28:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.816+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a1c004b20 con 0x7f7a2415bb10 2024-09-21T06:28:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.816+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a24157af0 con 0x7f7a2415bb10 2024-09-21T06:28:53.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.816+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c005020 con 0x7f7a2415bb10 2024-09-21T06:28:53.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.816+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2415bf50 con 0x7f7a2415bb10 2024-09-21T06:28:53.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.817+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a24156630 con 0x7f7a2415bb10 2024-09-21T06:28:53.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.818+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a1c0051c0 con 0x7f7a2415bb10 2024-09-21T06:28:53.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.819+0000 7f7a1a000640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:53.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.819+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f7a1c09bd30 con 0x7f7a2415bb10 2024-09-21T06:28:53.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.820+0000 7f7a21e00640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:53.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.821+0000 7f7a21e00640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 0x7f7a0407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400bb40 tx=0x7f7a14005e70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:53.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:53.824+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a1c063d40 con 0x7f7a2415bb10 2024-09-21T06:28:54.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.021+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a240af000 con 0x7f7a2415bb10 2024-09-21T06:28:54.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.022+0000 7f7a1a000640 1 -- 172.21.15.37:0/3701631769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18756 (secure 0 0 0) 0x7f7a1c063640 con 0x7f7a2415bb10 2024-09-21T06:28:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:54.025 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:38.200188+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.24","mappings":[{"from":0,"to":7}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:54.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 msgr2=0x7f7a0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 0x7f7a0407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400bb40 tx=0x7f7a14005e70 comp rx=0 tx=0).stop 2024-09-21T06:28:54.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 msgr2=0x7f7a240b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00eab0 tx=0x7f7a1c00ef70 comp rx=0 tx=0).stop 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 shutdown_connections 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2415bb10 0x7f7a240b1030 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a24155560 0x7f7a24155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 --2- 172.21.15.37:0/3701631769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a240ccdf0 0x7f7a241528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 >> 172.21.15.37:0/3701631769 conn(0x7f7a2406f940 msgr2=0x7f7a24072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 shutdown_connections 2024-09-21T06:28:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.025+0000 7f7a23200640 1 -- 172.21.15.37:0/3701631769 wait complete. 2024-09-21T06:28:54.048 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:28:54.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:54.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 -- 172.21.15.37:0/3291144351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb14806e500 msgr2=0x7fb14815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3291144351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb14806e500 0x7fb14815ca00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb134009830 tx=0x7fb13402a540 comp rx=0 tx=0).stop 2024-09-21T06:28:54.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 -- 172.21.15.37:0/3291144351 shutdown_connections 2024-09-21T06:28:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3291144351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb14806e500 0x7fb14815ca00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 -- 172.21.15.37:0/3291144351 >> 172.21.15.37:0/3291144351 conn(0x7fb148073400 msgr2=0x7fb148073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 -- 172.21.15.37:0/3291144351 shutdown_connections 2024-09-21T06:28:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.275+0000 7fb14ca00640 1 -- 172.21.15.37:0/3291144351 wait complete. 2024-09-21T06:28:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.277+0000 7fb14ca00640 1 Processor -- start 2024-09-21T06:28:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.277+0000 7fb14ca00640 1 -- start start 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1480afb80 0x7fb1481e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1481e6430 0x7fb1480b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480cf640 con 0x7fb148073f20 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480cf4c0 con 0x7fb1481e6430 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb14ca00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb1480cf7c0 con 0x7fb1480afb80 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38752/0 (socket says 172.21.15.37:38752) 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 -- 172.21.15.37:0/3517649880 learned_addr learned my addr 172.21.15.37:0/3517649880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 -- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1480afb80 msgr2=0x7fb1481e5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1480afb80 0x7fb1481e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 -- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1481e6430 msgr2=0x7fb1480b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1481e6430 0x7fb1480b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.279+0000 7fb146800640 1 -- 172.21.15.37:0/3517649880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1481e6f30 con 0x7fb148073f20 2024-09-21T06:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb146800640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb138001220 tx=0x7fb13800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138016020 con 0x7fb148073f20 2024-09-21T06:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb138012920 con 0x7fb148073f20 2024-09-21T06:28:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1481e2cd0 con 0x7fb148073f20 2024-09-21T06:28:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138012db0 con 0x7fb148073f20 2024-09-21T06:28:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.280+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1481e3290 con 0x7fb148073f20 2024-09-21T06:28:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.281+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14815ca00 con 0x7fb148073f20 2024-09-21T06:28:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.283+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb138019020 con 0x7fb148073f20 2024-09-21T06:28:54.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.283+0000 7fb13ea00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 0x7fb12807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.284+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7fb13809ae50 con 0x7fb148073f20 2024-09-21T06:28:54.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.284+0000 7fb145e00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 0x7fb12807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.285+0000 7fb145e00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 0x7fb12807b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb134034000 tx=0x7fb1340046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:54.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.288+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb138020030 con 0x7fb148073f20 2024-09-21T06:28:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.525+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1480b0610 con 0x7fb148073f20 2024-09-21T06:28:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.526+0000 7fb13ea00640 1 -- 172.21.15.37:0/3517649880 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7fb138063020 con 0x7fb148073f20 2024-09-21T06:28:54.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:54.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 msgr2=0x7fb12807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 0x7fb12807b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb134034000 tx=0x7fb1340046c0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 msgr2=0x7fb1481f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb138001220 tx=0x7fb13800c940 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 shutdown_connections 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb128078b90 0x7fb12807b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1481e6430 0x7fb1480b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1480afb80 0x7fb1481e5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 --2- 172.21.15.37:0/3517649880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb148073f20 0x7fb1481f5800 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 >> 172.21.15.37:0/3517649880 conn(0x7fb148073400 msgr2=0x7fb1480af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:54.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 shutdown_connections 2024-09-21T06:28:54.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.528+0000 7fb14ca00640 1 -- 172.21.15.37:0/3517649880 wait complete. 2024-09-21T06:28:54.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:28:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.778+0000 7efc5ac00640 1 -- 172.21.15.37:0/565646210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 msgr2=0x7efc38005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.778+0000 7efc5ac00640 1 --2- 172.21.15.37:0/565646210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc38005240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c610 tx=0x7efc38005830 comp rx=0 tx=0).stop 2024-09-21T06:28:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.778+0000 7efc58c00640 1 -- 172.21.15.37:0/565646210 reap_dead start 2024-09-21T06:28:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.779+0000 7efc5ac00640 1 -- 172.21.15.37:0/565646210 shutdown_connections 2024-09-21T06:28:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.779+0000 7efc5ac00640 1 -- 172.21.15.37:0/565646210 >> 172.21.15.37:0/565646210 conn(0x7efc540cca40 msgr2=0x7efc540cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.779+0000 7efc5ac00640 1 -- 172.21.15.37:0/565646210 shutdown_connections 2024-09-21T06:28:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.779+0000 7efc5ac00640 1 -- 172.21.15.37:0/565646210 wait complete. 2024-09-21T06:28:54.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.781+0000 7efc5ac00640 1 Processor -- start 2024-09-21T06:28:54.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.781+0000 7efc5ac00640 1 -- start start 2024-09-21T06:28:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc540c54e0 0x7efc5406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5406f930 0x7efc5415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc540ce6b0 con 0x7efc38004d90 2024-09-21T06:28:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc540ce530 con 0x7efc5406f930 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.782+0000 7efc5ac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efc5406ddf0 con 0x7efc540c54e0 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38766/0 (socket says 172.21.15.37:38766) 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 -- 172.21.15.37:0/2671982446 learned_addr learned my addr 172.21.15.37:0/2671982446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc58c00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5406f930 0x7efc5415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53400640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc540c54e0 0x7efc5406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 -- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc540c54e0 msgr2=0x7efc5406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc540c54e0 0x7efc5406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 -- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5406f930 msgr2=0x7efc5415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5406f930 0x7efc5415cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 -- 172.21.15.37:0/2671982446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5406fd40 con 0x7efc38004d90 2024-09-21T06:28:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.783+0000 7efc53e00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efc44003ef0 tx=0x7efc44001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.784+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44015070 con 0x7efc38004d90 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.784+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc44011900 con 0x7efc38004d90 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.784+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc540c5ce0 con 0x7efc38004d90 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.784+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44011d50 con 0x7efc38004d90 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.784+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5415df00 con 0x7efc38004d90 2024-09-21T06:28:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.785+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5406e350 con 0x7efc38004d90 2024-09-21T06:28:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.786+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc44018070 con 0x7efc38004d90 2024-09-21T06:28:54.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.787+0000 7efc4be00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 0x7efc3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:54.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.787+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7efc4409be80 con 0x7efc38004d90 2024-09-21T06:28:54.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.788+0000 7efc53400640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 0x7efc3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.789+0000 7efc53400640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 0x7efc3407b0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc540cf340 tx=0x7efc40005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:54.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:54.791+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc5406fd40 con 0x7efc38004d90 2024-09-21T06:28:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.025+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc540a13c0 con 0x7efc38004d90 2024-09-21T06:28:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.025+0000 7efc4be00640 1 -- 172.21.15.37:0/2671982446 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5225 (secure 0 0 0) 0x7efc44063f20 con 0x7efc38004d90 2024-09-21T06:28:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:55.028 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-21T06:28:18:960515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2831473651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2831473651},{"type":"v1","addr":"172.21.15.37:6835","nonce":2831473651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1432974925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1432974925},{"type":"v1","addr":"172.21.15.150:6836","nonce":1432974925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2561906702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2561906702},{"type":"v1","addr":"172.21.15.37:6833","nonce":2561906702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:28:00.716481+0000","modified":"2024-09-21T06:28:18.477528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4399":{"gid":4399,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/2075485628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2075485628},{"type":"v1","addr":"172.21.15.150:6837","nonce":2075485628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4399,"qdb_cluster":[4399]},"id":1}]} 2024-09-21T06:28:55.028 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-21T06:28:55.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 msgr2=0x7efc3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 0x7efc3407b0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc540cf340 tx=0x7efc40005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 msgr2=0x7efc540c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efc44003ef0 tx=0x7efc44001800 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 shutdown_connections 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc34078bc0 0x7efc3407b0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5406f930 0x7efc5415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc540c54e0 0x7efc5406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 --2- 172.21.15.37:0/2671982446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc38004d90 0x7efc540c2a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 >> 172.21.15.37:0/2671982446 conn(0x7efc540cca40 msgr2=0x7efc54073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 shutdown_connections 2024-09-21T06:28:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.028+0000 7efc5ac00640 1 -- 172.21.15.37:0/2671982446 wait complete. 2024-09-21T06:28:55.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:28:55.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 -- 172.21.15.37:0/2702651855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 msgr2=0x7fc4a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 --2- 172.21.15.37:0/2702651855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4a4003e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009b40 tx=0x7fc4a4005830 comp rx=0 tx=0).stop 2024-09-21T06:28:55.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c4a00640 1 -- 172.21.15.37:0/2702651855 reap_dead start 2024-09-21T06:28:55.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 -- 172.21.15.37:0/2702651855 shutdown_connections 2024-09-21T06:28:55.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 -- 172.21.15.37:0/2702651855 >> 172.21.15.37:0/2702651855 conn(0x7fc4c00b5140 msgr2=0x7fc4c00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:55.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 -- 172.21.15.37:0/2702651855 shutdown_connections 2024-09-21T06:28:55.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.279+0000 7fc4c5400640 1 -- 172.21.15.37:0/2702651855 wait complete. 2024-09-21T06:28:55.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.281+0000 7fc4c5400640 1 Processor -- start 2024-09-21T06:28:55.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.281+0000 7fc4c5400640 1 -- start start 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c00c7320 0x7fc4c00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c00c7730 0x7fc4c00ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00b06f0 con 0x7fc4a40039e0 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00af560 con 0x7fc4c00c7320 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4c5400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c00b0870 con 0x7fc4c00c7730 2024-09-21T06:28:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38788/0 (socket says 172.21.15.37:38788) 2024-09-21T06:28:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 -- 172.21.15.37:0/787311926 learned_addr learned my addr 172.21.15.37:0/787311926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 -- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c00c7730 msgr2=0x7fc4c00ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c00c7730 0x7fc4c00ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4b7e00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c00c7320 0x7fc4c00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 -- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c00c7320 msgr2=0x7fc4c00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c00c7320 0x7fc4c00cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.283+0000 7fc4bfe00640 1 -- 172.21.15.37:0/787311926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01f2a30 con 0x7fc4a40039e0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4bfe00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009a60 tx=0x7fc4b0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0043070 con 0x7fc4a40039e0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b0004120 con 0x7fc4a40039e0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c01f2d10 con 0x7fc4a40039e0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0038040 con 0x7fc4a40039e0 2024-09-21T06:28:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.284+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01f32d0 con 0x7fc4a40039e0 2024-09-21T06:28:55.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.285+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c00d0ca0 con 0x7fc4a40039e0 2024-09-21T06:28:55.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.287+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4b0046070 con 0x7fc4a40039e0 2024-09-21T06:28:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.288+0000 7fc4bcc00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 0x7fc49c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.288+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7fc4b00bb6d0 con 0x7fc4a40039e0 2024-09-21T06:28:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.288+0000 7fc4b7e00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 0x7fc49c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.289+0000 7fc4b7e00640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 0x7fc49c07b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01f0510 tx=0x7fc4a8006b00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.292+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc4b0083090 con 0x7fc4a40039e0 2024-09-21T06:28:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.489+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4c0069850 con 0x7fc4a40039e0 2024-09-21T06:28:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.491+0000 7fc4bcc00640 1 -- 172.21.15.37:0/787311926 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18756 (secure 0 0 0) 0x7fc4b0087ac0 con 0x7fc4a40039e0 2024-09-21T06:28:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:28:55.493 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:28:38.200188+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:27:58.550218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:27:59.683562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:28:03.246648+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.24","mappings":[{"from":0,"to":7}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:28:55.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 msgr2=0x7fc49c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 0x7fc49c07b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01f0510 tx=0x7fc4a8006b00 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 msgr2=0x7fc4c01f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009a60 tx=0x7fc4b0033670 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 shutdown_connections 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc49c078b90 0x7fc49c07b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c00c7730 0x7fc4c00ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c00c7320 0x7fc4c00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 --2- 172.21.15.37:0/787311926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a40039e0 0x7fc4c01f57a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 >> 172.21.15.37:0/787311926 conn(0x7fc4c00b5140 msgr2=0x7fc4c0070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 shutdown_connections 2024-09-21T06:28:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.493+0000 7fc4c5400640 1 -- 172.21.15.37:0/787311926 wait complete. 2024-09-21T06:28:55.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:28:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 -- 172.21.15.37:0/685421744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eac004800 msgr2=0x7f0eac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 --2- 172.21.15.37:0/685421744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eac004800 0x7f0eac004c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40afab0 tx=0x7f0eac005830 comp rx=0 tx=0).stop 2024-09-21T06:28:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0eca000640 1 -- 172.21.15.37:0/685421744 reap_dead start 2024-09-21T06:28:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 -- 172.21.15.37:0/685421744 shutdown_connections 2024-09-21T06:28:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 -- 172.21.15.37:0/685421744 >> 172.21.15.37:0/685421744 conn(0x7f0ec40b1b90 msgr2=0x7f0ec40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 -- 172.21.15.37:0/685421744 shutdown_connections 2024-09-21T06:28:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.744+0000 7f0ecc000640 1 -- 172.21.15.37:0/685421744 wait complete. 2024-09-21T06:28:55.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.746+0000 7f0ecc000640 1 Processor -- start 2024-09-21T06:28:55.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.746+0000 7f0ecc000640 1 -- start start 2024-09-21T06:28:55.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eac004800 0x7f0ec40b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ec40c6f20 0x7f0ec4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4071000 con 0x7f0ec41f98d0 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4070e80 con 0x7f0eac004800 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ecc000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec4163960 con 0x7f0ec40c6f20 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0eca000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0eca000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38804/0 (socket says 172.21.15.37:38804) 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0eca000640 1 -- 172.21.15.37:0/1914980740 learned_addr learned my addr 172.21.15.37:0/1914980740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ec8c00640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ec40c6f20 0x7f0ec4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.748+0000 7f0ec9600640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eac004800 0x7f0ec40b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 -- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ec40c6f20 msgr2=0x7f0ec4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ec40c6f20 0x7f0ec4159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 -- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eac004800 msgr2=0x7f0ec40b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eac004800 0x7f0ec40b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 -- 172.21.15.37:0/1914980740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec40b2f70 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0eca000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40caf40 tx=0x7f0ebc00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc003b80 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ebc003d20 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc004250 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec40a0b60 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.749+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec40b3210 con 0x7f0ec41f98d0 2024-09-21T06:28:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.750+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec4070070 con 0x7f0ec41f98d0 2024-09-21T06:28:55.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.752+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ebc0043f0 con 0x7f0ec41f98d0 2024-09-21T06:28:55.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.752+0000 7f0ec1600640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 0x7f0ea407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:55.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.753+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f0ebc09b890 con 0x7f0ec41f98d0 2024-09-21T06:28:55.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.753+0000 7f0ec9600640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 0x7f0ea407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.754+0000 7f0ec9600640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 0x7f0ea407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8000c00 tx=0x7f0eb8006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.757+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ebc01f030 con 0x7f0ec41f98d0 2024-09-21T06:28:56.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:55.998+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0ec40af7e0 con 0x7f0ec41f98d0 2024-09-21T06:28:56.001 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:28:55.999+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:28:56.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.001+0000 7f0ec1600640 1 -- 172.21.15.37:0/1914980740 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f0ebc063930 con 0x7f0ec41f98d0 2024-09-21T06:28:56.003 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:28:56.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 msgr2=0x7f0ea407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 0x7f0ea407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8000c00 tx=0x7f0eb8006b10 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 msgr2=0x7f0ec40b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40caf40 tx=0x7f0ebc00d9e0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 shutdown_connections 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0ea4078b90 0x7f0ea407b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41f98d0 0x7f0ec40b39b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ec40c6f20 0x7f0ec4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 --2- 172.21.15.37:0/1914980740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eac004800 0x7f0ec40b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 >> 172.21.15.37:0/1914980740 conn(0x7f0ec40b1b90 msgr2=0x7f0ec40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 shutdown_connections 2024-09-21T06:28:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.004+0000 7f0ecc000640 1 -- 172.21.15.37:0/1914980740 wait complete. 2024-09-21T06:28:56.018 INFO:tasks.ceph.mds.d.smithi150.stderr: -5097> 2024-09-21T06:28:05.806+0000 7f8fc3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:28:56.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:28:56.141 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 -- 172.21.15.37:0/1743712858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 msgr2=0x7f784c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 --2- 172.21.15.37:0/1743712858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c06e200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7838009830 tx=0x7f783802a520 comp rx=0 tx=0).stop 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 -- 172.21.15.37:0/1743712858 shutdown_connections 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 --2- 172.21.15.37:0/1743712858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 0x7f784c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 --2- 172.21.15.37:0/1743712858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c06e200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 -- 172.21.15.37:0/1743712858 >> 172.21.15.37:0/1743712858 conn(0x7f784c0b1b90 msgr2=0x7f784c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 -- 172.21.15.37:0/1743712858 shutdown_connections 2024-09-21T06:28:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.256+0000 7f7853600640 1 -- 172.21.15.37:0/1743712858 wait complete. 2024-09-21T06:28:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.258+0000 7f7853600640 1 Processor -- start 2024-09-21T06:28:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.259+0000 7f7853600640 1 -- start start 2024-09-21T06:28:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 0x7f784c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0cf850 con 0x7f784c156a50 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f784c161060 con 0x7f784c06ddf0 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7853600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0cf9d0 con 0x7f784c0cef00 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.260+0000 7f7851600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38818/0 (socket says 172.21.15.37:38818) 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 -- 172.21.15.37:0/277254970 learned_addr learned my addr 172.21.15.37:0/277254970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7850c00640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f784be00640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 0x7f784c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 -- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 msgr2=0x7f784c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 0x7f784c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 -- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 msgr2=0x7f784c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c1591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 -- 172.21.15.37:0/277254970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1f90b0 con 0x7f784c156a50 2024-09-21T06:28:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.261+0000 7f7851600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f784400efc0 tx=0x7f784400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.262+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844003970 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.262+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7844003c00 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.262+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c1f9390 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.262+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844004020 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.262+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1f9950 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.263+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c06e200 con 0x7f784c156a50 2024-09-21T06:28:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.264+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78440041c0 con 0x7f784c156a50 2024-09-21T06:28:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.265+0000 7f7848c00640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 0x7f782c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.265+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6705+0+0 (secure 0 0 0) 0x7f784409ba00 con 0x7f784c156a50 2024-09-21T06:28:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.265+0000 7f7850c00640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 0x7f782c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.266+0000 7f7850c00640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 0x7f782c07b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7840001720 tx=0x7f7840009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.270+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7844063360 con 0x7f784c156a50 2024-09-21T06:28:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:56.510+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f784c15a950 con 0x7f784c156a50 2024-09-21T06:28:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.003+0000 7f7848c00640 1 -- 172.21.15.37:0/277254970 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f7844067d90 con 0x7f784c156a50 2024-09-21T06:28:57.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 msgr2=0x7f782c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 0x7f782c07b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7840001720 tx=0x7f7840009040 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 msgr2=0x7f784c1f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f784400efc0 tx=0x7f784400c800 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 shutdown_connections 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f782c078b90 0x7f782c07b0a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f784c156a50 0x7f784c1f5b30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784c0cef00 0x7f784c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 --2- 172.21.15.37:0/277254970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f784c06ddf0 0x7f784c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 >> 172.21.15.37:0/277254970 conn(0x7f784c0b1b90 msgr2=0x7f784c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 shutdown_connections 2024-09-21T06:28:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.006+0000 7f7853600640 1 -- 172.21.15.37:0/277254970 wait complete. 2024-09-21T06:28:57.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 -- 172.21.15.37:0/2767829598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940cfdd0 msgr2=0x7f3e9415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 --2- 172.21.15.37:0/2767829598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940cfdd0 0x7f3e9415c680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005fd0 tx=0x7f3e8c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 -- 172.21.15.37:0/2767829598 shutdown_connections 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 --2- 172.21.15.37:0/2767829598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940cfdd0 0x7f3e9415c680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 --2- 172.21.15.37:0/2767829598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940719b0 0x7f3e941524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 -- 172.21.15.37:0/2767829598 >> 172.21.15.37:0/2767829598 conn(0x7f3e94073ce0 msgr2=0x7f3e940740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 -- 172.21.15.37:0/2767829598 shutdown_connections 2024-09-21T06:28:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.266+0000 7f3e9c600640 1 -- 172.21.15.37:0/2767829598 wait complete. 2024-09-21T06:28:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.268+0000 7f3e9c600640 1 Processor -- start 2024-09-21T06:28:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.269+0000 7f3e9c600640 1 -- start start 2024-09-21T06:28:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940719b0 0x7f3e941e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940cf9c0 0x7f3e941438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9415cf50 con 0x7f3e940cfdd0 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9415cdd0 con 0x7f3e940719b0 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9c600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9415d0d0 con 0x7f3e940cf9c0 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38820/0 (socket says 172.21.15.37:38820) 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e9a600640 1 -- 172.21.15.37:0/1956192953 learned_addr learned my addr 172.21.15.37:0/1956192953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.270+0000 7f3e99200640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940cf9c0 0x7f3e941438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e99c00640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940719b0 0x7f3e941e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 -- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940cf9c0 msgr2=0x7f3e941438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940cf9c0 0x7f3e941438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 -- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940719b0 msgr2=0x7f3e941e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940719b0 0x7f3e941e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 -- 172.21.15.37:0/1956192953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e941e50f0 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.271+0000 7f3e9a600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009cd0 tx=0x7f3e8c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.272+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031da0 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.272+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e8c003a60 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.272+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e941e5530 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.272+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0330d0 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.272+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e941e5af0 con 0x7f3e940cfdd0 2024-09-21T06:28:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.273+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9415d580 con 0x7f3e940cfdd0 2024-09-21T06:28:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.274+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e8c033270 con 0x7f3e940cfdd0 2024-09-21T06:28:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.275+0000 7f3e91600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 0x7f3e7407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.276+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6705+0+0 (secure 0 0 0) 0x7f3e8c0bc440 con 0x7f3e940cfdd0 2024-09-21T06:28:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.276+0000 7f3e99c00640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 0x7f3e7407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.277+0000 7f3e99c00640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 0x7f3e7407b0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e880095f0 tx=0x7f3e88005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.280+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e8c03f030 con 0x7f3e940cfdd0 2024-09-21T06:28:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:57.476+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e941e52c0 con 0x7f3e940cfdd0 2024-09-21T06:28:58.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.239+0000 7f3e91600640 1 -- 172.21.15.37:0/1956192953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f3e8c083e30 con 0x7f3e940cfdd0 2024-09-21T06:28:58.241 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 msgr2=0x7f3e7407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 0x7f3e7407b0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e880095f0 tx=0x7f3e88005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 msgr2=0x7f3e941400d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009cd0 tx=0x7f3e8c0316f0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 shutdown_connections 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e74078bc0 0x7f3e7407b0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e940cfdd0 0x7f3e941400d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e940cf9c0 0x7f3e941438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 --2- 172.21.15.37:0/1956192953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e940719b0 0x7f3e941e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.242+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 >> 172.21.15.37:0/1956192953 conn(0x7f3e94073ce0 msgr2=0x7f3e94151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.243+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 shutdown_connections 2024-09-21T06:28:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.243+0000 7f3e9c600640 1 -- 172.21.15.37:0/1956192953 wait complete. 2024-09-21T06:28:58.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.515+0000 7f1a6f200640 1 -- 172.21.15.37:0/3307719415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 msgr2=0x7f1a54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.515+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3307719415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a54004c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a58001660 tx=0x7f1a54005830 comp rx=0 tx=0).stop 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.515+0000 7f1a6e800640 1 -- 172.21.15.37:0/3307719415 reap_dead start 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.515+0000 7f1a6f200640 1 -- 172.21.15.37:0/3307719415 shutdown_connections 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.515+0000 7f1a6f200640 1 -- 172.21.15.37:0/3307719415 >> 172.21.15.37:0/3307719415 conn(0x7f1a700b4010 msgr2=0x7f1a700b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.516+0000 7f1a6f200640 1 -- 172.21.15.37:0/3307719415 shutdown_connections 2024-09-21T06:28:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.516+0000 7f1a6f200640 1 -- 172.21.15.37:0/3307719415 wait complete. 2024-09-21T06:28:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.518+0000 7f1a6f200640 1 Processor -- start 2024-09-21T06:28:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.518+0000 7f1a6f200640 1 -- start start 2024-09-21T06:28:58.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.519+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:58.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.519+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a701f5990 0x7f1a700cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:58.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.519+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a700cbd30 0x7f1a700c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:58.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70071710 con 0x7f1a54004800 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70071590 con 0x7f1a700cbd30 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1a70071890 con 0x7f1a701f5990 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38842/0 (socket says 172.21.15.37:38842) 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 -- 172.21.15.37:0/3700903123 learned_addr learned my addr 172.21.15.37:0/3700903123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a65400640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a701f5990 0x7f1a700cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 -- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a701f5990 msgr2=0x7f1a700cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a701f5990 0x7f1a700cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 -- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a700cbd30 msgr2=0x7f1a700c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a700cbd30 0x7f1a700c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 -- 172.21.15.37:0/3700903123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7014fe50 con 0x7f1a54004800 2024-09-21T06:28:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.520+0000 7f1a6de00640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001d10 tx=0x7f1a600067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.521+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000fac0 con 0x7f1a54004800 2024-09-21T06:28:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.521+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a60016020 con 0x7f1a54004800 2024-09-21T06:28:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.521+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a70150130 con 0x7f1a54004800 2024-09-21T06:28:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.521+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60011430 con 0x7f1a54004800 2024-09-21T06:28:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.521+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a700c9220 con 0x7f1a54004800 2024-09-21T06:28:58.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.522+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a700cc690 con 0x7f1a54004800 2024-09-21T06:28:58.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.523+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a600115d0 con 0x7f1a54004800 2024-09-21T06:28:58.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.524+0000 7f1a66a00640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 0x7f1a4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:58.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.524+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6267+0+0 (secure 0 0 0) 0x7f1a6009beb0 con 0x7f1a54004800 2024-09-21T06:28:58.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.525+0000 7f1a65400640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 0x7f1a4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:58.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.526+0000 7f1a65400640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 0x7f1a4c07b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a58032000 tx=0x7f1a58001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.529+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a60020020 con 0x7f1a54004800 2024-09-21T06:28:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:58.726+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1a700cf940 con 0x7f1a54004800 2024-09-21T06:28:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.246+0000 7f1a66a00640 1 -- 172.21.15.37:0/3700903123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f1a60003e20 con 0x7f1a54004800 2024-09-21T06:28:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:28:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 msgr2=0x7f1a4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 0x7f1a4c07b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a58032000 tx=0x7f1a58001190 comp rx=0 tx=0).stop 2024-09-21T06:28:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 msgr2=0x7f1a700cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001d10 tx=0x7f1a600067a0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 shutdown_connections 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a4c078b90 0x7f1a4c07b0a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a700cbd30 0x7f1a700c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a701f5990 0x7f1a700cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 --2- 172.21.15.37:0/3700903123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a54004800 0x7f1a700cab10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 >> 172.21.15.37:0/3700903123 conn(0x7f1a700b4010 msgr2=0x7f1a70143c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 shutdown_connections 2024-09-21T06:28:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.249+0000 7f1a6f200640 1 -- 172.21.15.37:0/3700903123 wait complete. 2024-09-21T06:28:59.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 -- 172.21.15.37:0/1479541275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f640806ef30 msgr2=0x7f640806f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 --2- 172.21.15.37:0/1479541275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f640806ef30 0x7f640806f340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6408075dd0 tx=0x7f640002aa30 comp rx=0 tx=0).stop 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 -- 172.21.15.37:0/1479541275 shutdown_connections 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 --2- 172.21.15.37:0/1479541275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f640806ef30 0x7f640806f340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 --2- 172.21.15.37:0/1479541275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6408160640 0x7f6408160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 --2- 172.21.15.37:0/1479541275 >> v2:172.21.15.150:3300/0 conn(0x7f64080ca690 0x7f6408160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 -- 172.21.15.37:0/1479541275 >> 172.21.15.37:0/1479541275 conn(0x7f64080ccac0 msgr2=0x7f64080cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 -- 172.21.15.37:0/1479541275 shutdown_connections 2024-09-21T06:28:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.507+0000 7f640dc00640 1 -- 172.21.15.37:0/1479541275 wait complete. 2024-09-21T06:28:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.509+0000 7f640dc00640 1 Processor -- start 2024-09-21T06:28:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.509+0000 7f640dc00640 1 -- start start 2024-09-21T06:28:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 0x7f64080ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408160640 0x7f6408158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f640806fc10 con 0x7f64080ca690 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f640806fa90 con 0x7f6408160640 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f640dc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f640806fd90 con 0x7f640806ef30 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f63fe000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f63fe000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38860/0 (socket says 172.21.15.37:38860) 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f63fe000640 1 -- 172.21.15.37:0/2617247315 learned_addr learned my addr 172.21.15.37:0/2617247315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f6407400640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408160640 0x7f6408158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.511+0000 7f6406a00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 0x7f64080ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 -- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 msgr2=0x7f64080ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 0x7f64080ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 -- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408160640 msgr2=0x7f6408158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408160640 0x7f6408158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 -- 172.21.15.37:0/2617247315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64080cfbb0 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f6406a00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 0x7f64080ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63fe000640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d780 tx=0x7f63f000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0017070 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f000de90 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640815f990 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00099d0 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.512+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640815f6b0 con 0x7f64080ca690 2024-09-21T06:28:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.513+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640815f420 con 0x7f64080ca690 2024-09-21T06:28:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.515+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63f0009b70 con 0x7f64080ca690 2024-09-21T06:28:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.516+0000 7f63ff400640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 0x7f63e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.516+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f63f009b110 con 0x7f64080ca690 2024-09-21T06:28:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.516+0000 7f6406a00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 0x7f63e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.517+0000 7f6406a00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 0x7f63e407b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63f800bb40 tx=0x7f63f8005e70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:59.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.521+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63f0020020 con 0x7f64080ca690 2024-09-21T06:28:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:28:59.719+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f64080b5f20 con 0x7f64080ca690 2024-09-21T06:29:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.254+0000 7f63ff400640 1 -- 172.21.15.37:0/2617247315 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f63f0063450 con 0x7f64080ca690 2024-09-21T06:29:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' removed 2024-09-21T06:29:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 msgr2=0x7f63e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 0x7f63e407b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63f800bb40 tx=0x7f63f8005e70 comp rx=0 tx=0).stop 2024-09-21T06:29:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 msgr2=0x7f64080cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d780 tx=0x7f63f000dc40 comp rx=0 tx=0).stop 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 shutdown_connections 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63e4078bc0 0x7f63e407b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64080ca690 0x7f64080cf670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408160640 0x7f6408158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 --2- 172.21.15.37:0/2617247315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f640806ef30 0x7f64080ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 >> 172.21.15.37:0/2617247315 conn(0x7f64080ccac0 msgr2=0x7f64080b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 shutdown_connections 2024-09-21T06:29:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.257+0000 7f640dc00640 1 -- 172.21.15.37:0/2617247315 wait complete. 2024-09-21T06:29:00.288 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:29:00.288 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:29:00.289 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:29:00.289 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:29:00.289 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:29:00.289 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:29:00.289 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:29:00.289 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:29:00.289 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:29:00.289 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:29:00.290 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:29:00.290 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:29:00.290 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:29:00.290 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:29:00.290 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:29:00.290 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:29:00.313 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:29:00.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:29:00.315 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:29:00.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:29:00.316 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:29:00.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:29:00.317 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:29:00.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:29:00.359 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:29:00.359 DEBUG:teuthology.parallel:result is None 2024-09-21T06:29:00.360 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:29:00.360 DEBUG:teuthology.parallel:result is None 2024-09-21T06:29:00.361 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:29:00.361 DEBUG:teuthology.parallel:result is None 2024-09-21T06:29:00.363 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:29:00.363 DEBUG:teuthology.parallel:result is None 2024-09-21T06:29:00.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:29:00.557 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:29:00.557 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:29:00.558 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:29:00.558 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 -- 172.21.15.37:0/4188840179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec15fc50 msgr2=0x7fe2ec06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 --2- 172.21.15.37:0/4188840179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec15fc50 0x7fe2ec06ddf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009830 tx=0x7fe2e002a500 comp rx=0 tx=0).stop 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 -- 172.21.15.37:0/4188840179 shutdown_connections 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 --2- 172.21.15.37:0/4188840179 >> v2:172.21.15.150:3300/0 conn(0x7fe2ec06e330 0x7fe2ec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 --2- 172.21.15.37:0/4188840179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec15fc50 0x7fe2ec06ddf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 -- 172.21.15.37:0/4188840179 >> 172.21.15.37:0/4188840179 conn(0x7fe2ec0b1b90 msgr2=0x7fe2ec0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 -- 172.21.15.37:0/4188840179 shutdown_connections 2024-09-21T06:29:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.596+0000 7fe2f4200640 1 -- 172.21.15.37:0/4188840179 wait complete. 2024-09-21T06:29:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.599+0000 7fe2f4200640 1 Processor -- start 2024-09-21T06:29:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.599+0000 7fe2f4200640 1 -- start start 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec06e330 0x7fe2ec0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2ec15fc50 0x7fe2ec1f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec070d20 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec070ba0 con 0x7fe2ec15fc50 2024-09-21T06:29:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.600+0000 7fe2f4200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec0734a0 con 0x7fe2ec06e330 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38878/0 (socket says 172.21.15.37:38878) 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 -- 172.21.15.37:0/116768067 learned_addr learned my addr 172.21.15.37:0/116768067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f1800640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec06e330 0x7fe2ec0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f2200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2ec15fc50 0x7fe2ec1f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 -- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec06e330 msgr2=0x7fe2ec0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec06e330 0x7fe2ec0cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 -- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2ec15fc50 msgr2=0x7fe2ec1f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2ec15fc50 0x7fe2ec1f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.601+0000 7fe2f0e00640 1 -- 172.21.15.37:0/116768067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec09e480 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2f0e00640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e9f0 tx=0x7fe2d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800cd80 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec09e760 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2d8004020 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004470 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.602+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec09ed20 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.603+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ec09e1d0 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.604+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2d80035f0 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.605+0000 7fe2e9600640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 0x7fe2cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.606+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7fe2d809b6f0 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.606+0000 7fe2f1800640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 0x7fe2cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:00.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.606+0000 7fe2f1800640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 0x7fe2cc07b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034000 tx=0x7fe2e0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.614+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2d8063530 con 0x7fe2ec0cf7d0 2024-09-21T06:29:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:00.832+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe2ec0739c0 con 0x7fe2ec0cf7d0 2024-09-21T06:29:01.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.274+0000 7fe2e9600640 1 -- 172.21.15.37:0/116768067 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7fe2d8067f60 con 0x7fe2ec0cf7d0 2024-09-21T06:29:01.277 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:29:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 msgr2=0x7fe2cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 0x7fe2cc07b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034000 tx=0x7fe2e0004730 comp rx=0 tx=0).stop 2024-09-21T06:29:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 msgr2=0x7fe2ec0cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e9f0 tx=0x7fe2d800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 shutdown_connections 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe2cc078b90 0x7fe2cc07b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ec0cf7d0 0x7fe2ec0cbfb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2ec15fc50 0x7fe2ec1f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 --2- 172.21.15.37:0/116768067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2ec06e330 0x7fe2ec0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 >> 172.21.15.37:0/116768067 conn(0x7fe2ec0b1b90 msgr2=0x7fe2ec0b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 shutdown_connections 2024-09-21T06:29:01.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.278+0000 7fe2f4200640 1 -- 172.21.15.37:0/116768067 wait complete. 2024-09-21T06:29:01.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 -- 172.21.15.37:0/2858785125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 msgr2=0x7f08d015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 --2- 172.21.15.37:0/2858785125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d015e8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009860 tx=0x7f08c402a540 comp rx=0 tx=0).stop 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 -- 172.21.15.37:0/2858785125 shutdown_connections 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 --2- 172.21.15.37:0/2858785125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d015ede0 0x7f08d00cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 --2- 172.21.15.37:0/2858785125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d015e8a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 -- 172.21.15.37:0/2858785125 >> 172.21.15.37:0/2858785125 conn(0x7f08d006f940 msgr2=0x7f08d00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 -- 172.21.15.37:0/2858785125 shutdown_connections 2024-09-21T06:29:01.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.542+0000 7f08d6800640 1 -- 172.21.15.37:0/2858785125 wait complete. 2024-09-21T06:29:01.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.544+0000 7f08d6800640 1 Processor -- start 2024-09-21T06:29:01.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.545+0000 7f08d6800640 1 -- start start 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d00ccd60 0x7f08d01f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d0143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0070e70 con 0x7f08d015ede0 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0070cf0 con 0x7f08d015b820 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d6800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f08d00ce4d0 con 0x7f08d00ccd60 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38882/0 (socket says 172.21.15.37:38882) 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.546+0000 7f08d5e00640 1 -- 172.21.15.37:0/439481326 learned_addr learned my addr 172.21.15.37:0/439481326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5400640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d00ccd60 0x7f08d01f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d4a00640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d0143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:01.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 -- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d00ccd60 msgr2=0x7f08d01f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d00ccd60 0x7f08d01f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 -- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 msgr2=0x7f08d0143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d0143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 -- 172.21.15.37:0/439481326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d0143ca0 con 0x7f08d015ede0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08d5e00640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08c800cad0 tx=0x7f08c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.547+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8016020 con 0x7f08d015ede0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.548+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08c800ad20 con 0x7f08d015ede0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.548+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8012680 con 0x7f08d015ede0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.548+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d01f9640 con 0x7f08d015ede0 2024-09-21T06:29:01.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.548+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d0153460 con 0x7f08d015ede0 2024-09-21T06:29:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.549+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d01f9b60 con 0x7f08d015ede0 2024-09-21T06:29:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.550+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08c8019020 con 0x7f08d015ede0 2024-09-21T06:29:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.551+0000 7f08cd600640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 0x7f08b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.551+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f08c809b130 con 0x7f08d015ede0 2024-09-21T06:29:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.551+0000 7f08d5400640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 0x7f08b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.552+0000 7f08d5400640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 0x7f08b007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08c4034000 tx=0x7f08c4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:01.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.556+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08d01f9b60 con 0x7f08d015ede0 2024-09-21T06:29:01.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.826+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f08d00ce9f0 con 0x7f08d015ede0 2024-09-21T06:29:01.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.826+0000 7f08cd600640 1 -- 172.21.15.37:0/439481326 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f08c8020020 con 0x7f08d015ede0 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:29:01.830 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:29:01.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:29:01.834 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.836 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.838 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:29:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mds", 2024-09-21T06:29:01.840 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5m2JxoOxAA/jawHpMbXKW8G16gTCpQGA==", 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5mfq5oOxAAxx9l+NssLmmiT9PT8dn/Qw==", 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-osd", 2024-09-21T06:29:01.841 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5mZ8BoOxAA5RqY8GxlRjgN1wwh59rxxQ==", 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5mqtBoOxAAJnT+wvBHGbiMPtghyXMucg==", 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T06:29:01.842 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5mV+FoOxAAhT3as5DeY+n8gOjrbLJFfA==", 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTZ+5m3/FoOxAArt/EdNApHjYsSAnRN2S/9A==", 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:29:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:29:01.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.845 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:01.845 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:29:01.845 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 msgr2=0x7f08b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 0x7f08b007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08c4034000 tx=0x7f08c4004730 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 msgr2=0x7f08d0157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08c800cad0 tx=0x7f08c800cf90 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 shutdown_connections 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f08b0078b90 0x7f08b007b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d015ede0 0x7f08d0157430 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08d015b820 0x7f08d0143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 --2- 172.21.15.37:0/439481326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08d00ccd60 0x7f08d01f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 >> 172.21.15.37:0/439481326 conn(0x7f08d006f940 msgr2=0x7f08d00cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 shutdown_connections 2024-09-21T06:29:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:01.829+0000 7f08d6800640 1 -- 172.21.15.37:0/439481326 wait complete. 2024-09-21T06:29:01.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 -- 172.21.15.37:0/1004847457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e006e330 msgr2=0x7fb8e00cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/1004847457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e006e330 0x7fb8e00cdc70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b2c70 tx=0x7fb8d802aa30 comp rx=0 tx=0).stop 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 -- 172.21.15.37:0/1004847457 shutdown_connections 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/1004847457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e006e330 0x7fb8e00cdc70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/1004847457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0074b60 0x7fb8e006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/1004847457 >> v2:172.21.15.150:3300/0 conn(0x7fb8e0156a50 0x7fb8e0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.078+0000 7fb8e5c00640 1 -- 172.21.15.37:0/1004847457 >> 172.21.15.37:0/1004847457 conn(0x7fb8e00b1b90 msgr2=0x7fb8e00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.079+0000 7fb8e5c00640 1 -- 172.21.15.37:0/1004847457 shutdown_connections 2024-09-21T06:29:02.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.079+0000 7fb8e5c00640 1 -- 172.21.15.37:0/1004847457 wait complete. 2024-09-21T06:29:02.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.081+0000 7fb8e5c00640 1 Processor -- start 2024-09-21T06:29:02.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.081+0000 7fb8e5c00640 1 -- start start 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e006e330 0x7fb8e01f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0156a50 0x7fb8e00bcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00ce330 con 0x7fb8e0074b60 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00ce1b0 con 0x7fb8e006e330 2024-09-21T06:29:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.082+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00ce4b0 con 0x7fb8e0156a50 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8dea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e006e330 0x7fb8e01f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38904/0 (socket says 172.21.15.37:38904) 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 -- 172.21.15.37:0/4229926174 learned_addr learned my addr 172.21.15.37:0/4229926174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8df400640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0156a50 0x7fb8e00bcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 -- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0156a50 msgr2=0x7fb8e00bcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0156a50 0x7fb8e00bcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 -- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e006e330 msgr2=0x7fb8e01f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e006e330 0x7fb8e01f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 -- 172.21.15.37:0/4229926174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00b4730 con 0x7fb8e0074b60 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.083+0000 7fb8de000640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00e7c0 tx=0x7fb8cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.084+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00ccc0 con 0x7fb8e0074b60 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.084+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8cc00ce60 con 0x7fb8e0074b60 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.084+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e00c70d0 con 0x7fb8e0074b60 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.084+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc005280 con 0x7fb8e0074b60 2024-09-21T06:29:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.084+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e00c7690 con 0x7fb8e0074b60 2024-09-21T06:29:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.085+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e00b44a0 con 0x7fb8e0074b60 2024-09-21T06:29:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.086+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8cc005420 con 0x7fb8e0074b60 2024-09-21T06:29:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.087+0000 7fb8d6a00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 0x7fb8c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.087+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fb8cc09b430 con 0x7fb8e0074b60 2024-09-21T06:29:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.088+0000 7fb8dea00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 0x7fb8c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.088+0000 7fb8dea00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 0x7fb8c007b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0010e60 tx=0x7fb8d0012040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.092+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8cc0633d0 con 0x7fb8e0074b60 2024-09-21T06:29:02.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.366+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fb8e00ce8d0 con 0x7fb8e0074b60 2024-09-21T06:29:02.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.369+0000 7fb8d6a00640 1 -- 172.21.15.37:0/4229926174 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fb8cc067e00 con 0x7fb8e0074b60 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 msgr2=0x7fb8c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 0x7fb8c007b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0010e60 tx=0x7fb8d0012040 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 msgr2=0x7fb8e00bc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00e7c0 tx=0x7fb8cc00ec80 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 shutdown_connections 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb8c0078b90 0x7fb8c007b0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e0074b60 0x7fb8e00bc610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8e0156a50 0x7fb8e00bcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 --2- 172.21.15.37:0/4229926174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e006e330 0x7fb8e01f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.372+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 >> 172.21.15.37:0/4229926174 conn(0x7fb8e00b1b90 msgr2=0x7fb8e00ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:02.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.373+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 shutdown_connections 2024-09-21T06:29:02.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.373+0000 7fb8e5c00640 1 -- 172.21.15.37:0/4229926174 wait complete. 2024-09-21T06:29:02.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T06:29:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 -- 172.21.15.37:0/2011343014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee00cef00 msgr2=0x7fdee00cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 --2- 172.21.15.37:0/2011343014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee00cef00 0x7fdee00cf310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fded8005fd0 tx=0x7fded802aa90 comp rx=0 tx=0).stop 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 -- 172.21.15.37:0/2011343014 shutdown_connections 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 --2- 172.21.15.37:0/2011343014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee00cef00 0x7fdee00cf310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 --2- 172.21.15.37:0/2011343014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee006ddf0 0x7fdee006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 -- 172.21.15.37:0/2011343014 >> 172.21.15.37:0/2011343014 conn(0x7fdee00b1b90 msgr2=0x7fdee00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 -- 172.21.15.37:0/2011343014 shutdown_connections 2024-09-21T06:29:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.622+0000 7fdee5000640 1 -- 172.21.15.37:0/2011343014 wait complete. 2024-09-21T06:29:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.625+0000 7fdee5000640 1 Processor -- start 2024-09-21T06:29:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.625+0000 7fdee5000640 1 -- start start 2024-09-21T06:29:02.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdee006ddf0 0x7fdee0155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee00cef00 0x7fdee01f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00cf850 con 0x7fdee0156a50 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01610a0 con 0x7fdee006ddf0 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdee5000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00cf9d0 con 0x7fdee00cef00 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdede800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.626+0000 7fdede800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38920/0 (socket says 172.21.15.37:38920) 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 -- 172.21.15.37:0/1201999685 learned_addr learned my addr 172.21.15.37:0/1201999685 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdedd400640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee00cef00 0x7fdee01f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdedde00640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdee006ddf0 0x7fdee0155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 -- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee00cef00 msgr2=0x7fdee01f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee00cef00 0x7fdee01f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 -- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdee006ddf0 msgr2=0x7fdee0155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdee006ddf0 0x7fdee0155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 -- 172.21.15.37:0/1201999685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee009f5e0 con 0x7fdee0156a50 2024-09-21T06:29:02.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.627+0000 7fdede800640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fded8006010 tx=0x7fded8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:02.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.628+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8044070 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.628+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fded802e7f0 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.628+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee0156430 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.628+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80044b0 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.628+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01f95d0 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.629+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee006e200 con 0x7fdee0156a50 2024-09-21T06:29:02.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.630+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fded8047070 con 0x7fdee0156a50 2024-09-21T06:29:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.631+0000 7fded6000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 0x7fdec007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.631+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fded80bbac0 con 0x7fdee0156a50 2024-09-21T06:29:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.631+0000 7fdedde00640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 0x7fdec007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.632+0000 7fdedde00640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 0x7fdec007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fded0001720 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.636+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fded8032030 con 0x7fdee0156a50 2024-09-21T06:29:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.908+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fdee00b26b0 con 0x7fdee0156a50 2024-09-21T06:29:02.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.911+0000 7fded6000640 1 -- 172.21.15.37:0/1201999685 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fded80839d0 con 0x7fdee0156a50 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 msgr2=0x7fdec007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 0x7fdec007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fded0001720 tx=0x7fded0009040 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 msgr2=0x7fdee01f8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fded8006010 tx=0x7fded8003f10 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 shutdown_connections 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdec0078b90 0x7fdec007b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0156a50 0x7fdee01f8f40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdee00cef00 0x7fdee01f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 --2- 172.21.15.37:0/1201999685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdee006ddf0 0x7fdee0155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 >> 172.21.15.37:0/1201999685 conn(0x7fdee00b1b90 msgr2=0x7fdee00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 shutdown_connections 2024-09-21T06:29:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:02.913+0000 7fdee5000640 1 -- 172.21.15.37:0/1201999685 wait complete. 2024-09-21T06:29:02.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T06:29:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.165+0000 7f557a000640 1 -- 172.21.15.37:0/1063443716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55580052c0 msgr2=0x7f555800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.165+0000 7f557a000640 1 --2- 172.21.15.37:0/1063443716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55580052c0 0x7f555800b9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5574075bd0 tx=0x7f5558005830 comp rx=0 tx=0).stop 2024-09-21T06:29:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.165+0000 7f5573e00640 1 -- 172.21.15.37:0/1063443716 reap_dead start 2024-09-21T06:29:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.165+0000 7f557a000640 1 -- 172.21.15.37:0/1063443716 shutdown_connections 2024-09-21T06:29:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.166+0000 7f557a000640 1 -- 172.21.15.37:0/1063443716 >> 172.21.15.37:0/1063443716 conn(0x7f55740b1b90 msgr2=0x7f55740b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.166+0000 7f557a000640 1 -- 172.21.15.37:0/1063443716 shutdown_connections 2024-09-21T06:29:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.166+0000 7f557a000640 1 -- 172.21.15.37:0/1063443716 wait complete. 2024-09-21T06:29:03.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.168+0000 7f557a000640 1 Processor -- start 2024-09-21T06:29:03.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.168+0000 7f557a000640 1 -- start start 2024-09-21T06:29:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55580052c0 0x7f55741f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557415c410 0x7f55741f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5574073320 con 0x7f55741f31e0 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55740731a0 con 0x7f557415c410 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f557a000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f55740734a0 con 0x7f55580052c0 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38948/0 (socket says 172.21.15.37:38948) 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 -- 172.21.15.37:0/254154161 learned_addr learned my addr 172.21.15.37:0/254154161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5573e00640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557415c410 0x7f55741f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5573400640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55580052c0 0x7f55741f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 -- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55580052c0 msgr2=0x7f55741f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55580052c0 0x7f55741f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 -- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557415c410 msgr2=0x7f55741f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557415c410 0x7f55741f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.170+0000 7f5572a00640 1 -- 172.21.15.37:0/254154161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557415c820 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f5572a00640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5560003ec0 tx=0x7f5560001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560015070 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5560011920 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560011d70 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741f3c30 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.171+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741586f0 con 0x7f55741f31e0 2024-09-21T06:29:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.172+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5574073a30 con 0x7f55741f31e0 2024-09-21T06:29:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.173+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5560010420 con 0x7f55741f31e0 2024-09-21T06:29:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.174+0000 7f556b400640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 0x7f555407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.174+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f556009b7f0 con 0x7f55741f31e0 2024-09-21T06:29:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.175+0000 7f5573400640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 0x7f555407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.175+0000 7f5573400640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 0x7f555407b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5564006c30 tx=0x7f55640014f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.179+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f557415c820 con 0x7f55741f31e0 2024-09-21T06:29:03.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.449+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f55741f39c0 con 0x7f55741f31e0 2024-09-21T06:29:03.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.452+0000 7f556b400640 1 -- 172.21.15.37:0/254154161 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f5560063de0 con 0x7f55741f31e0 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 msgr2=0x7f555407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 0x7f555407b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5564006c30 tx=0x7f55640014f0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 msgr2=0x7f557415bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5560003ec0 tx=0x7f5560001800 comp rx=0 tx=0).stop 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 shutdown_connections 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5554078b90 0x7f555407b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55741f31e0 0x7f557415bed0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557415c410 0x7f55741f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 --2- 172.21.15.37:0/254154161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55580052c0 0x7f55741f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 >> 172.21.15.37:0/254154161 conn(0x7f55740b1b90 msgr2=0x7f55740b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:03.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 shutdown_connections 2024-09-21T06:29:03.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.455+0000 7f557a000640 1 -- 172.21.15.37:0/254154161 wait complete. 2024-09-21T06:29:03.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 -- 172.21.15.37:0/631004055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4163550 msgr2=0x7f45d4165970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 --2- 172.21.15.37:0/631004055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4163550 0x7f45d4165970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005fd0 tx=0x7f45cc02bb60 comp rx=0 tx=0).stop 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 -- 172.21.15.37:0/631004055 shutdown_connections 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 --2- 172.21.15.37:0/631004055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4163550 0x7f45d4165970 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 -- 172.21.15.37:0/631004055 >> 172.21.15.37:0/631004055 conn(0x7f45d40b1b90 msgr2=0x7f45d40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 -- 172.21.15.37:0/631004055 shutdown_connections 2024-09-21T06:29:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.708+0000 7f45dc000640 1 -- 172.21.15.37:0/631004055 wait complete. 2024-09-21T06:29:03.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.710+0000 7f45dc000640 1 Processor -- start 2024-09-21T06:29:03.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.710+0000 7f45dc000640 1 -- start start 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d415ac20 0x7f45d40c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45d4163550 0x7f45d4072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4071000 con 0x7f45d406ddf0 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4070e80 con 0x7f45d415ac20 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45dc000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f45d4165eb0 con 0x7f45d4163550 2024-09-21T06:29:03.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45d9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45d9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38960/0 (socket says 172.21.15.37:38960) 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.712+0000 7f45d9600640 1 -- 172.21.15.37:0/3760277237 learned_addr learned my addr 172.21.15.37:0/3760277237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45da000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45d4163550 0x7f45d4072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d8c00640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d415ac20 0x7f45d40c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 -- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45d4163550 msgr2=0x7f45d4072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45d4163550 0x7f45d4072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 -- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d415ac20 msgr2=0x7f45d40c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d415ac20 0x7f45d40c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 -- 172.21.15.37:0/3760277237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d40c76c0 con 0x7f45d406ddf0 2024-09-21T06:29:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d9600640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c430 tx=0x7f45c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:03.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8017070 con 0x7f45d406ddf0 2024-09-21T06:29:03.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45c80053e0 con 0x7f45d406ddf0 2024-09-21T06:29:03.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d40c79a0 con 0x7f45d406ddf0 2024-09-21T06:29:03.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.713+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8003850 con 0x7f45d406ddf0 2024-09-21T06:29:03.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.714+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d40c7f60 con 0x7f45d406ddf0 2024-09-21T06:29:03.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.715+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d4070070 con 0x7f45d406ddf0 2024-09-21T06:29:03.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.716+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45c801a070 con 0x7f45d406ddf0 2024-09-21T06:29:03.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.717+0000 7f45d1600640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 0x7f45b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:03.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.717+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f45c809a0a0 con 0x7f45d406ddf0 2024-09-21T06:29:03.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.717+0000 7f45d8c00640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 0x7f45b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:03.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.718+0000 7f45d8c00640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 0x7f45b407b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45d40c8cd0 tx=0x7f45c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.721+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45c8062580 con 0x7f45d406ddf0 2024-09-21T06:29:03.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.991+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f45d40af7e0 con 0x7f45d406ddf0 2024-09-21T06:29:03.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.994+0000 7f45d1600640 1 -- 172.21.15.37:0/3760277237 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f45c8061e80 con 0x7f45d406ddf0 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 msgr2=0x7f45b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 0x7f45b407b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45d40c8cd0 tx=0x7f45c0009040 comp rx=0 tx=0).stop 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 msgr2=0x7f45d40a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c430 tx=0x7f45c800c8f0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 shutdown_connections 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f45b4078b90 0x7f45b407b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45d4163550 0x7f45d4072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d415ac20 0x7f45d40c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 --2- 172.21.15.37:0/3760277237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d406ddf0 0x7f45d40a0920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 >> 172.21.15.37:0/3760277237 conn(0x7f45d40b1b90 msgr2=0x7f45d4159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 shutdown_connections 2024-09-21T06:29:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:03.996+0000 7f45dc000640 1 -- 172.21.15.37:0/3760277237 wait complete. 2024-09-21T06:29:04.024 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 -- 172.21.15.37:0/3031767666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 msgr2=0x7fcbf006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/3031767666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf006ddf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005fd0 tx=0x7fcbd802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 -- 172.21.15.37:0/3031767666 shutdown_connections 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/3031767666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf006ddf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/3031767666 >> v2:172.21.15.150:3300/0 conn(0x7fcbf0156a50 0x7fcbf0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 -- 172.21.15.37:0/3031767666 >> 172.21.15.37:0/3031767666 conn(0x7fcbf00b1b90 msgr2=0x7fcbf00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 -- 172.21.15.37:0/3031767666 shutdown_connections 2024-09-21T06:29:04.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.250+0000 7fcbf6c00640 1 -- 172.21.15.37:0/3031767666 wait complete. 2024-09-21T06:29:04.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.252+0000 7fcbf6c00640 1 Processor -- start 2024-09-21T06:29:04.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.253+0000 7fcbf6c00640 1 -- start start 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf006e330 0x7fcbf0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf00afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf00ce500 con 0x7fcbf0156a50 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf00ce380 con 0x7fcbf006e330 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf00ce680 con 0x7fcbf0074b60 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49792/0 (socket says 172.21.15.37:49792) 2024-09-21T06:29:04.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.254+0000 7fcbf4c00640 1 -- 172.21.15.37:0/1764368098 learned_addr learned my addr 172.21.15.37:0/1764368098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbe7e00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf00afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbefe00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf006e330 0x7fcbf0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 -- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 msgr2=0x7fcbf00afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf00afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 -- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf006e330 msgr2=0x7fcbf0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf006e330 0x7fcbf0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 -- 172.21.15.37:0/1764368098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf00b50f0 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbf4c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c7b0 tx=0x7fcbe800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.255+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8003970 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.256+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbe8003b30 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.256+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800a850 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.256+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf01f3cf0 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.256+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf01f42b0 con 0x7fcbf0156a50 2024-09-21T06:29:04.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.257+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf00b2350 con 0x7fcbf0156a50 2024-09-21T06:29:04.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.258+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcbe800a9f0 con 0x7fcbf0156a50 2024-09-21T06:29:04.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.259+0000 7fcbecc00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 0x7fcbcc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.260+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fcbe809a870 con 0x7fcbf0156a50 2024-09-21T06:29:04.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.260+0000 7fcbefe00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 0x7fcbcc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.261+0000 7fcbefe00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 0x7fcbcc07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00095f0 tx=0x7fcbe0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:04.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.264+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcbf006e740 con 0x7fcbf0156a50 2024-09-21T06:29:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.535+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fcbf00c8340 con 0x7fcbf0156a50 2024-09-21T06:29:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.538+0000 7fcbecc00640 1 -- 172.21.15.37:0/1764368098 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fcbe8062780 con 0x7fcbf0156a50 2024-09-21T06:29:04.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 msgr2=0x7fcbcc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 0x7fcbcc07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00095f0 tx=0x7fcbe0005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:04.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 msgr2=0x7fcbf00aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c7b0 tx=0x7fcbe800cbb0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 shutdown_connections 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcbcc078b90 0x7fcbcc07b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf0156a50 0x7fcbf00aff40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbf0074b60 0x7fcbf00afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 --2- 172.21.15.37:0/1764368098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf006e330 0x7fcbf0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 >> 172.21.15.37:0/1764368098 conn(0x7fcbf00b1b90 msgr2=0x7fcbf0156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 shutdown_connections 2024-09-21T06:29:04.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.541+0000 7fcbf6c00640 1 -- 172.21.15.37:0/1764368098 wait complete. 2024-09-21T06:29:04.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T06:29:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 -- 172.21.15.37:0/2882349690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 msgr2=0x7fce0c073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 --2- 172.21.15.37:0/2882349690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c073f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009830 tx=0x7fcdf802a530 comp rx=0 tx=0).stop 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 -- 172.21.15.37:0/2882349690 shutdown_connections 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 --2- 172.21.15.37:0/2882349690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c073f20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 -- 172.21.15.37:0/2882349690 >> 172.21.15.37:0/2882349690 conn(0x7fce0c0d1190 msgr2=0x7fce0c073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 -- 172.21.15.37:0/2882349690 shutdown_connections 2024-09-21T06:29:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.793+0000 7fce12400640 1 -- 172.21.15.37:0/2882349690 wait complete. 2024-09-21T06:29:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.795+0000 7fce12400640 1 Processor -- start 2024-09-21T06:29:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.796+0000 7fce12400640 1 -- start start 2024-09-21T06:29:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c06f940 0x7fce0c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce0c0d00e0 0x7fce0c1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0707f0 con 0x7fce0c073b10 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c070670 con 0x7fce0c06f940 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce12400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c070970 con 0x7fce0c0d00e0 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce0be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce0be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49806/0 (socket says 172.21.15.37:49806) 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.797+0000 7fce0be00640 1 -- 172.21.15.37:0/3988144099 learned_addr learned my addr 172.21.15.37:0/3988144099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce11000640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c06f940 0x7fce0c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce11a00640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce0c0d00e0 0x7fce0c1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 -- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce0c0d00e0 msgr2=0x7fce0c1f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce0c0d00e0 0x7fce0c1f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 -- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c06f940 msgr2=0x7fce0c0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:04.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c06f940 0x7fce0c0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 -- 172.21.15.37:0/3988144099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c0af180 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.798+0000 7fce0be00640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009950 tx=0x7fcdf8033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.799+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8031d60 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.799+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdf8033070 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.799+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80334c0 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.799+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c0adbc0 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.799+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c0ae180 con 0x7fce0c073b10 2024-09-21T06:29:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.800+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c0ad930 con 0x7fce0c073b10 2024-09-21T06:29:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.801+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdf8033660 con 0x7fce0c073b10 2024-09-21T06:29:04.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.802+0000 7fce08c00640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 0x7fcdec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:04.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.803+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fcdf80bc160 con 0x7fce0c073b10 2024-09-21T06:29:04.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.803+0000 7fce11000640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 0x7fcdec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:04.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.804+0000 7fce11000640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 0x7fcdec07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce00000f60 tx=0x7fce00009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:04.807+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcdf803f030 con 0x7fce0c073b10 2024-09-21T06:29:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.076+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fce0c0cfe60 con 0x7fce0c073b10 2024-09-21T06:29:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.079+0000 7fce08c00640 1 -- 172.21.15.37:0/3988144099 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fce0c0cfe60 con 0x7fce0c073b10 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 msgr2=0x7fcdec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 0x7fcdec07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce00000f60 tx=0x7fce00009040 comp rx=0 tx=0).stop 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 msgr2=0x7fce0c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009950 tx=0x7fcdf8033d60 comp rx=0 tx=0).stop 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 shutdown_connections 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdec078b90 0x7fcdec07b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0c073b10 0x7fce0c15bf70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce0c0d00e0 0x7fce0c1f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 --2- 172.21.15.37:0/3988144099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c06f940 0x7fce0c0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 >> 172.21.15.37:0/3988144099 conn(0x7fce0c0d1190 msgr2=0x7fce0c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 shutdown_connections 2024-09-21T06:29:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.082+0000 7fce12400640 1 -- 172.21.15.37:0/3988144099 wait complete. 2024-09-21T06:29:05.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 -- 172.21.15.37:0/3634111461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc06e500 msgr2=0x7f63dc0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 --2- 172.21.15.37:0/3634111461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc06e500 0x7f63dc0cc020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f63dc076360 tx=0x7f63d402aac0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 -- 172.21.15.37:0/3634111461 shutdown_connections 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 --2- 172.21.15.37:0/3634111461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc06e500 0x7f63dc0cc020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 --2- 172.21.15.37:0/3634111461 >> v2:172.21.15.150:3300/0 conn(0x7f63dc074cb0 0x7f63dc073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 -- 172.21.15.37:0/3634111461 >> 172.21.15.37:0/3634111461 conn(0x7f63dc0cca40 msgr2=0x7f63dc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.333+0000 7f63e4000640 1 -- 172.21.15.37:0/3634111461 shutdown_connections 2024-09-21T06:29:05.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.334+0000 7f63e4000640 1 -- 172.21.15.37:0/3634111461 wait complete. 2024-09-21T06:29:05.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.336+0000 7f63e4000640 1 Processor -- start 2024-09-21T06:29:05.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.336+0000 7f63e4000640 1 -- start start 2024-09-21T06:29:05.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63dc074cb0 0x7f63dc09d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63dc151e00 0x7f63dc0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc06f3c0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc06f240 con 0x7f63dc074cb0 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.337+0000 7f63e4000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc06f540 con 0x7f63dc151e00 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49814/0 (socket says 172.21.15.37:49814) 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 -- 172.21.15.37:0/2393911622 learned_addr learned my addr 172.21.15.37:0/2393911622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e1600640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63dc074cb0 0x7f63dc09d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e2000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63dc151e00 0x7f63dc0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 -- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63dc151e00 msgr2=0x7f63dc0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63dc151e00 0x7f63dc0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 -- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63dc074cb0 msgr2=0x7f63dc09d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63dc074cb0 0x7f63dc09d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 -- 172.21.15.37:0/2393911622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc152210 con 0x7f63dc0cbc10 2024-09-21T06:29:05.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.338+0000 7f63e0c00640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c370 tx=0x7f63c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.339+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8017070 con 0x7f63dc0cbc10 2024-09-21T06:29:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.339+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63c80053c0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.339+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc0b44f0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.339+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8003850 con 0x7f63dc0cbc10 2024-09-21T06:29:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.339+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc0b4ab0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.340+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc06f9f0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.341+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63c801a070 con 0x7f63dc0cbc10 2024-09-21T06:29:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.342+0000 7f63d9600640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 0x7f63bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.342+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f63c8099e10 con 0x7f63dc0cbc10 2024-09-21T06:29:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.343+0000 7f63e1600640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 0x7f63bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.343+0000 7f63e1600640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 0x7f63bc07b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63dc15a730 tx=0x7f63d000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.347+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63c809d030 con 0x7f63dc0cbc10 2024-09-21T06:29:05.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.542+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f63dc0b4280 con 0x7f63dc0cbc10 2024-09-21T06:29:05.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.543+0000 7f63d9600640 1 -- 172.21.15.37:0/2393911622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7f63c8061db0 con 0x7f63dc0cbc10 2024-09-21T06:29:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:05.545 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:29:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:29:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:29:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:29:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:05.546 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 msgr2=0x7f63bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 0x7f63bc07b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63dc15a730 tx=0x7f63d000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 msgr2=0x7f63dc09e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c370 tx=0x7f63c800c830 comp rx=0 tx=0).stop 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 shutdown_connections 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63bc078bc0 0x7f63bc07b0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63dc0cbc10 0x7f63dc09e170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63dc151e00 0x7f63dc0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 --2- 172.21.15.37:0/2393911622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63dc074cb0 0x7f63dc09d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.545+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 >> 172.21.15.37:0/2393911622 conn(0x7f63dc0cca40 msgr2=0x7f63dc15dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:05.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.546+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 shutdown_connections 2024-09-21T06:29:05.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.546+0000 7f63e4000640 1 -- 172.21.15.37:0/2393911622 wait complete. 2024-09-21T06:29:05.569 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:29:05.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:29:05.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 -- 172.21.15.37:0/1640794506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7958004800 msgr2=0x7f7958004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 --2- 172.21.15.37:0/1640794506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7958004800 0x7f7958004c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f795c009b90 tx=0x7f7958005830 comp rx=0 tx=0).stop 2024-09-21T06:29:05.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7973400640 1 -- 172.21.15.37:0/1640794506 reap_dead start 2024-09-21T06:29:05.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 -- 172.21.15.37:0/1640794506 shutdown_connections 2024-09-21T06:29:05.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 -- 172.21.15.37:0/1640794506 >> 172.21.15.37:0/1640794506 conn(0x7f79740d2280 msgr2=0x7f79740d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:05.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 -- 172.21.15.37:0/1640794506 shutdown_connections 2024-09-21T06:29:05.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.800+0000 7f7979800640 1 -- 172.21.15.37:0/1640794506 wait complete. 2024-09-21T06:29:05.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.802+0000 7f7979800640 1 Processor -- start 2024-09-21T06:29:05.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.802+0000 7f7979800640 1 -- start start 2024-09-21T06:29:05.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7958004800 0x7f797415f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f797415d9d0 0x7f79741f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79740d0410 con 0x7f79741f4c80 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79740cd400 con 0x7f7958004800 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7979800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f797415c500 con 0x7f797415d9d0 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7973400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7973400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49824/0 (socket says 172.21.15.37:49824) 2024-09-21T06:29:05.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.804+0000 7f7973400640 1 -- 172.21.15.37:0/3974476281 learned_addr learned my addr 172.21.15.37:0/3974476281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:05.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7972a00640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7958004800 0x7f797415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f796a000640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f797415d9d0 0x7f79741f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 -- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f797415d9d0 msgr2=0x7f79741f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f797415d9d0 0x7f79741f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 -- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7958004800 msgr2=0x7f797415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7958004800 0x7f797415f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 -- 172.21.15.37:0/3974476281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797406def0 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7973400640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7974072480 tx=0x7f796c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c003d80 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f796c004b20 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79741f5450 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c004030 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.805+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741f5a10 con 0x7f79741f4c80 2024-09-21T06:29:05.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.806+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79741f51c0 con 0x7f79741f4c80 2024-09-21T06:29:05.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.808+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f796c0041d0 con 0x7f79741f4c80 2024-09-21T06:29:05.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.809+0000 7f796b400640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 0x7f795007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:05.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.809+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f796c09bbf0 con 0x7f79741f4c80 2024-09-21T06:29:05.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.809+0000 7f7972a00640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 0x7f795007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:05.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.810+0000 7f7972a00640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 0x7f795007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7964001eb0 tx=0x7f7964001360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:05.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:05.813+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f796c064150 con 0x7f79741f4c80 2024-09-21T06:29:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.010+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f79740ccae0 con 0x7f79741f4c80 2024-09-21T06:29:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.283+0000 7f796b400640 1 -- 172.21.15.37:0/3974476281 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7f796c063a50 con 0x7f79741f4c80 2024-09-21T06:29:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:29:06.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 msgr2=0x7f795007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 0x7f795007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7964001eb0 tx=0x7f7964001360 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 msgr2=0x7f7974071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7974072480 tx=0x7f796c00f790 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 shutdown_connections 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7950078bc0 0x7f795007b0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79741f4c80 0x7f7974071570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f797415d9d0 0x7f79741f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 --2- 172.21.15.37:0/3974476281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7958004800 0x7f797415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 >> 172.21.15.37:0/3974476281 conn(0x7f79740d2280 msgr2=0x7f79740b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 shutdown_connections 2024-09-21T06:29:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.286+0000 7f7979800640 1 -- 172.21.15.37:0/3974476281 wait complete. 2024-09-21T06:29:06.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 -- 172.21.15.37:0/1197569023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4074cb0 msgr2=0x7fa6c4073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 --2- 172.21.15.37:0/1197569023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4074cb0 0x7fa6c4073dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009830 tx=0x7fa6b402a520 comp rx=0 tx=0).stop 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 -- 172.21.15.37:0/1197569023 shutdown_connections 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 --2- 172.21.15.37:0/1197569023 >> v2:172.21.15.150:3300/0 conn(0x7fa6c406e500 0x7fa6c40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 --2- 172.21.15.37:0/1197569023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4074cb0 0x7fa6c4073dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 -- 172.21.15.37:0/1197569023 >> 172.21.15.37:0/1197569023 conn(0x7fa6c40cca40 msgr2=0x7fa6c40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 -- 172.21.15.37:0/1197569023 shutdown_connections 2024-09-21T06:29:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.551+0000 7fa6caa00640 1 -- 172.21.15.37:0/1197569023 wait complete. 2024-09-21T06:29:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.554+0000 7fa6caa00640 1 Processor -- start 2024-09-21T06:29:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.554+0000 7fa6caa00640 1 -- start start 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40cbc10 0x7fa6c4155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40a0db0 0x7fa6c4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c406f360 con 0x7fa6c4074cb0 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c406f1e0 con 0x7fa6c40a0db0 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.555+0000 7fa6caa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c406f4e0 con 0x7fa6c40cbc10 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49834/0 (socket says 172.21.15.37:49834) 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 -- 172.21.15.37:0/3643591592 learned_addr learned my addr 172.21.15.37:0/3643591592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6bbe00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40cbc10 0x7fa6c4155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c8a00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40a0db0 0x7fa6c4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 -- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40cbc10 msgr2=0x7fa6c4155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40cbc10 0x7fa6c4155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 -- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40a0db0 msgr2=0x7fa6c4155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40a0db0 0x7fa6c4155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 -- 172.21.15.37:0/3643591592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4156a50 con 0x7fa6c4074cb0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.556+0000 7fa6c3e00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4000c00 tx=0x7fa6b4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.557+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4005c90 con 0x7fa6c4074cb0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.557+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6b4031e50 con 0x7fa6c4074cb0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.557+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c4156d30 con 0x7fa6c4074cb0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.557+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4033310 con 0x7fa6c4074cb0 2024-09-21T06:29:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.557+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c40b3210 con 0x7fa6c4074cb0 2024-09-21T06:29:06.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.558+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c406fa00 con 0x7fa6c4074cb0 2024-09-21T06:29:06.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.559+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6b40334b0 con 0x7fa6c4074cb0 2024-09-21T06:29:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.560+0000 7fa6c0c00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 0x7fa6a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.560+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7fa6b40bbf60 con 0x7fa6c4074cb0 2024-09-21T06:29:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.561+0000 7fa6bbe00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 0x7fa6a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.562+0000 7fa6bbe00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 0x7fa6a007b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac007fd0 tx=0x7fa6ac001390 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.565+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6b403f030 con 0x7fa6c4074cb0 2024-09-21T06:29:06.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:06.764+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa6c406e450 con 0x7fa6c4074cb0 2024-09-21T06:29:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.293+0000 7fa6c0c00640 1 -- 172.21.15.37:0/3643591592 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7fa6b4083d10 con 0x7fa6c4074cb0 2024-09-21T06:29:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 msgr2=0x7fa6a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 0x7fa6a007b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac007fd0 tx=0x7fa6ac001390 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 msgr2=0x7fa6c40b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4000c00 tx=0x7fa6b4033c20 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 shutdown_connections 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa6a0078b90 0x7fa6a007b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40a0db0 0x7fa6c4155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40cbc10 0x7fa6c4155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 --2- 172.21.15.37:0/3643591592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4074cb0 0x7fa6c40b10c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 >> 172.21.15.37:0/3643591592 conn(0x7fa6c40cca40 msgr2=0x7fa6c41508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.296+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 shutdown_connections 2024-09-21T06:29:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.297+0000 7fa6caa00640 1 -- 172.21.15.37:0/3643591592 wait complete. 2024-09-21T06:29:07.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:29:07.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 -- 172.21.15.37:0/3234491463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68ac004800 msgr2=0x7f68ac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:07.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 --2- 172.21.15.37:0/3234491463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68ac004800 0x7f68ac004c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68c8071cc0 tx=0x7f68ac005830 comp rx=0 tx=0).stop 2024-09-21T06:29:07.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cd000640 1 -- 172.21.15.37:0/3234491463 reap_dead start 2024-09-21T06:29:07.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 -- 172.21.15.37:0/3234491463 shutdown_connections 2024-09-21T06:29:07.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 -- 172.21.15.37:0/3234491463 >> 172.21.15.37:0/3234491463 conn(0x7f68c80b1b90 msgr2=0x7f68c80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:07.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 -- 172.21.15.37:0/3234491463 shutdown_connections 2024-09-21T06:29:07.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.568+0000 7f68cf000640 1 -- 172.21.15.37:0/3234491463 wait complete. 2024-09-21T06:29:07.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.570+0000 7f68cf000640 1 Processor -- start 2024-09-21T06:29:07.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.570+0000 7f68cf000640 1 -- start start 2024-09-21T06:29:07.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac004800 0x7f68c81f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:07.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:07.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c80b44c0 0x7f68c80b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:07.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68c806df70 con 0x7f68c80ca860 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68c806ddf0 con 0x7f68ac004800 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cf000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f68c806e0f0 con 0x7f68c80b44c0 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49844/0 (socket says 172.21.15.37:49844) 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 -- 172.21.15.37:0/3277035902 learned_addr learned my addr 172.21.15.37:0/3277035902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68cd000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c80b44c0 0x7f68c80b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68c7e00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac004800 0x7f68c81f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 -- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c80b44c0 msgr2=0x7f68c80b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c80b44c0 0x7f68c80b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 -- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac004800 msgr2=0x7f68c81f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac004800 0x7f68c81f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:07.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.572+0000 7f68bfe00640 1 -- 172.21.15.37:0/3277035902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80b4e10 con 0x7f68c80ca860 2024-09-21T06:29:07.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68bfe00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68b0001eb0 tx=0x7f68b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:07.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0038040 con 0x7f68c80ca860 2024-09-21T06:29:07.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68b002b680 con 0x7f68c80ca860 2024-09-21T06:29:07.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c80caf70 con 0x7f68c80ca860 2024-09-21T06:29:07.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0003850 con 0x7f68c80ca860 2024-09-21T06:29:07.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.573+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81f6ad0 con 0x7f68c80ca860 2024-09-21T06:29:07.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.574+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c8074600 con 0x7f68c80ca860 2024-09-21T06:29:07.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.576+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68b0043020 con 0x7f68c80ca860 2024-09-21T06:29:07.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.577+0000 7f68c4c00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 0x7f68a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:07.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.577+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7f68b00bb7f0 con 0x7f68c80ca860 2024-09-21T06:29:07.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.577+0000 7f68c7e00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 0x7f68a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:07.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.578+0000 7f68c7e00640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 0x7f68a407b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009c10 tx=0x7f68b8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:07.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.580+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68b004a020 con 0x7f68c80ca860 2024-09-21T06:29:07.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:07.814+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f68c80cad00 con 0x7f68c80ca860 2024-09-21T06:29:08.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.295+0000 7f68c4c00640 1 -- 172.21.15.37:0/3277035902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f68c80b4e10 con 0x7f68c80ca860 2024-09-21T06:29:08.297 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 5 and data pool 6 2024-09-21T06:29:08.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.298+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 msgr2=0x7f68a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.298+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 0x7f68a407b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009c10 tx=0x7f68b8005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:08.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.298+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 msgr2=0x7f68c8154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.298+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68b0001eb0 tx=0x7f68b00042f0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 shutdown_connections 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68a4078b90 0x7f68a407b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68c80ca860 0x7f68c8154620 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c80b44c0 0x7f68c80b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 --2- 172.21.15.37:0/3277035902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac004800 0x7f68c81f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 >> 172.21.15.37:0/3277035902 conn(0x7f68c80b1b90 msgr2=0x7f68c815c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 shutdown_connections 2024-09-21T06:29:08.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.301+0000 7f68cf000640 1 -- 172.21.15.37:0/3277035902 wait complete. 2024-09-21T06:29:08.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 -- 172.21.15.37:0/3221381103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 msgr2=0x7f340c073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 --2- 172.21.15.37:0/3221381103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c073dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f340c09e110 tx=0x7f33fc02a5f0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 -- 172.21.15.37:0/3221381103 shutdown_connections 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 --2- 172.21.15.37:0/3221381103 >> v2:172.21.15.150:3300/0 conn(0x7f340c0cbc10 0x7f340c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 --2- 172.21.15.37:0/3221381103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c073dd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 -- 172.21.15.37:0/3221381103 >> 172.21.15.37:0/3221381103 conn(0x7f340c0cca40 msgr2=0x7f340c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 -- 172.21.15.37:0/3221381103 shutdown_connections 2024-09-21T06:29:08.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.584+0000 7f3412e00640 1 -- 172.21.15.37:0/3221381103 wait complete. 2024-09-21T06:29:08.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.587+0000 7f3412e00640 1 Processor -- start 2024-09-21T06:29:08.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.587+0000 7f3412e00640 1 -- start start 2024-09-21T06:29:08.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340c06ddf0 0x7f340c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:08.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:08.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 0x7f340c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:08.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f340c06f360 con 0x7f340c074cb0 2024-09-21T06:29:08.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f340c06f1e0 con 0x7f340c06ddf0 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f3412e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f340c06f4e0 con 0x7f340c0cbc10 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f340b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.588+0000 7f340b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49856/0 (socket says 172.21.15.37:49856) 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 -- 172.21.15.37:0/3723942534 learned_addr learned my addr 172.21.15.37:0/3723942534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340be00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340c06ddf0 0x7f340c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f3410e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 0x7f340c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 -- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 msgr2=0x7f340c1f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 0x7f340c1f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 -- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340c06ddf0 msgr2=0x7f340c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340c06ddf0 0x7f340c0b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 -- 172.21.15.37:0/3723942534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c158880 con 0x7f340c074cb0 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f3410e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 0x7f340c1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f340b400640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33f800e820 tx=0x7f33f800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.589+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f800ccc0 con 0x7f340c074cb0 2024-09-21T06:29:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.590+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f8010430 con 0x7f340c074cb0 2024-09-21T06:29:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.590+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c0b1330 con 0x7f340c074cb0 2024-09-21T06:29:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.590+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8010860 con 0x7f340c074cb0 2024-09-21T06:29:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.590+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c0b4730 con 0x7f340c074cb0 2024-09-21T06:29:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.591+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c06f990 con 0x7f340c074cb0 2024-09-21T06:29:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.592+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33f8010a00 con 0x7f340c074cb0 2024-09-21T06:29:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.593+0000 7f3403e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 0x7f33ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:08.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.594+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f33f809b7b0 con 0x7f340c074cb0 2024-09-21T06:29:08.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.594+0000 7f340be00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 0x7f33ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:08.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.595+0000 7f340be00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 0x7f33ec07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f33fc033000 tx=0x7f33fc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:08.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.598+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33f8063490 con 0x7f340c074cb0 2024-09-21T06:29:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.795+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f340c158a50 con 0x7f340c074cb0 2024-09-21T06:29:08.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.796+0000 7f3403e00640 1 -- 172.21.15.37:0/3723942534 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17027 (secure 0 0 0) 0x7f33f806c020 con 0x7f340c074cb0 2024-09-21T06:29:08.799 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:08.799 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:29:07.816109+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:29:06.023459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T06:29:06.776946+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 msgr2=0x7f33ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 0x7f33ec07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f33fc033000 tx=0x7f33fc004730 comp rx=0 tx=0).stop 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 msgr2=0x7f340c1f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33f800e820 tx=0x7f33f800ece0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 shutdown_connections 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f33ec078b90 0x7f33ec07b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c074cb0 0x7f340c1f3bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f340c0cbc10 0x7f340c1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 --2- 172.21.15.37:0/3723942534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340c06ddf0 0x7f340c0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 >> 172.21.15.37:0/3723942534 conn(0x7f340c0cca40 msgr2=0x7f340c159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 shutdown_connections 2024-09-21T06:29:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:08.799+0000 7f3412e00640 1 -- 172.21.15.37:0/3723942534 wait complete. 2024-09-21T06:29:08.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.049+0000 7fa697800640 1 -- 172.21.15.37:0/2548010003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900af420 msgr2=0x7fa69015e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.049+0000 7fa697800640 1 --2- 172.21.15.37:0/2548010003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900af420 0x7fa69015e7c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa68802a940 comp rx=0 tx=0).stop 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.049+0000 7fa697800640 1 -- 172.21.15.37:0/2548010003 shutdown_connections 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.049+0000 7fa697800640 1 --2- 172.21.15.37:0/2548010003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900af420 0x7fa69015e7c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.049+0000 7fa697800640 1 -- 172.21.15.37:0/2548010003 >> 172.21.15.37:0/2548010003 conn(0x7fa6900cca40 msgr2=0x7fa6900cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.050+0000 7fa697800640 1 -- 172.21.15.37:0/2548010003 shutdown_connections 2024-09-21T06:29:09.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.050+0000 7fa697800640 1 -- 172.21.15.37:0/2548010003 wait complete. 2024-09-21T06:29:09.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.052+0000 7fa697800640 1 Processor -- start 2024-09-21T06:29:09.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.052+0000 7fa697800640 1 -- start start 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6900af420 0x7fa690161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6900b5650 0x7fa6901f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900ce8a0 con 0x7fa6900b5240 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900ce720 con 0x7fa6900af420 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.053+0000 7fa697800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa6900cea20 con 0x7fa6900b5650 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49874/0 (socket says 172.21.15.37:49874) 2024-09-21T06:29:09.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 -- 172.21.15.37:0/4163761953 learned_addr learned my addr 172.21.15.37:0/4163761953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa695800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6900b5650 0x7fa6901f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa694e00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6900af420 0x7fa690161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 -- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6900b5650 msgr2=0x7fa6901f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6900b5650 0x7fa6901f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 -- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6900af420 msgr2=0x7fa690161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6900af420 0x7fa690161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.054+0000 7fa68fe00640 1 -- 172.21.15.37:0/4163761953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6900a11e0 con 0x7fa6900b5240 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa68fe00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00c400 tx=0x7fa67c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011790 con 0x7fa6900b5240 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa67c017070 con 0x7fa6900b5240 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa690160900 con 0x7fa6900b5240 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011cc0 con 0x7fa6900b5240 2024-09-21T06:29:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.055+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa690160ec0 con 0x7fa6900b5240 2024-09-21T06:29:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.056+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6900cb750 con 0x7fa6900b5240 2024-09-21T06:29:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.057+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa67c01c460 con 0x7fa6900b5240 2024-09-21T06:29:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.058+0000 7fa68cc00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 0x7fa67007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.058+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa67c09acb0 con 0x7fa6900b5240 2024-09-21T06:29:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.059+0000 7fa694e00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 0x7fa67007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.060+0000 7fa694e00640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 0x7fa67007b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa684009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.062+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa67c062870 con 0x7fa6900b5240 2024-09-21T06:29:09.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.263+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6900b5f40 con 0x7fa6900b5240 2024-09-21T06:29:09.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.264+0000 7fa68cc00640 1 -- 172.21.15.37:0/4163761953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17027 (secure 0 0 0) 0x7fa67c0672a0 con 0x7fa6900b5240 2024-09-21T06:29:09.267 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:09.267 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:29:07.816109+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:29:06.023459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T06:29:06.776946+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:29:09.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.266+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 msgr2=0x7fa67007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.266+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 0x7fa67007b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa684009040 comp rx=0 tx=0).stop 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.266+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 msgr2=0x7fa6901f83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.266+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00c400 tx=0x7fa67c00c8c0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 shutdown_connections 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa670078b90 0x7fa67007b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6900b5240 0x7fa6901f83f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6900b5650 0x7fa6901f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 --2- 172.21.15.37:0/4163761953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6900af420 0x7fa690161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 >> 172.21.15.37:0/4163761953 conn(0x7fa6900cca40 msgr2=0x7fa6901e7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:09.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 shutdown_connections 2024-09-21T06:29:09.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.267+0000 7fa697800640 1 -- 172.21.15.37:0/4163761953 wait complete. 2024-09-21T06:29:09.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:29:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 -- 172.21.15.37:0/3354037795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f9006ddf0 msgr2=0x7f5f9006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 --2- 172.21.15.37:0/3354037795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f9006ddf0 0x7f5f9006e200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009830 tx=0x7f5f8002a510 comp rx=0 tx=0).stop 2024-09-21T06:29:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 -- 172.21.15.37:0/3354037795 shutdown_connections 2024-09-21T06:29:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 --2- 172.21.15.37:0/3354037795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f900cef00 0x7f5f900cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 --2- 172.21.15.37:0/3354037795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f9006ddf0 0x7f5f9006e200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 -- 172.21.15.37:0/3354037795 >> 172.21.15.37:0/3354037795 conn(0x7f5f900b1b90 msgr2=0x7f5f900b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:09.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 -- 172.21.15.37:0/3354037795 shutdown_connections 2024-09-21T06:29:09.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.525+0000 7f5f98800640 1 -- 172.21.15.37:0/3354037795 wait complete. 2024-09-21T06:29:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.527+0000 7f5f98800640 1 Processor -- start 2024-09-21T06:29:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.527+0000 7f5f98800640 1 -- start start 2024-09-21T06:29:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 0x7f5f901f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f90156a50 0x7f5f90157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900cf850 con 0x7f5f900cef00 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90161060 con 0x7f5f9006ddf0 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f98800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900cf9d0 con 0x7f5f90156a50 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f95400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f95400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49886/0 (socket says 172.21.15.37:49886) 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f95400640 1 -- 172.21.15.37:0/768184854 learned_addr learned my addr 172.21.15.37:0/768184854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f96800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f90156a50 0x7f5f90157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.529+0000 7f5f95e00640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 0x7f5f901f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 -- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f90156a50 msgr2=0x7f5f90157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f90156a50 0x7f5f90157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 -- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 msgr2=0x7f5f901f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 0x7f5f901f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 -- 172.21.15.37:0/768184854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f901599f0 con 0x7f5f900cef00 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95e00640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 0x7f5f901f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f95400640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002ab00 tx=0x7f5f80033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005c90 con 0x7f5f900cef00 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f80031e50 con 0x7f5f900cef00 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f901f7890 con 0x7f5f900cef00 2024-09-21T06:29:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005260 con 0x7f5f900cef00 2024-09-21T06:29:09.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.530+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f90159f70 con 0x7f5f900cef00 2024-09-21T06:29:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.532+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f900c6400 con 0x7f5f900cef00 2024-09-21T06:29:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.533+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f80005400 con 0x7f5f900cef00 2024-09-21T06:29:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.535+0000 7f5f8a000640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 0x7f5f7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.535+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f5f800bbff0 con 0x7f5f900cef00 2024-09-21T06:29:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.535+0000 7f5f95e00640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 0x7f5f7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.536+0000 7f5f95e00640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 0x7f5f7407b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f840013f0 tx=0x7f5f84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.539+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f900c6400 con 0x7f5f900cef00 2024-09-21T06:29:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:09.774+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5f9006e200 con 0x7f5f900cef00 2024-09-21T06:29:10.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.329+0000 7f5f8a000640 1 -- 172.21.15.37:0/768184854 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f5f8003f030 con 0x7f5f900cef00 2024-09-21T06:29:10.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 msgr2=0x7f5f7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:10.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 0x7f5f7407b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f840013f0 tx=0x7f5f84009040 comp rx=0 tx=0).stop 2024-09-21T06:29:10.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 msgr2=0x7f5f901f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:10.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002ab00 tx=0x7f5f80033c20 comp rx=0 tx=0).stop 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 shutdown_connections 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f74078b90 0x7f5f7407b0a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f900cef00 0x7f5f901f6fc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f90156a50 0x7f5f90157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 --2- 172.21.15.37:0/768184854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f9006ddf0 0x7f5f901f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 >> 172.21.15.37:0/768184854 conn(0x7f5f900b1b90 msgr2=0x7f5f900737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:10.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 shutdown_connections 2024-09-21T06:29:10.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.332+0000 7f5f98800640 1 -- 172.21.15.37:0/768184854 wait complete. 2024-09-21T06:29:10.358 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:29:10.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:29:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 -- 172.21.15.37:0/3395808885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135406e370 msgr2=0x7f13540b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 --2- 172.21.15.37:0/3395808885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135406e370 0x7f13540b4f90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1348005fd0 tx=0x7f134802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 -- 172.21.15.37:0/3395808885 shutdown_connections 2024-09-21T06:29:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 --2- 172.21.15.37:0/3395808885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1354156c60 0x7f13540cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 --2- 172.21.15.37:0/3395808885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135406e370 0x7f13540b4f90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.586+0000 7f135a800640 1 -- 172.21.15.37:0/3395808885 >> 172.21.15.37:0/3395808885 conn(0x7f135406f920 msgr2=0x7f13540b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.587+0000 7f135a800640 1 -- 172.21.15.37:0/3395808885 shutdown_connections 2024-09-21T06:29:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.587+0000 7f135a800640 1 -- 172.21.15.37:0/3395808885 wait complete. 2024-09-21T06:29:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.589+0000 7f135a800640 1 Processor -- start 2024-09-21T06:29:10.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.589+0000 7f135a800640 1 -- start start 2024-09-21T06:29:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 0x7f13540c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13540cc8b0 0x7f1354144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 0x7f13541f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f135406d510 con 0x7f1354156c60 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f135406d390 con 0x7f135406e370 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.590+0000 7f135a800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f135406d690 con 0x7f13540cc8b0 2024-09-21T06:29:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 0x7f13541f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 0x7f13540c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 0x7f13540c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:45842/0 (socket says 172.21.15.37:45842) 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359400640 1 -- 172.21.15.37:0/1327872807 learned_addr learned my addr 172.21.15.37:0/1327872807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1358a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13540cc8b0 0x7f1354144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 -- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13540cc8b0 msgr2=0x7f1354144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13540cc8b0 0x7f1354144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 -- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 msgr2=0x7f13540c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 0x7f13540c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.591+0000 7f1359e00640 1 -- 172.21.15.37:0/1327872807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13540c4fc0 con 0x7f1354156c60 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f1359e00640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 0x7f13541f6b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c760 tx=0x7f134c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c003b70 con 0x7f1354156c60 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f134c003d10 con 0x7f1354156c60 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c005330 con 0x7f1354156c60 2024-09-21T06:29:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541f78b0 con 0x7f1354156c60 2024-09-21T06:29:10.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.592+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13540cb860 con 0x7f1354156c60 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.593+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13541f7620 con 0x7f1354156c60 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.594+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f134c0054d0 con 0x7f1354156c60 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.595+0000 7f1351600640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 0x7f133407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.596+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f134c09aad0 con 0x7f1354156c60 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.596+0000 7f1359400640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 0x7f133407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.597+0000 7f1359400640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 0x7f133407b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c00 tx=0x7f1348004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.600+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f134c062690 con 0x7f1354156c60 2024-09-21T06:29:10.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:10.837+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f13540b3aa0 con 0x7f1354156c60 2024-09-21T06:29:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.337+0000 7f1351600640 1 -- 172.21.15.37:0/1327872807 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f13540b3aa0 con 0x7f1354156c60 2024-09-21T06:29:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 msgr2=0x7f133407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 0x7f133407b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c00 tx=0x7f1348004730 comp rx=0 tx=0).stop 2024-09-21T06:29:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 msgr2=0x7f13541f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 0x7f13541f6b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c760 tx=0x7f134c00cc20 comp rx=0 tx=0).stop 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 shutdown_connections 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1334078b90 0x7f133407b0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1354156c60 0x7f13541f6b80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13540cc8b0 0x7f1354144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 --2- 172.21.15.37:0/1327872807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135406e370 0x7f13540c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 >> 172.21.15.37:0/1327872807 conn(0x7f135406f920 msgr2=0x7f1354159170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 shutdown_connections 2024-09-21T06:29:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.340+0000 7f135a800640 1 -- 172.21.15.37:0/1327872807 wait complete. 2024-09-21T06:29:11.369 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:29:11.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 -- 172.21.15.37:0/39061217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc0cbc10 msgr2=0x7f40dc0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 --2- 172.21.15.37:0/39061217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc0cbc10 0x7f40dc0cc020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009830 tx=0x7f40c402a510 comp rx=0 tx=0).stop 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 -- 172.21.15.37:0/39061217 shutdown_connections 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 --2- 172.21.15.37:0/39061217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc0cbc10 0x7f40dc0cc020 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 --2- 172.21.15.37:0/39061217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc074cb0 0x7f40dc073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.595+0000 7f40e2a00640 1 -- 172.21.15.37:0/39061217 >> 172.21.15.37:0/39061217 conn(0x7f40dc0cca40 msgr2=0x7f40dc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.596+0000 7f40e2a00640 1 -- 172.21.15.37:0/39061217 shutdown_connections 2024-09-21T06:29:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.596+0000 7f40e2a00640 1 -- 172.21.15.37:0/39061217 wait complete. 2024-09-21T06:29:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.598+0000 7f40e2a00640 1 Processor -- start 2024-09-21T06:29:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.598+0000 7f40e2a00640 1 -- start start 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc074cb0 0x7f40dc1f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 0x7f40dc1f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc06f360 con 0x7f40dc06ddf0 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc06f1e0 con 0x7f40dc074cb0 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.599+0000 7f40e2a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc06f4e0 con 0x7f40dc0cbc10 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49904/0 (socket says 172.21.15.37:49904) 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 -- 172.21.15.37:0/1409108363 learned_addr learned my addr 172.21.15.37:0/1409108363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40d3400640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc074cb0 0x7f40dc1f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40e0a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 0x7f40dc1f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 -- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 msgr2=0x7f40dc1f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 0x7f40dc1f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 -- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc074cb0 msgr2=0x7f40dc1f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc074cb0 0x7f40dc1f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 -- 172.21.15.37:0/1409108363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc0c8890 con 0x7f40dc06ddf0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40e0a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 0x7f40dc1f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.600+0000 7f40dbe00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e940 tx=0x7f40cc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.601+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00cd20 con 0x7f40dc06ddf0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.601+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40cc010430 con 0x7f40dc06ddf0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.601+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc010880 con 0x7f40dc06ddf0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.601+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1f9ab0 con 0x7f40dc06ddf0 2024-09-21T06:29:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.601+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1540c0 con 0x7f40dc06ddf0 2024-09-21T06:29:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.602+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc06fa00 con 0x7f40dc06ddf0 2024-09-21T06:29:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.603+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40cc01f020 con 0x7f40dc06ddf0 2024-09-21T06:29:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.604+0000 7f40d8c00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 0x7f40b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.604+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f40cc09b5f0 con 0x7f40dc06ddf0 2024-09-21T06:29:11.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.605+0000 7f40d3400640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 0x7f40b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:11.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.605+0000 7f40d3400640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 0x7f40b807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40c4034000 tx=0x7f40c4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:11.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.609+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40cc063890 con 0x7f40dc06ddf0 2024-09-21T06:29:11.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.849+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40dc15e2a0 con 0x7f40dc06ddf0 2024-09-21T06:29:11.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.849+0000 7f40d8c00640 1 -- 172.21.15.37:0/1409108363 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f40cc063190 con 0x7f40dc06ddf0 2024-09-21T06:29:11.852 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:11.852 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"btime":"2024-09-21T06:29:11:331581+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:10.839150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":2}]} 2024-09-21T06:29:11.852 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-09-21T06:29:11.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 msgr2=0x7f40b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 0x7f40b807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40c4034000 tx=0x7f40c4004730 comp rx=0 tx=0).stop 2024-09-21T06:29:11.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 msgr2=0x7f40dc15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:11.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e940 tx=0x7f40cc00ee00 comp rx=0 tx=0).stop 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 shutdown_connections 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f40b8078b90 0x7f40b807b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40dc0cbc10 0x7f40dc1f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40dc074cb0 0x7f40dc1f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 --2- 172.21.15.37:0/1409108363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc06ddf0 0x7f40dc15a0b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 >> 172.21.15.37:0/1409108363 conn(0x7f40dc0cca40 msgr2=0x7f40dc0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 shutdown_connections 2024-09-21T06:29:11.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:11.852+0000 7f40e2a00640 1 -- 172.21.15.37:0/1409108363 wait complete. 2024-09-21T06:29:11.876 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 -- 172.21.15.37:0/1056267917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c073f20 msgr2=0x7fbf8c15ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 --2- 172.21.15.37:0/1056267917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c073f20 0x7fbf8c15ca00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009860 tx=0x7fbf7c02a530 comp rx=0 tx=0).stop 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 -- 172.21.15.37:0/1056267917 shutdown_connections 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 --2- 172.21.15.37:0/1056267917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c073f20 0x7fbf8c15ca00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 -- 172.21.15.37:0/1056267917 >> 172.21.15.37:0/1056267917 conn(0x7fbf8c073400 msgr2=0x7fbf8c073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 -- 172.21.15.37:0/1056267917 shutdown_connections 2024-09-21T06:29:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.101+0000 7fbf8be00640 1 -- 172.21.15.37:0/1056267917 wait complete. 2024-09-21T06:29:12.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.104+0000 7fbf8be00640 1 Processor -- start 2024-09-21T06:29:12.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.104+0000 7fbf8be00640 1 -- start start 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c0afb80 0x7fbf8c1f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8c156fd0 0x7fbf8c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0cf5f0 con 0x7fbf8c073f20 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0cf470 con 0x7fbf8c156fd0 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0cf770 con 0x7fbf8c0afb80 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49916/0 (socket says 172.21.15.37:49916) 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.105+0000 7fbf8aa00640 1 -- 172.21.15.37:0/2513816674 learned_addr learned my addr 172.21.15.37:0/2513816674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8b400640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8c156fd0 0x7fbf8c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf82000640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c0afb80 0x7fbf8c1f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 -- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c0afb80 msgr2=0x7fbf8c1f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c0afb80 0x7fbf8c1f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 -- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8c156fd0 msgr2=0x7fbf8c1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8c156fd0 0x7fbf8c1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 -- 172.21.15.37:0/2513816674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c154c40 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf8aa00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009980 tx=0x7fbf7c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c005c90 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf7c005020 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.106+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c005410 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.107+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c1f8260 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.107+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c153a30 con 0x7fbf8c073f20 2024-09-21T06:29:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.108+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8c0cfc20 con 0x7fbf8c073f20 2024-09-21T06:29:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.109+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf7c0317c0 con 0x7fbf8c073f20 2024-09-21T06:29:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.110+0000 7fbf83400640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 0x7fbf6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.110+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fbf7c0bbe70 con 0x7fbf8c073f20 2024-09-21T06:29:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.110+0000 7fbf82000640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 0x7fbf6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.111+0000 7fbf82000640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 0x7fbf6807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf74001220 tx=0x7fbf74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.115+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf7c03f030 con 0x7fbf8c073f20 2024-09-21T06:29:12.281 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:29:12.279+0000 7fdd99200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:29:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.313+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf8c0b1f80 con 0x7fbf8c073f20 2024-09-21T06:29:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.314+0000 7fbf83400640 1 -- 172.21.15.37:0/2513816674 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17014 (secure 0 0 0) 0x7fbf7c0841a0 con 0x7fbf8c073f20 2024-09-21T06:29:12.317 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:12.317 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:29:09.296596+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:29:06.023459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T06:29:06.776946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 msgr2=0x7fbf6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 0x7fbf6807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf74001220 tx=0x7fbf74009040 comp rx=0 tx=0).stop 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 msgr2=0x7fbf8c154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009980 tx=0x7fbf7c003c10 comp rx=0 tx=0).stop 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 shutdown_connections 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf68078b90 0x7fbf6807b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8c156fd0 0x7fbf8c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf8c0afb80 0x7fbf8c1f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 --2- 172.21.15.37:0/2513816674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf8c073f20 0x7fbf8c154700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.316+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 >> 172.21.15.37:0/2513816674 conn(0x7fbf8c073400 msgr2=0x7fbf8c159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.317+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 shutdown_connections 2024-09-21T06:29:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.317+0000 7fbf8be00640 1 -- 172.21.15.37:0/2513816674 wait complete. 2024-09-21T06:29:12.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:29:12.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:29:12.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.566+0000 7fe1d0c00640 1 -- 172.21.15.37:0/2764502932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b0005280 msgr2=0x7fe1b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.566+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/2764502932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b0005280 0x7fe1b000b9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c610 tx=0x7fe1b0005830 comp rx=0 tx=0).stop 2024-09-21T06:29:12.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.566+0000 7fe1d0c00640 1 -- 172.21.15.37:0/2764502932 shutdown_connections 2024-09-21T06:29:12.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.566+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/2764502932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b0005280 0x7fe1b000b9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.566+0000 7fe1d0c00640 1 -- 172.21.15.37:0/2764502932 >> 172.21.15.37:0/2764502932 conn(0x7fe1cc0cca40 msgr2=0x7fe1cc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:12.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.567+0000 7fe1d0c00640 1 -- 172.21.15.37:0/2764502932 shutdown_connections 2024-09-21T06:29:12.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.567+0000 7fe1d0c00640 1 -- 172.21.15.37:0/2764502932 wait complete. 2024-09-21T06:29:12.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.569+0000 7fe1d0c00640 1 Processor -- start 2024-09-21T06:29:12.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.569+0000 7fe1d0c00640 1 -- start start 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b0005280 0x7fe1cc1ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 0x7fe1cc0c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0cfd70 con 0x7fe1cc15af60 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0cfbf0 con 0x7fe1cc0cce50 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.570+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc15f9b0 con 0x7fe1b0005280 2024-09-21T06:29:12.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49926/0 (socket says 172.21.15.37:49926) 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 -- 172.21.15.37:0/759643958 learned_addr learned my addr 172.21.15.37:0/759643958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1ca800640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 0x7fe1cc0c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c9e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b0005280 0x7fe1cc1ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 -- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b0005280 msgr2=0x7fe1cc1ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b0005280 0x7fe1cc1ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 -- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 msgr2=0x7fe1cc0c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 0x7fe1cc0c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1c3e00640 1 -- 172.21.15.37:0/759643958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc1575b0 con 0x7fe1cc15af60 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.571+0000 7fe1ca800640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 0x7fe1cc0c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1c3e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4003d30 tx=0x7fe1b4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4003e20 con 0x7fe1cc15af60 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b4016020 con 0x7fe1cc15af60 2024-09-21T06:29:12.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc1f5770 con 0x7fe1cc15af60 2024-09-21T06:29:12.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4011880 con 0x7fe1cc15af60 2024-09-21T06:29:12.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.572+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc1577c0 con 0x7fe1cc15af60 2024-09-21T06:29:12.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.573+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc051080 con 0x7fe1cc15af60 2024-09-21T06:29:12.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.575+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1b4011a20 con 0x7fe1cc15af60 2024-09-21T06:29:12.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.576+0000 7fe1c2000640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 0x7fe1a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.576+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe1b409b9a0 con 0x7fe1cc15af60 2024-09-21T06:29:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.576+0000 7fe1c9e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 0x7fe1a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.577+0000 7fe1c9e00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 0x7fe1a807b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009bd0 tx=0x7fe1bc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.580+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1cc051080 con 0x7fe1cc15af60 2024-09-21T06:29:12.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.774+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1cc06e010 con 0x7fe1a8078bc0 2024-09-21T06:29:12.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.779+0000 7fe1c2000640 1 -- 172.21.15.37:0/759643958 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246912 (secure 0 0 0) 0x7fe1cc06e010 con 0x7fe1a8078bc0 2024-09-21T06:29:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:29:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 msgr2=0x7fe1a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 0x7fe1a807b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009bd0 tx=0x7fe1bc005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 msgr2=0x7fe1cc157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4003d30 tx=0x7fe1b4001800 comp rx=0 tx=0).stop 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 shutdown_connections 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe1a8078bc0 0x7fe1a807b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1cc15af60 0x7fe1cc157070 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0cce50 0x7fe1cc0c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 --2- 172.21.15.37:0/759643958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b0005280 0x7fe1cc1ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 >> 172.21.15.37:0/759643958 conn(0x7fe1cc0cca40 msgr2=0x7fe1cc15cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.783+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 shutdown_connections 2024-09-21T06:29:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:12.784+0000 7fe1d0c00640 1 -- 172.21.15.37:0/759643958 wait complete. 2024-09-21T06:29:12.808 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-21T06:29:11.105966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220824,"kb_used_data":4596,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753896,"statfs":{"total":773094113280,"available":772867989504,"internally_reserved":0,"allocated":4706304,"data_stored":2654596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819616"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305401+0000","last_change":"2024-09-21T06:29:07.305401+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305401+0000","last_undegraded":"2024-09-21T06:29:07.305401+0000","last_fullsized":"2024-09-21T06:29:07.305401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.302539+0000","last_change":"2024-09-21T06:29:07.302539+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.302539+0000","last_undegraded":"2024-09-21T06:29:07.302539+0000","last_fullsized":"2024-09-21T06:29:07.302539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.308117+0000","last_change":"2024-09-21T06:29:07.308117+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.308117+0000","last_undegraded":"2024-09-21T06:29:07.308117+0000","last_fullsized":"2024-09-21T06:29:07.308117+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310547+0000","last_change":"2024-09-21T06:29:07.310547+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310547+0000","last_undegraded":"2024-09-21T06:29:07.310547+0000","last_fullsized":"2024-09-21T06:29:07.310547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309095+0000","last_change":"2024-09-21T06:29:07.309095+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309095+0000","last_undegraded":"2024-09-21T06:29:07.309095+0000","last_fullsized":"2024-09-21T06:29:07.309095+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.299434+0000","last_change":"2024-09-21T06:29:07.299434+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.299434+0000","last_undegraded":"2024-09-21T06:29:07.299434+0000","last_fullsized":"2024-09-21T06:29:07.299434+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.303777+0000","last_change":"2024-09-21T06:29:07.303777+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.303777+0000","last_undegraded":"2024-09-21T06:29:07.303777+0000","last_fullsized":"2024-09-21T06:29:07.303777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309524+0000","last_change":"2024-09-21T06:29:07.309524+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309524+0000","last_undegraded":"2024-09-21T06:29:07.309524+0000","last_fullsized":"2024-09-21T06:29:07.309524+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.312501+0000","last_change":"2024-09-21T06:29:07.312501+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.312501+0000","last_undegraded":"2024-09-21T06:29:07.312501+0000","last_fullsized":"2024-09-21T06:29:07.312501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310447+0000","last_change":"2024-09-21T06:29:07.310447+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310447+0000","last_undegraded":"2024-09-21T06:29:07.310447+0000","last_fullsized":"2024-09-21T06:29:07.310447+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.313766+0000","last_change":"2024-09-21T06:29:07.313766+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.313766+0000","last_undegraded":"2024-09-21T06:29:07.313766+0000","last_fullsized":"2024-09-21T06:29:07.313766+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310875+0000","last_change":"2024-09-21T06:29:07.310875+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310875+0000","last_undegraded":"2024-09-21T06:29:07.310875+0000","last_fullsized":"2024-09-21T06:29:07.310875+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.308413+0000","last_change":"2024-09-21T06:29:07.308413+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.308413+0000","last_undegraded":"2024-09-21T06:29:07.308413+0000","last_fullsized":"2024-09-21T06:29:07.308413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306197+0000","last_change":"2024-09-21T06:29:07.306197+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306197+0000","last_undegraded":"2024-09-21T06:29:07.306197+0000","last_fullsized":"2024-09-21T06:29:07.306197+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309468+0000","last_change":"2024-09-21T06:29:07.309468+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309468+0000","last_undegraded":"2024-09-21T06:29:07.309468+0000","last_fullsized":"2024-09-21T06:29:07.309468+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.303195+0000","last_change":"2024-09-21T06:29:07.303195+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.303195+0000","last_undegraded":"2024-09-21T06:29:07.303195+0000","last_fullsized":"2024-09-21T06:29:07.303195+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310635+0000","last_change":"2024-09-21T06:29:07.310635+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310635+0000","last_undegraded":"2024-09-21T06:29:07.310635+0000","last_fullsized":"2024-09-21T06:29:07.310635+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305272+0000","last_change":"2024-09-21T06:29:07.305272+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305272+0000","last_undegraded":"2024-09-21T06:29:07.305272+0000","last_fullsized":"2024-09-21T06:29:07.305272+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336057+0000","last_change":"2024-09-21T06:29:08.326681+0000","last_active":"2024-09-21T06:29:08.336057+0000","last_peered":"2024-09-21T06:29:08.336057+0000","last_clean":"2024-09-21T06:29:08.336057+0000","last_became_active":"2024-09-21T06:29:08.326293+0000","last_became_peered":"2024-09-21T06:29:08.326293+0000","last_unstale":"2024-09-21T06:29:08.336057+0000","last_undegraded":"2024-09-21T06:29:08.336057+0000","last_fullsized":"2024-09-21T06:29:08.336057+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:02.903480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.313178+0000","last_change":"2024-09-21T06:29:07.313178+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.313178+0000","last_undegraded":"2024-09-21T06:29:07.313178+0000","last_fullsized":"2024-09-21T06:29:07.313178+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.314633+0000","last_change":"2024-09-21T06:29:07.314633+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.314633+0000","last_undegraded":"2024-09-21T06:29:07.314633+0000","last_fullsized":"2024-09-21T06:29:07.314633+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.304559+0000","last_change":"2024-09-21T06:29:07.304559+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.304559+0000","last_undegraded":"2024-09-21T06:29:07.304559+0000","last_fullsized":"2024-09-21T06:29:07.304559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.302346+0000","last_change":"2024-09-21T06:29:07.302346+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.302346+0000","last_undegraded":"2024-09-21T06:29:07.302346+0000","last_fullsized":"2024-09-21T06:29:07.302346+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309003+0000","last_change":"2024-09-21T06:29:07.309003+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309003+0000","last_undegraded":"2024-09-21T06:29:07.309003+0000","last_fullsized":"2024-09-21T06:29:07.309003+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305157+0000","last_change":"2024-09-21T06:29:07.305157+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305157+0000","last_undegraded":"2024-09-21T06:29:07.305157+0000","last_fullsized":"2024-09-21T06:29:07.305157+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.307564+0000","last_change":"2024-09-21T06:29:07.307564+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.307564+0000","last_undegraded":"2024-09-21T06:29:07.307564+0000","last_fullsized":"2024-09-21T06:29:07.307564+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.313258+0000","last_change":"2024-09-21T06:29:07.313258+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.313258+0000","last_undegraded":"2024-09-21T06:29:07.313258+0000","last_fullsized":"2024-09-21T06:29:07.313258+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309113+0000","last_change":"2024-09-21T06:29:07.309113+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309113+0000","last_undegraded":"2024-09-21T06:29:07.309113+0000","last_fullsized":"2024-09-21T06:29:07.309113+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.307704+0000","last_change":"2024-09-21T06:29:07.307704+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.307704+0000","last_undegraded":"2024-09-21T06:29:07.307704+0000","last_fullsized":"2024-09-21T06:29:07.307704+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306125+0000","last_change":"2024-09-21T06:29:07.306125+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306125+0000","last_undegraded":"2024-09-21T06:29:07.306125+0000","last_fullsized":"2024-09-21T06:29:07.306125+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.303308+0000","last_change":"2024-09-21T06:29:07.303308+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.303308+0000","last_undegraded":"2024-09-21T06:29:07.303308+0000","last_fullsized":"2024-09-21T06:29:07.303308+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.308057+0000","last_change":"2024-09-21T06:29:07.308057+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.308057+0000","last_undegraded":"2024-09-21T06:29:07.308057+0000","last_fullsized":"2024-09-21T06:29:07.308057+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305145+0000","last_change":"2024-09-21T06:29:07.305145+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305145+0000","last_undegraded":"2024-09-21T06:29:07.305145+0000","last_fullsized":"2024-09-21T06:29:07.305145+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.320571+0000","last_change":"2024-09-21T06:29:07.314974+0000","last_active":"2024-09-21T06:29:07.320571+0000","last_peered":"2024-09-21T06:29:07.320571+0000","last_clean":"2024-09-21T06:29:07.320571+0000","last_became_active":"2024-09-21T06:29:07.314561+0000","last_became_peered":"2024-09-21T06:29:07.314561+0000","last_unstale":"2024-09-21T06:29:07.320571+0000","last_undegraded":"2024-09-21T06:29:07.320571+0000","last_fullsized":"2024-09-21T06:29:07.320571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:13.238925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.299682+0000","last_change":"2024-09-21T06:29:07.299682+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.299682+0000","last_undegraded":"2024-09-21T06:29:07.299682+0000","last_fullsized":"2024-09-21T06:29:07.299682+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.320509+0000","last_change":"2024-09-21T06:29:07.315223+0000","last_active":"2024-09-21T06:29:07.320509+0000","last_peered":"2024-09-21T06:29:07.320509+0000","last_clean":"2024-09-21T06:29:07.320509+0000","last_became_active":"2024-09-21T06:29:07.314888+0000","last_became_peered":"2024-09-21T06:29:07.314888+0000","last_unstale":"2024-09-21T06:29:07.320509+0000","last_undegraded":"2024-09-21T06:29:07.320509+0000","last_fullsized":"2024-09-21T06:29:07.320509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:09.685248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.302074+0000","last_change":"2024-09-21T06:29:07.302074+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.302074+0000","last_undegraded":"2024-09-21T06:29:07.302074+0000","last_fullsized":"2024-09-21T06:29:07.302074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323806+0000","last_change":"2024-09-21T06:29:07.320934+0000","last_active":"2024-09-21T06:29:07.323806+0000","last_peered":"2024-09-21T06:29:07.323806+0000","last_clean":"2024-09-21T06:29:07.323806+0000","last_became_active":"2024-09-21T06:29:07.320506+0000","last_became_peered":"2024-09-21T06:29:07.320506+0000","last_unstale":"2024-09-21T06:29:07.323806+0000","last_undegraded":"2024-09-21T06:29:07.323806+0000","last_fullsized":"2024-09-21T06:29:07.323806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:04.773319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.302132+0000","last_change":"2024-09-21T06:29:07.302132+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.302132+0000","last_undegraded":"2024-09-21T06:29:07.302132+0000","last_fullsized":"2024-09-21T06:29:07.302132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.327833+0000","last_change":"2024-09-21T06:29:07.323512+0000","last_active":"2024-09-21T06:29:07.327833+0000","last_peered":"2024-09-21T06:29:07.327833+0000","last_clean":"2024-09-21T06:29:07.327833+0000","last_became_active":"2024-09-21T06:29:07.323205+0000","last_became_peered":"2024-09-21T06:29:07.323205+0000","last_unstale":"2024-09-21T06:29:07.327833+0000","last_undegraded":"2024-09-21T06:29:07.327833+0000","last_fullsized":"2024-09-21T06:29:07.327833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:48.268049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306460+0000","last_change":"2024-09-21T06:29:07.306460+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306460+0000","last_undegraded":"2024-09-21T06:29:07.306460+0000","last_fullsized":"2024-09-21T06:29:07.306460+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.325885+0000","last_change":"2024-09-21T06:29:07.322003+0000","last_active":"2024-09-21T06:29:07.325885+0000","last_peered":"2024-09-21T06:29:07.325885+0000","last_clean":"2024-09-21T06:29:07.325885+0000","last_became_active":"2024-09-21T06:29:07.320890+0000","last_became_peered":"2024-09-21T06:29:07.320890+0000","last_unstale":"2024-09-21T06:29:07.325885+0000","last_undegraded":"2024-09-21T06:29:07.325885+0000","last_fullsized":"2024-09-21T06:29:07.325885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:50.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.302893+0000","last_change":"2024-09-21T06:29:07.302893+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.302893+0000","last_undegraded":"2024-09-21T06:29:07.302893+0000","last_fullsized":"2024-09-21T06:29:07.302893+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.318861+0000","last_change":"2024-09-21T06:29:07.312417+0000","last_active":"2024-09-21T06:29:07.318861+0000","last_peered":"2024-09-21T06:29:07.318861+0000","last_clean":"2024-09-21T06:29:07.318861+0000","last_became_active":"2024-09-21T06:29:07.311268+0000","last_became_peered":"2024-09-21T06:29:07.311268+0000","last_unstale":"2024-09-21T06:29:07.318861+0000","last_undegraded":"2024-09-21T06:29:07.318861+0000","last_fullsized":"2024-09-21T06:29:07.318861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:42.782192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305642+0000","last_change":"2024-09-21T06:29:07.305642+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305642+0000","last_undegraded":"2024-09-21T06:29:07.305642+0000","last_fullsized":"2024-09-21T06:29:07.305642+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328742+0000","last_change":"2024-09-21T06:29:07.324154+0000","last_active":"2024-09-21T06:29:07.328742+0000","last_peered":"2024-09-21T06:29:07.328742+0000","last_clean":"2024-09-21T06:29:07.328742+0000","last_became_active":"2024-09-21T06:29:07.323831+0000","last_became_peered":"2024-09-21T06:29:07.323831+0000","last_unstale":"2024-09-21T06:29:07.328742+0000","last_undegraded":"2024-09-21T06:29:07.328742+0000","last_fullsized":"2024-09-21T06:29:07.328742+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:13.907420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.301887+0000","last_change":"2024-09-21T06:29:07.301887+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.301887+0000","last_undegraded":"2024-09-21T06:29:07.301887+0000","last_fullsized":"2024-09-21T06:29:07.301887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.330239+0000","last_change":"2024-09-21T06:29:07.326452+0000","last_active":"2024-09-21T06:29:07.330239+0000","last_peered":"2024-09-21T06:29:07.330239+0000","last_clean":"2024-09-21T06:29:07.330239+0000","last_became_active":"2024-09-21T06:29:07.326157+0000","last_became_peered":"2024-09-21T06:29:07.326157+0000","last_unstale":"2024-09-21T06:29:07.330239+0000","last_undegraded":"2024-09-21T06:29:07.330239+0000","last_fullsized":"2024-09-21T06:29:07.330239+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:04.539999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309776+0000","last_change":"2024-09-21T06:29:07.309776+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309776+0000","last_undegraded":"2024-09-21T06:29:07.309776+0000","last_fullsized":"2024-09-21T06:29:07.309776+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.319491+0000","last_change":"2024-09-21T06:29:07.313126+0000","last_active":"2024-09-21T06:29:07.319491+0000","last_peered":"2024-09-21T06:29:07.319491+0000","last_clean":"2024-09-21T06:29:07.319491+0000","last_became_active":"2024-09-21T06:29:07.312787+0000","last_became_peered":"2024-09-21T06:29:07.312787+0000","last_unstale":"2024-09-21T06:29:07.319491+0000","last_undegraded":"2024-09-21T06:29:07.319491+0000","last_fullsized":"2024-09-21T06:29:07.319491+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:17.575942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.304536+0000","last_change":"2024-09-21T06:29:07.304536+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.304536+0000","last_undegraded":"2024-09-21T06:29:07.304536+0000","last_fullsized":"2024-09-21T06:29:07.304536+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.319899+0000","last_change":"2024-09-21T06:29:07.311359+0000","last_active":"2024-09-21T06:29:07.319899+0000","last_peered":"2024-09-21T06:29:07.319899+0000","last_clean":"2024-09-21T06:29:07.319899+0000","last_became_active":"2024-09-21T06:29:07.308258+0000","last_became_peered":"2024-09-21T06:29:07.308258+0000","last_unstale":"2024-09-21T06:29:07.319899+0000","last_undegraded":"2024-09-21T06:29:07.319899+0000","last_fullsized":"2024-09-21T06:29:07.319899+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:33.961438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.307073+0000","last_change":"2024-09-21T06:29:07.307073+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.307073+0000","last_undegraded":"2024-09-21T06:29:07.307073+0000","last_fullsized":"2024-09-21T06:29:07.307073+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.324840+0000","last_change":"2024-09-21T06:29:07.321535+0000","last_active":"2024-09-21T06:29:07.324840+0000","last_peered":"2024-09-21T06:29:07.324840+0000","last_clean":"2024-09-21T06:29:07.324840+0000","last_became_active":"2024-09-21T06:29:07.321169+0000","last_became_peered":"2024-09-21T06:29:07.321169+0000","last_unstale":"2024-09-21T06:29:07.324840+0000","last_undegraded":"2024-09-21T06:29:07.324840+0000","last_fullsized":"2024-09-21T06:29:07.324840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:39.033320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310277+0000","last_change":"2024-09-21T06:29:07.310277+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310277+0000","last_undegraded":"2024-09-21T06:29:07.310277+0000","last_fullsized":"2024-09-21T06:29:07.310277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.319436+0000","last_change":"2024-09-21T06:29:07.313046+0000","last_active":"2024-09-21T06:29:07.319436+0000","last_peered":"2024-09-21T06:29:07.319436+0000","last_clean":"2024-09-21T06:29:07.319436+0000","last_became_active":"2024-09-21T06:29:07.312716+0000","last_became_peered":"2024-09-21T06:29:07.312716+0000","last_unstale":"2024-09-21T06:29:07.319436+0000","last_undegraded":"2024-09-21T06:29:07.319436+0000","last_fullsized":"2024-09-21T06:29:07.319436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:48.268049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305879+0000","last_change":"2024-09-21T06:29:07.305879+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305879+0000","last_undegraded":"2024-09-21T06:29:07.305879+0000","last_fullsized":"2024-09-21T06:29:07.305879+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.320451+0000","last_change":"2024-09-21T06:29:07.316562+0000","last_active":"2024-09-21T06:29:07.320451+0000","last_peered":"2024-09-21T06:29:07.320451+0000","last_clean":"2024-09-21T06:29:07.320451+0000","last_became_active":"2024-09-21T06:29:07.316237+0000","last_became_peered":"2024-09-21T06:29:07.316237+0000","last_unstale":"2024-09-21T06:29:07.320451+0000","last_undegraded":"2024-09-21T06:29:07.320451+0000","last_fullsized":"2024-09-21T06:29:07.320451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:45.349318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.299853+0000","last_change":"2024-09-21T06:29:07.299853+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.299853+0000","last_undegraded":"2024-09-21T06:29:07.299853+0000","last_fullsized":"2024-09-21T06:29:07.299853+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.319840+0000","last_change":"2024-09-21T06:29:07.311950+0000","last_active":"2024-09-21T06:29:07.319840+0000","last_peered":"2024-09-21T06:29:07.319840+0000","last_clean":"2024-09-21T06:29:07.319840+0000","last_became_active":"2024-09-21T06:29:07.311659+0000","last_became_peered":"2024-09-21T06:29:07.311659+0000","last_unstale":"2024-09-21T06:29:07.319840+0000","last_undegraded":"2024-09-21T06:29:07.319840+0000","last_fullsized":"2024-09-21T06:29:07.319840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:13.238925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.303729+0000","last_change":"2024-09-21T06:29:07.303729+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.303729+0000","last_undegraded":"2024-09-21T06:29:07.303729+0000","last_fullsized":"2024-09-21T06:29:07.303729+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323252+0000","last_change":"2024-09-21T06:29:07.317825+0000","last_active":"2024-09-21T06:29:07.323252+0000","last_peered":"2024-09-21T06:29:07.323252+0000","last_clean":"2024-09-21T06:29:07.323252+0000","last_became_active":"2024-09-21T06:29:07.316929+0000","last_became_peered":"2024-09-21T06:29:07.316929+0000","last_unstale":"2024-09-21T06:29:07.323252+0000","last_undegraded":"2024-09-21T06:29:07.323252+0000","last_fullsized":"2024-09-21T06:29:07.323252+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:22.556100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306556+0000","last_change":"2024-09-21T06:29:07.306556+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306556+0000","last_undegraded":"2024-09-21T06:29:07.306556+0000","last_fullsized":"2024-09-21T06:29:07.306556+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.324808+0000","last_change":"2024-09-21T06:29:07.316760+0000","last_active":"2024-09-21T06:29:07.324808+0000","last_peered":"2024-09-21T06:29:07.324808+0000","last_clean":"2024-09-21T06:29:07.324808+0000","last_became_active":"2024-09-21T06:29:07.316317+0000","last_became_peered":"2024-09-21T06:29:07.316317+0000","last_unstale":"2024-09-21T06:29:07.324808+0000","last_undegraded":"2024-09-21T06:29:07.324808+0000","last_fullsized":"2024-09-21T06:29:07.324808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:09.685248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.299697+0000","last_change":"2024-09-21T06:29:07.299697+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.299697+0000","last_undegraded":"2024-09-21T06:29:07.299697+0000","last_fullsized":"2024-09-21T06:29:07.299697+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.330246+0000","last_change":"2024-09-21T06:29:07.327438+0000","last_active":"2024-09-21T06:29:07.330246+0000","last_peered":"2024-09-21T06:29:07.330246+0000","last_clean":"2024-09-21T06:29:07.330246+0000","last_became_active":"2024-09-21T06:29:07.326953+0000","last_became_peered":"2024-09-21T06:29:07.326953+0000","last_unstale":"2024-09-21T06:29:07.330246+0000","last_undegraded":"2024-09-21T06:29:07.330246+0000","last_fullsized":"2024-09-21T06:29:07.330246+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:07.356405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310054+0000","last_change":"2024-09-21T06:29:07.310054+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310054+0000","last_undegraded":"2024-09-21T06:29:07.310054+0000","last_fullsized":"2024-09-21T06:29:07.310054+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323560+0000","last_change":"2024-09-21T06:29:07.319663+0000","last_active":"2024-09-21T06:29:07.323560+0000","last_peered":"2024-09-21T06:29:07.323560+0000","last_clean":"2024-09-21T06:29:07.323560+0000","last_became_active":"2024-09-21T06:29:07.319270+0000","last_became_peered":"2024-09-21T06:29:07.319270+0000","last_unstale":"2024-09-21T06:29:07.323560+0000","last_undegraded":"2024-09-21T06:29:07.323560+0000","last_fullsized":"2024-09-21T06:29:07.323560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:46.519576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306727+0000","last_change":"2024-09-21T06:29:07.306727+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306727+0000","last_undegraded":"2024-09-21T06:29:07.306727+0000","last_fullsized":"2024-09-21T06:29:07.306727+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328984+0000","last_change":"2024-09-21T06:29:07.325336+0000","last_active":"2024-09-21T06:29:07.328984+0000","last_peered":"2024-09-21T06:29:07.328984+0000","last_clean":"2024-09-21T06:29:07.328984+0000","last_became_active":"2024-09-21T06:29:07.324982+0000","last_became_peered":"2024-09-21T06:29:07.324982+0000","last_unstale":"2024-09-21T06:29:07.328984+0000","last_undegraded":"2024-09-21T06:29:07.328984+0000","last_fullsized":"2024-09-21T06:29:07.328984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:57.123372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336181+0000","last_change":"2024-09-21T06:29:08.326873+0000","last_active":"2024-09-21T06:29:08.336181+0000","last_peered":"2024-09-21T06:29:08.336181+0000","last_clean":"2024-09-21T06:29:08.336181+0000","last_became_active":"2024-09-21T06:29:08.326538+0000","last_became_peered":"2024-09-21T06:29:08.326538+0000","last_unstale":"2024-09-21T06:29:08.336181+0000","last_undegraded":"2024-09-21T06:29:08.336181+0000","last_fullsized":"2024-09-21T06:29:08.336181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:00.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.308732+0000","last_change":"2024-09-21T06:29:07.316341+0000","last_active":"2024-09-21T06:29:08.308732+0000","last_peered":"2024-09-21T06:29:08.308732+0000","last_clean":"2024-09-21T06:29:08.308732+0000","last_became_active":"2024-09-21T06:29:07.316013+0000","last_became_peered":"2024-09-21T06:29:07.316013+0000","last_unstale":"2024-09-21T06:29:08.308732+0000","last_undegraded":"2024-09-21T06:29:08.308732+0000","last_fullsized":"2024-09-21T06:29:08.308732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:10.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.301602+0000","last_change":"2024-09-21T06:29:07.301602+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.301602+0000","last_undegraded":"2024-09-21T06:29:07.301602+0000","last_fullsized":"2024-09-21T06:29:07.301602+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336776+0000","last_change":"2024-09-21T06:29:07.317403+0000","last_active":"2024-09-21T06:29:08.336776+0000","last_peered":"2024-09-21T06:29:08.336776+0000","last_clean":"2024-09-21T06:29:08.336776+0000","last_became_active":"2024-09-21T06:29:07.317101+0000","last_became_peered":"2024-09-21T06:29:07.317101+0000","last_unstale":"2024-09-21T06:29:08.336776+0000","last_undegraded":"2024-09-21T06:29:08.336776+0000","last_fullsized":"2024-09-21T06:29:08.336776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:02.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306236+0000","last_change":"2024-09-21T06:29:07.306236+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306236+0000","last_undegraded":"2024-09-21T06:29:07.306236+0000","last_fullsized":"2024-09-21T06:29:07.306236+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.326465+0000","last_change":"2024-09-21T06:29:07.319665+0000","last_active":"2024-09-21T06:29:07.326465+0000","last_peered":"2024-09-21T06:29:07.326465+0000","last_clean":"2024-09-21T06:29:07.326465+0000","last_became_active":"2024-09-21T06:29:07.318846+0000","last_became_peered":"2024-09-21T06:29:07.318846+0000","last_unstale":"2024-09-21T06:29:07.326465+0000","last_undegraded":"2024-09-21T06:29:07.326465+0000","last_fullsized":"2024-09-21T06:29:07.326465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:48.268049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.304099+0000","last_change":"2024-09-21T06:29:07.304099+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.304099+0000","last_undegraded":"2024-09-21T06:29:07.304099+0000","last_fullsized":"2024-09-21T06:29:07.304099+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328258+0000","last_change":"2024-09-21T06:29:07.325806+0000","last_active":"2024-09-21T06:29:07.328258+0000","last_peered":"2024-09-21T06:29:07.328258+0000","last_clean":"2024-09-21T06:29:07.328258+0000","last_became_active":"2024-09-21T06:29:07.325483+0000","last_became_peered":"2024-09-21T06:29:07.325483+0000","last_unstale":"2024-09-21T06:29:07.328258+0000","last_undegraded":"2024-09-21T06:29:07.328258+0000","last_fullsized":"2024-09-21T06:29:07.328258+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:33.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.306528+0000","last_change":"2024-09-21T06:29:07.306528+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.306528+0000","last_undegraded":"2024-09-21T06:29:07.306528+0000","last_fullsized":"2024-09-21T06:29:07.306528+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323925+0000","last_change":"2024-09-21T06:29:07.318878+0000","last_active":"2024-09-21T06:29:07.323925+0000","last_peered":"2024-09-21T06:29:07.323925+0000","last_clean":"2024-09-21T06:29:07.323925+0000","last_became_active":"2024-09-21T06:29:07.317361+0000","last_became_peered":"2024-09-21T06:29:07.317361+0000","last_unstale":"2024-09-21T06:29:07.323925+0000","last_undegraded":"2024-09-21T06:29:07.323925+0000","last_fullsized":"2024-09-21T06:29:07.323925+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:50.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.310359+0000","last_change":"2024-09-21T06:29:07.310359+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.310359+0000","last_undegraded":"2024-09-21T06:29:07.310359+0000","last_fullsized":"2024-09-21T06:29:07.310359+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328069+0000","last_change":"2024-09-21T06:29:07.323099+0000","last_active":"2024-09-21T06:29:07.328069+0000","last_peered":"2024-09-21T06:29:07.328069+0000","last_clean":"2024-09-21T06:29:07.328069+0000","last_became_active":"2024-09-21T06:29:07.322763+0000","last_became_peered":"2024-09-21T06:29:07.322763+0000","last_unstale":"2024-09-21T06:29:07.328069+0000","last_undegraded":"2024-09-21T06:29:07.328069+0000","last_fullsized":"2024-09-21T06:29:07.328069+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:44.834930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.311070+0000","last_change":"2024-09-21T06:29:07.311070+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.311070+0000","last_undegraded":"2024-09-21T06:29:07.311070+0000","last_fullsized":"2024-09-21T06:29:07.311070+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328982+0000","last_change":"2024-09-21T06:29:07.325007+0000","last_active":"2024-09-21T06:29:07.328982+0000","last_peered":"2024-09-21T06:29:07.328982+0000","last_clean":"2024-09-21T06:29:07.328982+0000","last_became_active":"2024-09-21T06:29:07.324590+0000","last_became_peered":"2024-09-21T06:29:07.324590+0000","last_unstale":"2024-09-21T06:29:07.328982+0000","last_undegraded":"2024-09-21T06:29:07.328982+0000","last_fullsized":"2024-09-21T06:29:07.328982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:09.025945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.335957+0000","last_change":"2024-09-21T06:29:08.326146+0000","last_active":"2024-09-21T06:29:08.335957+0000","last_peered":"2024-09-21T06:29:08.335957+0000","last_clean":"2024-09-21T06:29:08.335957+0000","last_became_active":"2024-09-21T06:29:08.325802+0000","last_became_peered":"2024-09-21T06:29:08.325802+0000","last_unstale":"2024-09-21T06:29:08.335957+0000","last_undegraded":"2024-09-21T06:29:08.335957+0000","last_fullsized":"2024-09-21T06:29:08.335957+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:57.060675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.328963+0000","last_change":"2024-09-21T06:29:07.324710+0000","last_active":"2024-09-21T06:29:07.328963+0000","last_peered":"2024-09-21T06:29:07.328963+0000","last_clean":"2024-09-21T06:29:07.328963+0000","last_became_active":"2024-09-21T06:29:07.324348+0000","last_became_peered":"2024-09-21T06:29:07.324348+0000","last_unstale":"2024-09-21T06:29:07.328963+0000","last_undegraded":"2024-09-21T06:29:07.328963+0000","last_fullsized":"2024-09-21T06:29:07.328963+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:24.621159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.308113+0000","last_change":"2024-09-21T06:29:07.308113+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.308113+0000","last_undegraded":"2024-09-21T06:29:07.308113+0000","last_fullsized":"2024-09-21T06:29:07.308113+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323743+0000","last_change":"2024-09-21T06:29:07.318527+0000","last_active":"2024-09-21T06:29:07.323743+0000","last_peered":"2024-09-21T06:29:07.323743+0000","last_clean":"2024-09-21T06:29:07.323743+0000","last_became_active":"2024-09-21T06:29:07.318203+0000","last_became_peered":"2024-09-21T06:29:07.318203+0000","last_unstale":"2024-09-21T06:29:07.323743+0000","last_undegraded":"2024-09-21T06:29:07.323743+0000","last_fullsized":"2024-09-21T06:29:07.323743+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:13.191624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.299185+0000","last_change":"2024-09-21T06:29:07.299185+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.299185+0000","last_undegraded":"2024-09-21T06:29:07.299185+0000","last_fullsized":"2024-09-21T06:29:07.299185+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.325620+0000","last_change":"2024-09-21T06:29:07.318905+0000","last_active":"2024-09-21T06:29:07.325620+0000","last_peered":"2024-09-21T06:29:07.325620+0000","last_clean":"2024-09-21T06:29:07.325620+0000","last_became_active":"2024-09-21T06:29:07.317691+0000","last_became_peered":"2024-09-21T06:29:07.317691+0000","last_unstale":"2024-09-21T06:29:07.325620+0000","last_undegraded":"2024-09-21T06:29:07.325620+0000","last_fullsized":"2024-09-21T06:29:07.325620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:48.268049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.305109+0000","last_change":"2024-09-21T06:29:07.305109+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.305109+0000","last_undegraded":"2024-09-21T06:29:07.305109+0000","last_fullsized":"2024-09-21T06:29:07.305109+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323288+0000","last_change":"2024-09-21T06:29:07.318752+0000","last_active":"2024-09-21T06:29:07.323288+0000","last_peered":"2024-09-21T06:29:07.323288+0000","last_clean":"2024-09-21T06:29:07.323288+0000","last_became_active":"2024-09-21T06:29:07.318441+0000","last_became_peered":"2024-09-21T06:29:07.318441+0000","last_unstale":"2024-09-21T06:29:07.323288+0000","last_undegraded":"2024-09-21T06:29:07.323288+0000","last_fullsized":"2024-09-21T06:29:07.323288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:06.186274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.309845+0000","last_change":"2024-09-21T06:29:07.309845+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.309845+0000","last_undegraded":"2024-09-21T06:29:07.309845+0000","last_fullsized":"2024-09-21T06:29:07.309845+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.323924+0000","last_change":"2024-09-21T06:29:07.318921+0000","last_active":"2024-09-21T06:29:07.323924+0000","last_peered":"2024-09-21T06:29:07.323924+0000","last_clean":"2024-09-21T06:29:07.323924+0000","last_became_active":"2024-09-21T06:29:07.318259+0000","last_became_peered":"2024-09-21T06:29:07.318259+0000","last_unstale":"2024-09-21T06:29:07.323924+0000","last_undegraded":"2024-09-21T06:29:07.323924+0000","last_fullsized":"2024-09-21T06:29:07.323924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:09.445848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-21T06:29:07.307662+0000","last_change":"2024-09-21T06:29:07.307662+0000","last_active":"2024-09-21T06:29:07.285114+0000","last_peered":"2024-09-21T06:29:07.285114+0000","last_clean":"2024-09-21T06:29:07.285114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:29:07.307662+0000","last_undegraded":"2024-09-21T06:29:07.307662+0000","last_fullsized":"2024-09-21T06:29:07.307662+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.326772+0000","last_change":"2024-09-21T06:29:07.321833+0000","last_active":"2024-09-21T06:29:07.326772+0000","last_peered":"2024-09-21T06:29:07.326772+0000","last_clean":"2024-09-21T06:29:07.326772+0000","last_became_active":"2024-09-21T06:29:07.321550+0000","last_became_peered":"2024-09-21T06:29:07.321550+0000","last_unstale":"2024-09-21T06:29:07.326772+0000","last_undegraded":"2024-09-21T06:29:07.326772+0000","last_fullsized":"2024-09-21T06:29:07.326772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:17.575942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":142,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T06:29:07.311457+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:29:07.311457+0000","last_peered":"2024-09-21T06:29:07.311457+0000","last_clean":"2024-09-21T06:29:07.311457+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:29:07.311457+0000","last_undegraded":"2024-09-21T06:29:07.311457+0000","last_fullsized":"2024-09-21T06:29:07.311457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":179097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":442368,"data_stored":185764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":769465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":179097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":783881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:29:12.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:29:13.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 -- 172.21.15.37:0/476430685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50806e220 msgr2=0x7fe5080cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 --2- 172.21.15.37:0/476430685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50806e220 0x7fe5080cf5c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0005fd0 tx=0x7fe4f002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 -- 172.21.15.37:0/476430685 shutdown_connections 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 --2- 172.21.15.37:0/476430685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50806e220 0x7fe5080cf5c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 --2- 172.21.15.37:0/476430685 >> v2:172.21.15.150:3300/0 conn(0x7fe5080cfb00 0x7fe5080b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 -- 172.21.15.37:0/476430685 >> 172.21.15.37:0/476430685 conn(0x7fe5080d4360 msgr2=0x7fe5080d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 -- 172.21.15.37:0/476430685 shutdown_connections 2024-09-21T06:29:13.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.076+0000 7fe507200640 1 -- 172.21.15.37:0/476430685 wait complete. 2024-09-21T06:29:13.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.078+0000 7fe507200640 1 Processor -- start 2024-09-21T06:29:13.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.078+0000 7fe507200640 1 -- start start 2024-09-21T06:29:13.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe50806e220 0x7fe50814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50815e780 0x7fe50815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe50806cf60 con 0x7fe5080cfb00 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe50806cde0 con 0x7fe50806e220 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe507200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe5080b7ef0 con 0x7fe50815e780 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe4fd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe4fd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49930/0 (socket says 172.21.15.37:49930) 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe4fd400640 1 -- 172.21.15.37:0/152325900 learned_addr learned my addr 172.21.15.37:0/152325900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe506800640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50815e780 0x7fe50815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.080+0000 7fe505e00640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe50806e220 0x7fe50814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 -- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50815e780 msgr2=0x7fe50815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50815e780 0x7fe50815f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 -- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe50806e220 msgr2=0x7fe50814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe50806e220 0x7fe50814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 -- 172.21.15.37:0/152325900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5080b6740 con 0x7fe5080cfb00 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fd400640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0034040 tx=0x7fe4f00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f003e030 con 0x7fe5080cfb00 2024-09-21T06:29:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4f0003ae0 con 0x7fe5080cfb00 2024-09-21T06:29:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50815fea0 con 0x7fe5080cfb00 2024-09-21T06:29:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.081+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0035040 con 0x7fe5080cfb00 2024-09-21T06:29:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.082+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5081609a0 con 0x7fe5080cfb00 2024-09-21T06:29:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.083+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5080b8450 con 0x7fe5080cfb00 2024-09-21T06:29:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.084+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4f0041030 con 0x7fe5080cfb00 2024-09-21T06:29:13.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.085+0000 7fe4fea00640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 0x7fe4e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.086+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe4f00bb710 con 0x7fe5080cfb00 2024-09-21T06:29:13.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.086+0000 7fe505e00640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 0x7fe4e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.087+0000 7fe505e00640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 0x7fe4e407b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80017a0 tx=0x7fe4f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.090+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4f00832d0 con 0x7fe5080cfb00 2024-09-21T06:29:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.282+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5080ca470 con 0x7fe4e4078bc0 2024-09-21T06:29:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.288+0000 7fe4fea00640 1 -- 172.21.15.37:0/152325900 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fe5080ca470 con 0x7fe4e4078bc0 2024-09-21T06:29:13.293 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:13.294 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:29:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 msgr2=0x7fe4e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 0x7fe4e407b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80017a0 tx=0x7fe4f8009040 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 msgr2=0x7fe5080b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0034040 tx=0x7fe4f00035f0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 shutdown_connections 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe4e4078bc0 0x7fe4e407b0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5080cfb00 0x7fe5080b6200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe50815e780 0x7fe50815f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 --2- 172.21.15.37:0/152325900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe50806e220 0x7fe50814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 >> 172.21.15.37:0/152325900 conn(0x7fe5080d4360 msgr2=0x7fe50815b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 shutdown_connections 2024-09-21T06:29:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.292+0000 7fe507200640 1 -- 172.21.15.37:0/152325900 wait complete. 2024-09-21T06:29:13.319 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-21T06:29:13.106427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221108,"kb_used_data":4860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753612,"statfs":{"total":773094113280,"available":772867698688,"internally_reserved":0,"allocated":4976640,"data_stored":2775848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001436"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930978+0000","last_change":"2024-09-21T06:29:08.333502+0000","last_active":"2024-09-21T06:29:10.930978+0000","last_peered":"2024-09-21T06:29:10.930978+0000","last_clean":"2024-09-21T06:29:10.930978+0000","last_became_active":"2024-09-21T06:29:08.332985+0000","last_became_peered":"2024-09-21T06:29:08.332985+0000","last_unstale":"2024-09-21T06:29:10.930978+0000","last_undegraded":"2024-09-21T06:29:10.930978+0000","last_fullsized":"2024-09-21T06:29:10.930978+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:23.830851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.304145+0000","last_change":"2024-09-21T06:29:08.333644+0000","last_active":"2024-09-21T06:29:09.304145+0000","last_peered":"2024-09-21T06:29:09.304145+0000","last_clean":"2024-09-21T06:29:09.304145+0000","last_became_active":"2024-09-21T06:29:08.332565+0000","last_became_peered":"2024-09-21T06:29:08.332565+0000","last_unstale":"2024-09-21T06:29:09.304145+0000","last_undegraded":"2024-09-21T06:29:09.304145+0000","last_fullsized":"2024-09-21T06:29:09.304145+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:47.527908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930489+0000","last_change":"2024-09-21T06:29:08.329524+0000","last_active":"2024-09-21T06:29:10.930489+0000","last_peered":"2024-09-21T06:29:10.930489+0000","last_clean":"2024-09-21T06:29:10.930489+0000","last_became_active":"2024-09-21T06:29:08.329162+0000","last_became_peered":"2024-09-21T06:29:08.329162+0000","last_unstale":"2024-09-21T06:29:10.930489+0000","last_undegraded":"2024-09-21T06:29:10.930489+0000","last_fullsized":"2024-09-21T06:29:10.930489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:02.611383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930237+0000","last_change":"2024-09-21T06:29:08.312375+0000","last_active":"2024-09-21T06:29:10.930237+0000","last_peered":"2024-09-21T06:29:10.930237+0000","last_clean":"2024-09-21T06:29:10.930237+0000","last_became_active":"2024-09-21T06:29:08.310905+0000","last_became_peered":"2024-09-21T06:29:08.310905+0000","last_unstale":"2024-09-21T06:29:10.930237+0000","last_undegraded":"2024-09-21T06:29:10.930237+0000","last_fullsized":"2024-09-21T06:29:10.930237+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:00.469289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.303237+0000","last_change":"2024-09-21T06:29:08.343063+0000","last_active":"2024-09-21T06:29:09.303237+0000","last_peered":"2024-09-21T06:29:09.303237+0000","last_clean":"2024-09-21T06:29:09.303237+0000","last_became_active":"2024-09-21T06:29:08.342607+0000","last_became_peered":"2024-09-21T06:29:08.342607+0000","last_unstale":"2024-09-21T06:29:09.303237+0000","last_undegraded":"2024-09-21T06:29:09.303237+0000","last_fullsized":"2024-09-21T06:29:09.303237+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:14.199956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302727+0000","last_change":"2024-09-21T06:29:08.330757+0000","last_active":"2024-09-21T06:29:09.302727+0000","last_peered":"2024-09-21T06:29:09.302727+0000","last_clean":"2024-09-21T06:29:09.302727+0000","last_became_active":"2024-09-21T06:29:08.330415+0000","last_became_peered":"2024-09-21T06:29:08.330415+0000","last_unstale":"2024-09-21T06:29:09.302727+0000","last_undegraded":"2024-09-21T06:29:09.302727+0000","last_fullsized":"2024-09-21T06:29:09.302727+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:07.027022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.218133+0000","last_change":"2024-09-21T06:29:08.336832+0000","last_active":"2024-09-21T06:29:11.218133+0000","last_peered":"2024-09-21T06:29:11.218133+0000","last_clean":"2024-09-21T06:29:11.218133+0000","last_became_active":"2024-09-21T06:29:08.336471+0000","last_became_peered":"2024-09-21T06:29:08.336471+0000","last_unstale":"2024-09-21T06:29:11.218133+0000","last_undegraded":"2024-09-21T06:29:11.218133+0000","last_fullsized":"2024-09-21T06:29:11.218133+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:20.985562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930007+0000","last_change":"2024-09-21T06:29:08.324622+0000","last_active":"2024-09-21T06:29:10.930007+0000","last_peered":"2024-09-21T06:29:10.930007+0000","last_clean":"2024-09-21T06:29:10.930007+0000","last_became_active":"2024-09-21T06:29:08.324092+0000","last_became_peered":"2024-09-21T06:29:08.324092+0000","last_unstale":"2024-09-21T06:29:10.930007+0000","last_undegraded":"2024-09-21T06:29:10.930007+0000","last_fullsized":"2024-09-21T06:29:10.930007+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:14.775900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.303385+0000","last_change":"2024-09-21T06:29:08.342878+0000","last_active":"2024-09-21T06:29:09.303385+0000","last_peered":"2024-09-21T06:29:09.303385+0000","last_clean":"2024-09-21T06:29:09.303385+0000","last_became_active":"2024-09-21T06:29:08.342279+0000","last_became_peered":"2024-09-21T06:29:08.342279+0000","last_unstale":"2024-09-21T06:29:09.303385+0000","last_undegraded":"2024-09-21T06:29:09.303385+0000","last_fullsized":"2024-09-21T06:29:09.303385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:14.915752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217964+0000","last_change":"2024-09-21T06:29:08.323766+0000","last_active":"2024-09-21T06:29:11.217964+0000","last_peered":"2024-09-21T06:29:11.217964+0000","last_clean":"2024-09-21T06:29:11.217964+0000","last_became_active":"2024-09-21T06:29:08.320560+0000","last_became_peered":"2024-09-21T06:29:08.320560+0000","last_unstale":"2024-09-21T06:29:11.217964+0000","last_undegraded":"2024-09-21T06:29:11.217964+0000","last_fullsized":"2024-09-21T06:29:11.217964+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:22.858134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179944+0000","last_change":"2024-09-21T06:29:08.333065+0000","last_active":"2024-09-21T06:29:10.179944+0000","last_peered":"2024-09-21T06:29:10.179944+0000","last_clean":"2024-09-21T06:29:10.179944+0000","last_became_active":"2024-09-21T06:29:08.331441+0000","last_became_peered":"2024-09-21T06:29:08.331441+0000","last_unstale":"2024-09-21T06:29:10.179944+0000","last_undegraded":"2024-09-21T06:29:10.179944+0000","last_fullsized":"2024-09-21T06:29:10.179944+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:28.161238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302632+0000","last_change":"2024-09-21T06:29:08.334617+0000","last_active":"2024-09-21T06:29:09.302632+0000","last_peered":"2024-09-21T06:29:09.302632+0000","last_clean":"2024-09-21T06:29:09.302632+0000","last_became_active":"2024-09-21T06:29:08.334341+0000","last_became_peered":"2024-09-21T06:29:08.334341+0000","last_unstale":"2024-09-21T06:29:09.302632+0000","last_undegraded":"2024-09-21T06:29:09.302632+0000","last_fullsized":"2024-09-21T06:29:09.302632+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:46.357650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217762+0000","last_change":"2024-09-21T06:29:08.320200+0000","last_active":"2024-09-21T06:29:11.217762+0000","last_peered":"2024-09-21T06:29:11.217762+0000","last_clean":"2024-09-21T06:29:11.217762+0000","last_became_active":"2024-09-21T06:29:08.319913+0000","last_became_peered":"2024-09-21T06:29:08.319913+0000","last_unstale":"2024-09-21T06:29:11.217762+0000","last_undegraded":"2024-09-21T06:29:11.217762+0000","last_fullsized":"2024-09-21T06:29:11.217762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:05.781650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.370801+0000","last_change":"2024-09-21T06:29:08.318111+0000","last_active":"2024-09-21T06:29:12.370801+0000","last_peered":"2024-09-21T06:29:12.370801+0000","last_clean":"2024-09-21T06:29:12.370801+0000","last_became_active":"2024-09-21T06:29:08.317719+0000","last_became_peered":"2024-09-21T06:29:08.317719+0000","last_unstale":"2024-09-21T06:29:12.370801+0000","last_undegraded":"2024-09-21T06:29:12.370801+0000","last_fullsized":"2024-09-21T06:29:12.370801+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:56.044605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300589+0000","last_change":"2024-09-21T06:29:08.334108+0000","last_active":"2024-09-21T06:29:09.300589+0000","last_peered":"2024-09-21T06:29:09.300589+0000","last_clean":"2024-09-21T06:29:09.300589+0000","last_became_active":"2024-09-21T06:29:08.333770+0000","last_became_peered":"2024-09-21T06:29:08.333770+0000","last_unstale":"2024-09-21T06:29:09.300589+0000","last_undegraded":"2024-09-21T06:29:09.300589+0000","last_fullsized":"2024-09-21T06:29:09.300589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:05.131615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217652+0000","last_change":"2024-09-21T06:29:08.322285+0000","last_active":"2024-09-21T06:29:11.217652+0000","last_peered":"2024-09-21T06:29:11.217652+0000","last_clean":"2024-09-21T06:29:11.217652+0000","last_became_active":"2024-09-21T06:29:08.320431+0000","last_became_peered":"2024-09-21T06:29:08.320431+0000","last_unstale":"2024-09-21T06:29:11.217652+0000","last_undegraded":"2024-09-21T06:29:11.217652+0000","last_fullsized":"2024-09-21T06:29:11.217652+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:10.454179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301720+0000","last_change":"2024-09-21T06:29:08.329732+0000","last_active":"2024-09-21T06:29:09.301720+0000","last_peered":"2024-09-21T06:29:09.301720+0000","last_clean":"2024-09-21T06:29:09.301720+0000","last_became_active":"2024-09-21T06:29:08.326253+0000","last_became_peered":"2024-09-21T06:29:08.326253+0000","last_unstale":"2024-09-21T06:29:09.301720+0000","last_undegraded":"2024-09-21T06:29:09.301720+0000","last_fullsized":"2024-09-21T06:29:09.301720+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:14.872121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301695+0000","last_change":"2024-09-21T06:29:08.334954+0000","last_active":"2024-09-21T06:29:09.301695+0000","last_peered":"2024-09-21T06:29:09.301695+0000","last_clean":"2024-09-21T06:29:09.301695+0000","last_became_active":"2024-09-21T06:29:08.334351+0000","last_became_peered":"2024-09-21T06:29:08.334351+0000","last_unstale":"2024-09-21T06:29:09.301695+0000","last_undegraded":"2024-09-21T06:29:09.301695+0000","last_fullsized":"2024-09-21T06:29:09.301695+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:47.088442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336057+0000","last_change":"2024-09-21T06:29:08.326681+0000","last_active":"2024-09-21T06:29:08.336057+0000","last_peered":"2024-09-21T06:29:08.336057+0000","last_clean":"2024-09-21T06:29:08.336057+0000","last_became_active":"2024-09-21T06:29:08.326293+0000","last_became_peered":"2024-09-21T06:29:08.326293+0000","last_unstale":"2024-09-21T06:29:08.336057+0000","last_undegraded":"2024-09-21T06:29:08.336057+0000","last_fullsized":"2024-09-21T06:29:08.336057+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:02.903480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179516+0000","last_change":"2024-09-21T06:29:08.320195+0000","last_active":"2024-09-21T06:29:10.179516+0000","last_peered":"2024-09-21T06:29:10.179516+0000","last_clean":"2024-09-21T06:29:10.179516+0000","last_became_active":"2024-09-21T06:29:08.319883+0000","last_became_peered":"2024-09-21T06:29:08.319883+0000","last_unstale":"2024-09-21T06:29:10.179516+0000","last_undegraded":"2024-09-21T06:29:10.179516+0000","last_fullsized":"2024-09-21T06:29:10.179516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:47.527908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179449+0000","last_change":"2024-09-21T06:29:08.315839+0000","last_active":"2024-09-21T06:29:10.179449+0000","last_peered":"2024-09-21T06:29:10.179449+0000","last_clean":"2024-09-21T06:29:10.179449+0000","last_became_active":"2024-09-21T06:29:08.315515+0000","last_became_peered":"2024-09-21T06:29:08.315515+0000","last_unstale":"2024-09-21T06:29:10.179449+0000","last_undegraded":"2024-09-21T06:29:10.179449+0000","last_fullsized":"2024-09-21T06:29:10.179449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:07.194605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179313+0000","last_change":"2024-09-21T06:29:08.320662+0000","last_active":"2024-09-21T06:29:10.179313+0000","last_peered":"2024-09-21T06:29:10.179313+0000","last_clean":"2024-09-21T06:29:10.179313+0000","last_became_active":"2024-09-21T06:29:08.320353+0000","last_became_peered":"2024-09-21T06:29:08.320353+0000","last_unstale":"2024-09-21T06:29:10.179313+0000","last_undegraded":"2024-09-21T06:29:10.179313+0000","last_fullsized":"2024-09-21T06:29:10.179313+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:07.027022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300641+0000","last_change":"2024-09-21T06:29:08.314228+0000","last_active":"2024-09-21T06:29:09.300641+0000","last_peered":"2024-09-21T06:29:09.300641+0000","last_clean":"2024-09-21T06:29:09.300641+0000","last_became_active":"2024-09-21T06:29:08.313891+0000","last_became_peered":"2024-09-21T06:29:08.313891+0000","last_unstale":"2024-09-21T06:29:09.300641+0000","last_undegraded":"2024-09-21T06:29:09.300641+0000","last_fullsized":"2024-09-21T06:29:09.300641+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:27.602110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930280+0000","last_change":"2024-09-21T06:29:08.316041+0000","last_active":"2024-09-21T06:29:10.930280+0000","last_peered":"2024-09-21T06:29:10.930280+0000","last_clean":"2024-09-21T06:29:10.930280+0000","last_became_active":"2024-09-21T06:29:08.315514+0000","last_became_peered":"2024-09-21T06:29:08.315514+0000","last_unstale":"2024-09-21T06:29:10.930280+0000","last_undegraded":"2024-09-21T06:29:10.930280+0000","last_fullsized":"2024-09-21T06:29:10.930280+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:07.048532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.371041+0000","last_change":"2024-09-21T06:29:08.328427+0000","last_active":"2024-09-21T06:29:12.371041+0000","last_peered":"2024-09-21T06:29:12.371041+0000","last_clean":"2024-09-21T06:29:12.371041+0000","last_became_active":"2024-09-21T06:29:08.328087+0000","last_became_peered":"2024-09-21T06:29:08.328087+0000","last_unstale":"2024-09-21T06:29:12.371041+0000","last_undegraded":"2024-09-21T06:29:12.371041+0000","last_fullsized":"2024-09-21T06:29:12.371041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:41.496760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930343+0000","last_change":"2024-09-21T06:29:08.320355+0000","last_active":"2024-09-21T06:29:10.930343+0000","last_peered":"2024-09-21T06:29:10.930343+0000","last_clean":"2024-09-21T06:29:10.930343+0000","last_became_active":"2024-09-21T06:29:08.318928+0000","last_became_peered":"2024-09-21T06:29:08.318928+0000","last_unstale":"2024-09-21T06:29:10.930343+0000","last_undegraded":"2024-09-21T06:29:10.930343+0000","last_fullsized":"2024-09-21T06:29:10.930343+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:22.640301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179959+0000","last_change":"2024-09-21T06:29:08.333383+0000","last_active":"2024-09-21T06:29:10.179959+0000","last_peered":"2024-09-21T06:29:10.179959+0000","last_clean":"2024-09-21T06:29:10.179959+0000","last_became_active":"2024-09-21T06:29:08.332298+0000","last_became_peered":"2024-09-21T06:29:08.332298+0000","last_unstale":"2024-09-21T06:29:10.179959+0000","last_undegraded":"2024-09-21T06:29:10.179959+0000","last_fullsized":"2024-09-21T06:29:10.179959+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:00.040001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217850+0000","last_change":"2024-09-21T06:29:08.322021+0000","last_active":"2024-09-21T06:29:11.217850+0000","last_peered":"2024-09-21T06:29:11.217850+0000","last_clean":"2024-09-21T06:29:11.217850+0000","last_became_active":"2024-09-21T06:29:08.321687+0000","last_became_peered":"2024-09-21T06:29:08.321687+0000","last_unstale":"2024-09-21T06:29:11.217850+0000","last_undegraded":"2024-09-21T06:29:11.217850+0000","last_fullsized":"2024-09-21T06:29:11.217850+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:14.915752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301198+0000","last_change":"2024-09-21T06:29:08.314310+0000","last_active":"2024-09-21T06:29:09.301198+0000","last_peered":"2024-09-21T06:29:09.301198+0000","last_clean":"2024-09-21T06:29:09.301198+0000","last_became_active":"2024-09-21T06:29:08.314001+0000","last_became_peered":"2024-09-21T06:29:08.314001+0000","last_unstale":"2024-09-21T06:29:09.301198+0000","last_undegraded":"2024-09-21T06:29:09.301198+0000","last_fullsized":"2024-09-21T06:29:09.301198+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:29.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300699+0000","last_change":"2024-09-21T06:29:08.313338+0000","last_active":"2024-09-21T06:29:09.300699+0000","last_peered":"2024-09-21T06:29:09.300699+0000","last_clean":"2024-09-21T06:29:09.300699+0000","last_became_active":"2024-09-21T06:29:08.312608+0000","last_became_peered":"2024-09-21T06:29:08.312608+0000","last_unstale":"2024-09-21T06:29:09.300699+0000","last_undegraded":"2024-09-21T06:29:09.300699+0000","last_fullsized":"2024-09-21T06:29:09.300699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:35.152769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217731+0000","last_change":"2024-09-21T06:29:08.324391+0000","last_active":"2024-09-21T06:29:11.217731+0000","last_peered":"2024-09-21T06:29:11.217731+0000","last_clean":"2024-09-21T06:29:11.217731+0000","last_became_active":"2024-09-21T06:29:08.322914+0000","last_became_peered":"2024-09-21T06:29:08.322914+0000","last_unstale":"2024-09-21T06:29:11.217731+0000","last_undegraded":"2024-09-21T06:29:11.217731+0000","last_fullsized":"2024-09-21T06:29:11.217731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:47.437253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930371+0000","last_change":"2024-09-21T06:29:08.322855+0000","last_active":"2024-09-21T06:29:10.930371+0000","last_peered":"2024-09-21T06:29:10.930371+0000","last_clean":"2024-09-21T06:29:10.930371+0000","last_became_active":"2024-09-21T06:29:08.322511+0000","last_became_peered":"2024-09-21T06:29:08.322511+0000","last_unstale":"2024-09-21T06:29:10.930371+0000","last_undegraded":"2024-09-21T06:29:10.930371+0000","last_fullsized":"2024-09-21T06:29:10.930371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:23.327850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301362+0000","last_change":"2024-09-21T06:29:08.319959+0000","last_active":"2024-09-21T06:29:09.301362+0000","last_peered":"2024-09-21T06:29:09.301362+0000","last_clean":"2024-09-21T06:29:09.301362+0000","last_became_active":"2024-09-21T06:29:08.319537+0000","last_became_peered":"2024-09-21T06:29:08.319537+0000","last_unstale":"2024-09-21T06:29:09.301362+0000","last_undegraded":"2024-09-21T06:29:09.301362+0000","last_fullsized":"2024-09-21T06:29:09.301362+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:04.281213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.371040+0000","last_change":"2024-09-21T06:29:07.314974+0000","last_active":"2024-09-21T06:29:12.371040+0000","last_peered":"2024-09-21T06:29:12.371040+0000","last_clean":"2024-09-21T06:29:12.371040+0000","last_became_active":"2024-09-21T06:29:07.314561+0000","last_became_peered":"2024-09-21T06:29:07.314561+0000","last_unstale":"2024-09-21T06:29:12.371040+0000","last_undegraded":"2024-09-21T06:29:12.371040+0000","last_fullsized":"2024-09-21T06:29:12.371040+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:13.191624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.304169+0000","last_change":"2024-09-21T06:29:08.329074+0000","last_active":"2024-09-21T06:29:09.304169+0000","last_peered":"2024-09-21T06:29:09.304169+0000","last_clean":"2024-09-21T06:29:09.304169+0000","last_became_active":"2024-09-21T06:29:08.326799+0000","last_became_peered":"2024-09-21T06:29:08.326799+0000","last_unstale":"2024-09-21T06:29:09.304169+0000","last_undegraded":"2024-09-21T06:29:09.304169+0000","last_fullsized":"2024-09-21T06:29:09.304169+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:40.041652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.304253+0000","last_change":"2024-09-21T06:29:07.315223+0000","last_active":"2024-09-21T06:29:09.304253+0000","last_peered":"2024-09-21T06:29:09.304253+0000","last_clean":"2024-09-21T06:29:09.304253+0000","last_became_active":"2024-09-21T06:29:07.314888+0000","last_became_peered":"2024-09-21T06:29:07.314888+0000","last_unstale":"2024-09-21T06:29:09.304253+0000","last_undegraded":"2024-09-21T06:29:09.304253+0000","last_fullsized":"2024-09-21T06:29:09.304253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:13.191624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301850+0000","last_change":"2024-09-21T06:29:08.321487+0000","last_active":"2024-09-21T06:29:09.301850+0000","last_peered":"2024-09-21T06:29:09.301850+0000","last_clean":"2024-09-21T06:29:09.301850+0000","last_became_active":"2024-09-21T06:29:08.321117+0000","last_became_peered":"2024-09-21T06:29:08.321117+0000","last_unstale":"2024-09-21T06:29:09.301850+0000","last_undegraded":"2024-09-21T06:29:09.301850+0000","last_fullsized":"2024-09-21T06:29:09.301850+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:48.583399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930715+0000","last_change":"2024-09-21T06:29:07.320934+0000","last_active":"2024-09-21T06:29:10.930715+0000","last_peered":"2024-09-21T06:29:10.930715+0000","last_clean":"2024-09-21T06:29:10.930715+0000","last_became_active":"2024-09-21T06:29:07.320506+0000","last_became_peered":"2024-09-21T06:29:07.320506+0000","last_unstale":"2024-09-21T06:29:10.930715+0000","last_undegraded":"2024-09-21T06:29:10.930715+0000","last_fullsized":"2024-09-21T06:29:10.930715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:30.519593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.304251+0000","last_change":"2024-09-21T06:29:08.331352+0000","last_active":"2024-09-21T06:29:09.304251+0000","last_peered":"2024-09-21T06:29:09.304251+0000","last_clean":"2024-09-21T06:29:09.304251+0000","last_became_active":"2024-09-21T06:29:08.327866+0000","last_became_peered":"2024-09-21T06:29:08.327866+0000","last_unstale":"2024-09-21T06:29:09.304251+0000","last_undegraded":"2024-09-21T06:29:09.304251+0000","last_fullsized":"2024-09-21T06:29:09.304251+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:45.843261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.370945+0000","last_change":"2024-09-21T06:29:07.323512+0000","last_active":"2024-09-21T06:29:12.370945+0000","last_peered":"2024-09-21T06:29:12.370945+0000","last_clean":"2024-09-21T06:29:12.370945+0000","last_became_active":"2024-09-21T06:29:07.323205+0000","last_became_peered":"2024-09-21T06:29:07.323205+0000","last_unstale":"2024-09-21T06:29:12.370945+0000","last_undegraded":"2024-09-21T06:29:12.370945+0000","last_fullsized":"2024-09-21T06:29:12.370945+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:09.445848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930624+0000","last_change":"2024-09-21T06:29:08.314400+0000","last_active":"2024-09-21T06:29:10.930624+0000","last_peered":"2024-09-21T06:29:10.930624+0000","last_clean":"2024-09-21T06:29:10.930624+0000","last_became_active":"2024-09-21T06:29:08.314083+0000","last_became_peered":"2024-09-21T06:29:08.314083+0000","last_unstale":"2024-09-21T06:29:10.930624+0000","last_undegraded":"2024-09-21T06:29:10.930624+0000","last_fullsized":"2024-09-21T06:29:10.930624+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:30.594717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.370785+0000","last_change":"2024-09-21T06:29:07.322003+0000","last_active":"2024-09-21T06:29:12.370785+0000","last_peered":"2024-09-21T06:29:12.370785+0000","last_clean":"2024-09-21T06:29:12.370785+0000","last_became_active":"2024-09-21T06:29:07.320890+0000","last_became_peered":"2024-09-21T06:29:07.320890+0000","last_unstale":"2024-09-21T06:29:12.370785+0000","last_undegraded":"2024-09-21T06:29:12.370785+0000","last_fullsized":"2024-09-21T06:29:12.370785+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:45.349318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179357+0000","last_change":"2024-09-21T06:29:08.317018+0000","last_active":"2024-09-21T06:29:10.179357+0000","last_peered":"2024-09-21T06:29:10.179357+0000","last_clean":"2024-09-21T06:29:10.179357+0000","last_became_active":"2024-09-21T06:29:08.316693+0000","last_became_peered":"2024-09-21T06:29:08.316693+0000","last_unstale":"2024-09-21T06:29:10.179357+0000","last_undegraded":"2024-09-21T06:29:10.179357+0000","last_fullsized":"2024-09-21T06:29:10.179357+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:34.969770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217590+0000","last_change":"2024-09-21T06:29:07.312417+0000","last_active":"2024-09-21T06:29:11.217590+0000","last_peered":"2024-09-21T06:29:11.217590+0000","last_clean":"2024-09-21T06:29:11.217590+0000","last_became_active":"2024-09-21T06:29:07.311268+0000","last_became_peered":"2024-09-21T06:29:07.311268+0000","last_unstale":"2024-09-21T06:29:11.217590+0000","last_undegraded":"2024-09-21T06:29:11.217590+0000","last_fullsized":"2024-09-21T06:29:11.217590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:57.123372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300731+0000","last_change":"2024-09-21T06:29:08.313712+0000","last_active":"2024-09-21T06:29:09.300731+0000","last_peered":"2024-09-21T06:29:09.300731+0000","last_clean":"2024-09-21T06:29:09.300731+0000","last_became_active":"2024-09-21T06:29:08.313001+0000","last_became_peered":"2024-09-21T06:29:08.313001+0000","last_unstale":"2024-09-21T06:29:09.300731+0000","last_undegraded":"2024-09-21T06:29:09.300731+0000","last_fullsized":"2024-09-21T06:29:09.300731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:54.407773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.929894+0000","last_change":"2024-09-21T06:29:07.324154+0000","last_active":"2024-09-21T06:29:10.929894+0000","last_peered":"2024-09-21T06:29:10.929894+0000","last_clean":"2024-09-21T06:29:10.929894+0000","last_became_active":"2024-09-21T06:29:07.323831+0000","last_became_peered":"2024-09-21T06:29:07.323831+0000","last_unstale":"2024-09-21T06:29:10.929894+0000","last_undegraded":"2024-09-21T06:29:10.929894+0000","last_fullsized":"2024-09-21T06:29:10.929894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:21.849802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.303967+0000","last_change":"2024-09-21T06:29:08.323014+0000","last_active":"2024-09-21T06:29:09.303967+0000","last_peered":"2024-09-21T06:29:09.303967+0000","last_clean":"2024-09-21T06:29:09.303967+0000","last_became_active":"2024-09-21T06:29:08.322602+0000","last_became_peered":"2024-09-21T06:29:08.322602+0000","last_unstale":"2024-09-21T06:29:09.303967+0000","last_undegraded":"2024-09-21T06:29:09.303967+0000","last_fullsized":"2024-09-21T06:29:09.303967+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:23.564432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217593+0000","last_change":"2024-09-21T06:29:07.326452+0000","last_active":"2024-09-21T06:29:11.217593+0000","last_peered":"2024-09-21T06:29:11.217593+0000","last_clean":"2024-09-21T06:29:11.217593+0000","last_became_active":"2024-09-21T06:29:07.326157+0000","last_became_peered":"2024-09-21T06:29:07.326157+0000","last_unstale":"2024-09-21T06:29:11.217593+0000","last_undegraded":"2024-09-21T06:29:11.217593+0000","last_fullsized":"2024-09-21T06:29:11.217593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:24.621159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930341+0000","last_change":"2024-09-21T06:29:08.312355+0000","last_active":"2024-09-21T06:29:10.930341+0000","last_peered":"2024-09-21T06:29:10.930341+0000","last_clean":"2024-09-21T06:29:10.930341+0000","last_became_active":"2024-09-21T06:29:08.311244+0000","last_became_peered":"2024-09-21T06:29:08.311244+0000","last_unstale":"2024-09-21T06:29:10.930341+0000","last_undegraded":"2024-09-21T06:29:10.930341+0000","last_fullsized":"2024-09-21T06:29:10.930341+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:49.169546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930849+0000","last_change":"2024-09-21T06:29:07.313126+0000","last_active":"2024-09-21T06:29:10.930849+0000","last_peered":"2024-09-21T06:29:10.930849+0000","last_clean":"2024-09-21T06:29:10.930849+0000","last_became_active":"2024-09-21T06:29:07.312787+0000","last_became_peered":"2024-09-21T06:29:07.312787+0000","last_unstale":"2024-09-21T06:29:10.930849+0000","last_undegraded":"2024-09-21T06:29:10.930849+0000","last_fullsized":"2024-09-21T06:29:10.930849+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:21.429054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.370869+0000","last_change":"2024-09-21T06:29:08.326191+0000","last_active":"2024-09-21T06:29:12.370869+0000","last_peered":"2024-09-21T06:29:12.370869+0000","last_clean":"2024-09-21T06:29:12.370869+0000","last_became_active":"2024-09-21T06:29:08.325277+0000","last_became_peered":"2024-09-21T06:29:08.325277+0000","last_unstale":"2024-09-21T06:29:12.370869+0000","last_undegraded":"2024-09-21T06:29:12.370869+0000","last_fullsized":"2024-09-21T06:29:12.370869+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:52.898421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301710+0000","last_change":"2024-09-21T06:29:07.311359+0000","last_active":"2024-09-21T06:29:09.301710+0000","last_peered":"2024-09-21T06:29:09.301710+0000","last_clean":"2024-09-21T06:29:09.301710+0000","last_became_active":"2024-09-21T06:29:07.308258+0000","last_became_peered":"2024-09-21T06:29:07.308258+0000","last_unstale":"2024-09-21T06:29:09.301710+0000","last_undegraded":"2024-09-21T06:29:09.301710+0000","last_fullsized":"2024-09-21T06:29:09.301710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:59.031670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930637+0000","last_change":"2024-09-21T06:29:08.337559+0000","last_active":"2024-09-21T06:29:10.930637+0000","last_peered":"2024-09-21T06:29:10.930637+0000","last_clean":"2024-09-21T06:29:10.930637+0000","last_became_active":"2024-09-21T06:29:08.337079+0000","last_became_peered":"2024-09-21T06:29:08.337079+0000","last_unstale":"2024-09-21T06:29:10.930637+0000","last_undegraded":"2024-09-21T06:29:10.930637+0000","last_fullsized":"2024-09-21T06:29:10.930637+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:36.454110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301247+0000","last_change":"2024-09-21T06:29:07.321535+0000","last_active":"2024-09-21T06:29:09.301247+0000","last_peered":"2024-09-21T06:29:09.301247+0000","last_clean":"2024-09-21T06:29:09.301247+0000","last_became_active":"2024-09-21T06:29:07.321169+0000","last_became_peered":"2024-09-21T06:29:07.321169+0000","last_unstale":"2024-09-21T06:29:09.301247+0000","last_undegraded":"2024-09-21T06:29:09.301247+0000","last_fullsized":"2024-09-21T06:29:09.301247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:53.791260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217918+0000","last_change":"2024-09-21T06:29:08.336833+0000","last_active":"2024-09-21T06:29:11.217918+0000","last_peered":"2024-09-21T06:29:11.217918+0000","last_clean":"2024-09-21T06:29:11.217918+0000","last_became_active":"2024-09-21T06:29:08.336450+0000","last_became_peered":"2024-09-21T06:29:08.336450+0000","last_unstale":"2024-09-21T06:29:11.217918+0000","last_undegraded":"2024-09-21T06:29:11.217918+0000","last_fullsized":"2024-09-21T06:29:11.217918+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:15.837431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930479+0000","last_change":"2024-09-21T06:29:07.313046+0000","last_active":"2024-09-21T06:29:10.930479+0000","last_peered":"2024-09-21T06:29:10.930479+0000","last_clean":"2024-09-21T06:29:10.930479+0000","last_became_active":"2024-09-21T06:29:07.312716+0000","last_became_peered":"2024-09-21T06:29:07.312716+0000","last_unstale":"2024-09-21T06:29:10.930479+0000","last_undegraded":"2024-09-21T06:29:10.930479+0000","last_fullsized":"2024-09-21T06:29:10.930479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:19.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930671+0000","last_change":"2024-09-21T06:29:08.314186+0000","last_active":"2024-09-21T06:29:10.930671+0000","last_peered":"2024-09-21T06:29:10.930671+0000","last_clean":"2024-09-21T06:29:10.930671+0000","last_became_active":"2024-09-21T06:29:08.313738+0000","last_became_peered":"2024-09-21T06:29:08.313738+0000","last_unstale":"2024-09-21T06:29:10.930671+0000","last_undegraded":"2024-09-21T06:29:10.930671+0000","last_fullsized":"2024-09-21T06:29:10.930671+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:20.131671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930102+0000","last_change":"2024-09-21T06:29:07.316562+0000","last_active":"2024-09-21T06:29:10.930102+0000","last_peered":"2024-09-21T06:29:10.930102+0000","last_clean":"2024-09-21T06:29:10.930102+0000","last_became_active":"2024-09-21T06:29:07.316237+0000","last_became_peered":"2024-09-21T06:29:07.316237+0000","last_unstale":"2024-09-21T06:29:10.930102+0000","last_undegraded":"2024-09-21T06:29:10.930102+0000","last_fullsized":"2024-09-21T06:29:10.930102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.759860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302633+0000","last_change":"2024-09-21T06:29:08.335318+0000","last_active":"2024-09-21T06:29:09.302633+0000","last_peered":"2024-09-21T06:29:09.302633+0000","last_clean":"2024-09-21T06:29:09.302633+0000","last_became_active":"2024-09-21T06:29:08.335004+0000","last_became_peered":"2024-09-21T06:29:08.335004+0000","last_unstale":"2024-09-21T06:29:09.302633+0000","last_undegraded":"2024-09-21T06:29:09.302633+0000","last_fullsized":"2024-09-21T06:29:09.302633+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.936391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302926+0000","last_change":"2024-09-21T06:29:07.311950+0000","last_active":"2024-09-21T06:29:09.302926+0000","last_peered":"2024-09-21T06:29:09.302926+0000","last_clean":"2024-09-21T06:29:09.302926+0000","last_became_active":"2024-09-21T06:29:07.311659+0000","last_became_peered":"2024-09-21T06:29:07.311659+0000","last_unstale":"2024-09-21T06:29:09.302926+0000","last_undegraded":"2024-09-21T06:29:09.302926+0000","last_fullsized":"2024-09-21T06:29:09.302926+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:45.349318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179626+0000","last_change":"2024-09-21T06:29:08.333347+0000","last_active":"2024-09-21T06:29:10.179626+0000","last_peered":"2024-09-21T06:29:10.179626+0000","last_clean":"2024-09-21T06:29:10.179626+0000","last_became_active":"2024-09-21T06:29:08.332965+0000","last_became_peered":"2024-09-21T06:29:08.332965+0000","last_unstale":"2024-09-21T06:29:10.179626+0000","last_undegraded":"2024-09-21T06:29:10.179626+0000","last_fullsized":"2024-09-21T06:29:10.179626+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:45.843261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300823+0000","last_change":"2024-09-21T06:29:07.317825+0000","last_active":"2024-09-21T06:29:09.300823+0000","last_peered":"2024-09-21T06:29:09.300823+0000","last_clean":"2024-09-21T06:29:09.300823+0000","last_became_active":"2024-09-21T06:29:07.316929+0000","last_became_peered":"2024-09-21T06:29:07.316929+0000","last_unstale":"2024-09-21T06:29:09.300823+0000","last_undegraded":"2024-09-21T06:29:09.300823+0000","last_fullsized":"2024-09-21T06:29:09.300823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:34.954307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.303286+0000","last_change":"2024-09-21T06:29:08.341260+0000","last_active":"2024-09-21T06:29:09.303286+0000","last_peered":"2024-09-21T06:29:09.303286+0000","last_clean":"2024-09-21T06:29:09.303286+0000","last_became_active":"2024-09-21T06:29:08.340729+0000","last_became_peered":"2024-09-21T06:29:08.340729+0000","last_unstale":"2024-09-21T06:29:09.303286+0000","last_undegraded":"2024-09-21T06:29:09.303286+0000","last_fullsized":"2024-09-21T06:29:09.303286+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:05.781650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:12.370785+0000","last_change":"2024-09-21T06:29:07.316760+0000","last_active":"2024-09-21T06:29:12.370785+0000","last_peered":"2024-09-21T06:29:12.370785+0000","last_clean":"2024-09-21T06:29:12.370785+0000","last_became_active":"2024-09-21T06:29:07.316317+0000","last_became_peered":"2024-09-21T06:29:07.316317+0000","last_unstale":"2024-09-21T06:29:12.370785+0000","last_undegraded":"2024-09-21T06:29:12.370785+0000","last_fullsized":"2024-09-21T06:29:12.370785+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:17.575942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.304019+0000","last_change":"2024-09-21T06:29:08.326873+0000","last_active":"2024-09-21T06:29:09.304019+0000","last_peered":"2024-09-21T06:29:09.304019+0000","last_clean":"2024-09-21T06:29:09.304019+0000","last_became_active":"2024-09-21T06:29:08.326556+0000","last_became_peered":"2024-09-21T06:29:08.326556+0000","last_unstale":"2024-09-21T06:29:09.304019+0000","last_undegraded":"2024-09-21T06:29:09.304019+0000","last_fullsized":"2024-09-21T06:29:09.304019+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:07.194605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.218026+0000","last_change":"2024-09-21T06:29:07.327438+0000","last_active":"2024-09-21T06:29:11.218026+0000","last_peered":"2024-09-21T06:29:11.218026+0000","last_clean":"2024-09-21T06:29:11.218026+0000","last_became_active":"2024-09-21T06:29:07.326953+0000","last_became_peered":"2024-09-21T06:29:07.326953+0000","last_unstale":"2024-09-21T06:29:11.218026+0000","last_undegraded":"2024-09-21T06:29:11.218026+0000","last_fullsized":"2024-09-21T06:29:11.218026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:14.425126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217858+0000","last_change":"2024-09-21T06:29:08.320244+0000","last_active":"2024-09-21T06:29:11.217858+0000","last_peered":"2024-09-21T06:29:11.217858+0000","last_clean":"2024-09-21T06:29:11.217858+0000","last_became_active":"2024-09-21T06:29:08.319895+0000","last_became_peered":"2024-09-21T06:29:08.319895+0000","last_unstale":"2024-09-21T06:29:11.217858+0000","last_undegraded":"2024-09-21T06:29:11.217858+0000","last_fullsized":"2024-09-21T06:29:11.217858+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:56.768191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301829+0000","last_change":"2024-09-21T06:29:07.319663+0000","last_active":"2024-09-21T06:29:09.301829+0000","last_peered":"2024-09-21T06:29:09.301829+0000","last_clean":"2024-09-21T06:29:09.301829+0000","last_became_active":"2024-09-21T06:29:07.319270+0000","last_became_peered":"2024-09-21T06:29:07.319270+0000","last_unstale":"2024-09-21T06:29:09.301829+0000","last_undegraded":"2024-09-21T06:29:09.301829+0000","last_fullsized":"2024-09-21T06:29:09.301829+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.104997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.218152+0000","last_change":"2024-09-21T06:29:08.330802+0000","last_active":"2024-09-21T06:29:11.218152+0000","last_peered":"2024-09-21T06:29:11.218152+0000","last_clean":"2024-09-21T06:29:11.218152+0000","last_became_active":"2024-09-21T06:29:08.324208+0000","last_became_peered":"2024-09-21T06:29:08.324208+0000","last_unstale":"2024-09-21T06:29:11.218152+0000","last_undegraded":"2024-09-21T06:29:11.218152+0000","last_fullsized":"2024-09-21T06:29:11.218152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:53.826046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302548+0000","last_change":"2024-09-21T06:29:07.325336+0000","last_active":"2024-09-21T06:29:09.302548+0000","last_peered":"2024-09-21T06:29:09.302548+0000","last_clean":"2024-09-21T06:29:09.302548+0000","last_became_active":"2024-09-21T06:29:07.324982+0000","last_became_peered":"2024-09-21T06:29:07.324982+0000","last_unstale":"2024-09-21T06:29:09.302548+0000","last_undegraded":"2024-09-21T06:29:09.302548+0000","last_fullsized":"2024-09-21T06:29:09.302548+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:05.553176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336181+0000","last_change":"2024-09-21T06:29:08.326873+0000","last_active":"2024-09-21T06:29:08.336181+0000","last_peered":"2024-09-21T06:29:08.336181+0000","last_clean":"2024-09-21T06:29:08.336181+0000","last_became_active":"2024-09-21T06:29:08.326538+0000","last_became_peered":"2024-09-21T06:29:08.326538+0000","last_unstale":"2024-09-21T06:29:08.336181+0000","last_undegraded":"2024-09-21T06:29:08.336181+0000","last_fullsized":"2024-09-21T06:29:08.336181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:00.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.308732+0000","last_change":"2024-09-21T06:29:07.316341+0000","last_active":"2024-09-21T06:29:08.308732+0000","last_peered":"2024-09-21T06:29:08.308732+0000","last_clean":"2024-09-21T06:29:08.308732+0000","last_became_active":"2024-09-21T06:29:07.316013+0000","last_became_peered":"2024-09-21T06:29:07.316013+0000","last_unstale":"2024-09-21T06:29:08.308732+0000","last_undegraded":"2024-09-21T06:29:08.308732+0000","last_fullsized":"2024-09-21T06:29:08.308732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:10.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300582+0000","last_change":"2024-09-21T06:29:08.334040+0000","last_active":"2024-09-21T06:29:09.300582+0000","last_peered":"2024-09-21T06:29:09.300582+0000","last_clean":"2024-09-21T06:29:09.300582+0000","last_became_active":"2024-09-21T06:29:08.333700+0000","last_became_peered":"2024-09-21T06:29:08.333700+0000","last_unstale":"2024-09-21T06:29:09.300582+0000","last_undegraded":"2024-09-21T06:29:09.300582+0000","last_fullsized":"2024-09-21T06:29:09.300582+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:40.773896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.336776+0000","last_change":"2024-09-21T06:29:07.317403+0000","last_active":"2024-09-21T06:29:08.336776+0000","last_peered":"2024-09-21T06:29:08.336776+0000","last_clean":"2024-09-21T06:29:08.336776+0000","last_became_active":"2024-09-21T06:29:07.317101+0000","last_became_peered":"2024-09-21T06:29:07.317101+0000","last_unstale":"2024-09-21T06:29:08.336776+0000","last_undegraded":"2024-09-21T06:29:08.336776+0000","last_fullsized":"2024-09-21T06:29:08.336776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:02.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930009+0000","last_change":"2024-09-21T06:29:08.322091+0000","last_active":"2024-09-21T06:29:10.930009+0000","last_peered":"2024-09-21T06:29:10.930009+0000","last_clean":"2024-09-21T06:29:10.930009+0000","last_became_active":"2024-09-21T06:29:08.321738+0000","last_became_peered":"2024-09-21T06:29:08.321738+0000","last_unstale":"2024-09-21T06:29:10.930009+0000","last_undegraded":"2024-09-21T06:29:10.930009+0000","last_fullsized":"2024-09-21T06:29:10.930009+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:34.417076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302868+0000","last_change":"2024-09-21T06:29:07.319665+0000","last_active":"2024-09-21T06:29:09.302868+0000","last_peered":"2024-09-21T06:29:09.302868+0000","last_clean":"2024-09-21T06:29:09.302868+0000","last_became_active":"2024-09-21T06:29:07.318846+0000","last_became_peered":"2024-09-21T06:29:07.318846+0000","last_unstale":"2024-09-21T06:29:09.302868+0000","last_undegraded":"2024-09-21T06:29:09.302868+0000","last_fullsized":"2024-09-21T06:29:09.302868+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:09.445848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179342+0000","last_change":"2024-09-21T06:29:08.327413+0000","last_active":"2024-09-21T06:29:10.179342+0000","last_peered":"2024-09-21T06:29:10.179342+0000","last_clean":"2024-09-21T06:29:10.179342+0000","last_became_active":"2024-09-21T06:29:08.327096+0000","last_became_peered":"2024-09-21T06:29:08.327096+0000","last_unstale":"2024-09-21T06:29:10.179342+0000","last_undegraded":"2024-09-21T06:29:10.179342+0000","last_fullsized":"2024-09-21T06:29:10.179342+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:23.564432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217926+0000","last_change":"2024-09-21T06:29:07.325806+0000","last_active":"2024-09-21T06:29:11.217926+0000","last_peered":"2024-09-21T06:29:11.217926+0000","last_clean":"2024-09-21T06:29:11.217926+0000","last_became_active":"2024-09-21T06:29:07.325483+0000","last_became_peered":"2024-09-21T06:29:07.325483+0000","last_unstale":"2024-09-21T06:29:11.217926+0000","last_undegraded":"2024-09-21T06:29:11.217926+0000","last_fullsized":"2024-09-21T06:29:11.217926+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:09.025945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930411+0000","last_change":"2024-09-21T06:29:08.325497+0000","last_active":"2024-09-21T06:29:10.930411+0000","last_peered":"2024-09-21T06:29:10.930411+0000","last_clean":"2024-09-21T06:29:10.930411+0000","last_became_active":"2024-09-21T06:29:08.325200+0000","last_became_peered":"2024-09-21T06:29:08.325200+0000","last_unstale":"2024-09-21T06:29:10.930411+0000","last_undegraded":"2024-09-21T06:29:10.930411+0000","last_fullsized":"2024-09-21T06:29:10.930411+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:53.319131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302291+0000","last_change":"2024-09-21T06:29:07.318878+0000","last_active":"2024-09-21T06:29:09.302291+0000","last_peered":"2024-09-21T06:29:09.302291+0000","last_clean":"2024-09-21T06:29:09.302291+0000","last_became_active":"2024-09-21T06:29:07.317361+0000","last_became_peered":"2024-09-21T06:29:07.317361+0000","last_unstale":"2024-09-21T06:29:09.302291+0000","last_undegraded":"2024-09-21T06:29:09.302291+0000","last_fullsized":"2024-09-21T06:29:09.302291+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:17.575942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302893+0000","last_change":"2024-09-21T06:29:08.340879+0000","last_active":"2024-09-21T06:29:09.302893+0000","last_peered":"2024-09-21T06:29:09.302893+0000","last_clean":"2024-09-21T06:29:09.302893+0000","last_became_active":"2024-09-21T06:29:08.340202+0000","last_became_peered":"2024-09-21T06:29:08.340202+0000","last_unstale":"2024-09-21T06:29:09.302893+0000","last_undegraded":"2024-09-21T06:29:09.302893+0000","last_fullsized":"2024-09-21T06:29:09.302893+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:10.454179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301879+0000","last_change":"2024-09-21T06:29:07.323099+0000","last_active":"2024-09-21T06:29:09.301879+0000","last_peered":"2024-09-21T06:29:09.301879+0000","last_clean":"2024-09-21T06:29:09.301879+0000","last_became_active":"2024-09-21T06:29:07.322763+0000","last_became_peered":"2024-09-21T06:29:07.322763+0000","last_unstale":"2024-09-21T06:29:09.301879+0000","last_undegraded":"2024-09-21T06:29:09.301879+0000","last_fullsized":"2024-09-21T06:29:09.301879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:27.152906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.180103+0000","last_change":"2024-09-21T06:29:08.334626+0000","last_active":"2024-09-21T06:29:10.180103+0000","last_peered":"2024-09-21T06:29:10.180103+0000","last_clean":"2024-09-21T06:29:10.180103+0000","last_became_active":"2024-09-21T06:29:08.334326+0000","last_became_peered":"2024-09-21T06:29:08.334326+0000","last_unstale":"2024-09-21T06:29:10.180103+0000","last_undegraded":"2024-09-21T06:29:10.180103+0000","last_fullsized":"2024-09-21T06:29:10.180103+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:54.799592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302444+0000","last_change":"2024-09-21T06:29:07.325007+0000","last_active":"2024-09-21T06:29:09.302444+0000","last_peered":"2024-09-21T06:29:09.302444+0000","last_clean":"2024-09-21T06:29:09.302444+0000","last_became_active":"2024-09-21T06:29:07.324590+0000","last_became_peered":"2024-09-21T06:29:07.324590+0000","last_unstale":"2024-09-21T06:29:09.302444+0000","last_undegraded":"2024-09-21T06:29:09.302444+0000","last_fullsized":"2024-09-21T06:29:09.302444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:24.915408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T06:29:08.335957+0000","last_change":"2024-09-21T06:29:08.326146+0000","last_active":"2024-09-21T06:29:08.335957+0000","last_peered":"2024-09-21T06:29:08.335957+0000","last_clean":"2024-09-21T06:29:08.335957+0000","last_became_active":"2024-09-21T06:29:08.325802+0000","last_became_peered":"2024-09-21T06:29:08.325802+0000","last_unstale":"2024-09-21T06:29:08.335957+0000","last_undegraded":"2024-09-21T06:29:08.335957+0000","last_fullsized":"2024-09-21T06:29:08.335957+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:57.060675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302714+0000","last_change":"2024-09-21T06:29:07.324710+0000","last_active":"2024-09-21T06:29:09.302714+0000","last_peered":"2024-09-21T06:29:09.302714+0000","last_clean":"2024-09-21T06:29:09.302714+0000","last_became_active":"2024-09-21T06:29:07.324348+0000","last_became_peered":"2024-09-21T06:29:07.324348+0000","last_unstale":"2024-09-21T06:29:09.302714+0000","last_undegraded":"2024-09-21T06:29:09.302714+0000","last_fullsized":"2024-09-21T06:29:09.302714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:43.337772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.303601+0000","last_change":"2024-09-21T06:29:08.338016+0000","last_active":"2024-09-21T06:29:09.303601+0000","last_peered":"2024-09-21T06:29:09.303601+0000","last_clean":"2024-09-21T06:29:09.303601+0000","last_became_active":"2024-09-21T06:29:08.336292+0000","last_became_peered":"2024-09-21T06:29:08.336292+0000","last_unstale":"2024-09-21T06:29:09.303601+0000","last_undegraded":"2024-09-21T06:29:09.303601+0000","last_fullsized":"2024-09-21T06:29:09.303601+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:56.768191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930851+0000","last_change":"2024-09-21T06:29:07.318527+0000","last_active":"2024-09-21T06:29:10.930851+0000","last_peered":"2024-09-21T06:29:10.930851+0000","last_clean":"2024-09-21T06:29:10.930851+0000","last_became_active":"2024-09-21T06:29:07.318203+0000","last_became_peered":"2024-09-21T06:29:07.318203+0000","last_unstale":"2024-09-21T06:29:10.930851+0000","last_undegraded":"2024-09-21T06:29:10.930851+0000","last_fullsized":"2024-09-21T06:29:10.930851+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:14.829099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.302917+0000","last_change":"2024-09-21T06:29:08.328635+0000","last_active":"2024-09-21T06:29:09.302917+0000","last_peered":"2024-09-21T06:29:09.302917+0000","last_clean":"2024-09-21T06:29:09.302917+0000","last_became_active":"2024-09-21T06:29:08.328336+0000","last_became_peered":"2024-09-21T06:29:08.328336+0000","last_unstale":"2024-09-21T06:29:09.302917+0000","last_undegraded":"2024-09-21T06:29:09.302917+0000","last_fullsized":"2024-09-21T06:29:09.302917+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:34.969770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179593+0000","last_change":"2024-09-21T06:29:07.318905+0000","last_active":"2024-09-21T06:29:10.179593+0000","last_peered":"2024-09-21T06:29:10.179593+0000","last_clean":"2024-09-21T06:29:10.179593+0000","last_became_active":"2024-09-21T06:29:07.317691+0000","last_became_peered":"2024-09-21T06:29:07.317691+0000","last_unstale":"2024-09-21T06:29:10.179593+0000","last_undegraded":"2024-09-21T06:29:10.179593+0000","last_fullsized":"2024-09-21T06:29:10.179593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:45.349318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300943+0000","last_change":"2024-09-21T06:29:08.308035+0000","last_active":"2024-09-21T06:29:09.300943+0000","last_peered":"2024-09-21T06:29:09.300943+0000","last_clean":"2024-09-21T06:29:09.300943+0000","last_became_active":"2024-09-21T06:29:08.307717+0000","last_became_peered":"2024-09-21T06:29:08.307717+0000","last_unstale":"2024-09-21T06:29:09.300943+0000","last_undegraded":"2024-09-21T06:29:09.300943+0000","last_fullsized":"2024-09-21T06:29:09.300943+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:28.277051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.300911+0000","last_change":"2024-09-21T06:29:07.318752+0000","last_active":"2024-09-21T06:29:09.300911+0000","last_peered":"2024-09-21T06:29:09.300911+0000","last_clean":"2024-09-21T06:29:09.300911+0000","last_became_active":"2024-09-21T06:29:07.318441+0000","last_became_peered":"2024-09-21T06:29:07.318441+0000","last_unstale":"2024-09-21T06:29:09.300911+0000","last_undegraded":"2024-09-21T06:29:09.300911+0000","last_fullsized":"2024-09-21T06:29:09.300911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:02.453310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179548+0000","last_change":"2024-09-21T06:29:08.319050+0000","last_active":"2024-09-21T06:29:10.179548+0000","last_peered":"2024-09-21T06:29:10.179548+0000","last_clean":"2024-09-21T06:29:10.179548+0000","last_became_active":"2024-09-21T06:29:08.318697+0000","last_became_peered":"2024-09-21T06:29:08.318697+0000","last_unstale":"2024-09-21T06:29:10.179548+0000","last_undegraded":"2024-09-21T06:29:10.179548+0000","last_fullsized":"2024-09-21T06:29:10.179548+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:47.346724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.930923+0000","last_change":"2024-09-21T06:29:07.318921+0000","last_active":"2024-09-21T06:29:10.930923+0000","last_peered":"2024-09-21T06:29:10.930923+0000","last_clean":"2024-09-21T06:29:10.930923+0000","last_became_active":"2024-09-21T06:29:07.318259+0000","last_became_peered":"2024-09-21T06:29:07.318259+0000","last_unstale":"2024-09-21T06:29:10.930923+0000","last_undegraded":"2024-09-21T06:29:10.930923+0000","last_fullsized":"2024-09-21T06:29:10.930923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:09.301698+0000","last_change":"2024-09-21T06:29:08.315807+0000","last_active":"2024-09-21T06:29:09.301698+0000","last_peered":"2024-09-21T06:29:09.301698+0000","last_clean":"2024-09-21T06:29:09.301698+0000","last_became_active":"2024-09-21T06:29:08.315492+0000","last_became_peered":"2024-09-21T06:29:08.315492+0000","last_unstale":"2024-09-21T06:29:09.301698+0000","last_undegraded":"2024-09-21T06:29:09.301698+0000","last_fullsized":"2024-09-21T06:29:09.301698+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:07.285114+0000","last_clean_scrub_stamp":"2024-09-21T06:29:07.285114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:27.626576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:10.179474+0000","last_change":"2024-09-21T06:29:07.321833+0000","last_active":"2024-09-21T06:29:10.179474+0000","last_peered":"2024-09-21T06:29:10.179474+0000","last_clean":"2024-09-21T06:29:10.179474+0000","last_became_active":"2024-09-21T06:29:07.321550+0000","last_became_peered":"2024-09-21T06:29:07.321550+0000","last_unstale":"2024-09-21T06:29:10.179474+0000","last_undegraded":"2024-09-21T06:29:10.179474+0000","last_fullsized":"2024-09-21T06:29:10.179474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:06.276782+0000","last_clean_scrub_stamp":"2024-09-21T06:29:06.276782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:09.445848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":146,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T06:29:11.217343+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:29:11.217343+0000","last_peered":"2024-09-21T06:29:11.217343+0000","last_clean":"2024-09-21T06:29:11.217343+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:29:11.217343+0000","last_undegraded":"2024-09-21T06:29:11.217343+0000","last_fullsized":"2024-09-21T06:29:11.217343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":471040,"data_stored":199693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":471040,"data_stored":199657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":201359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":471040,"data_stored":199713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":201817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1064960,"data_stored":790013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":199715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":783881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:29:13.323 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:29:13.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.594+0000 7febc3400640 1 -- 172.21.15.37:0/3074807231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba0003a50 msgr2=0x7feba0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc3400640 1 --2- 172.21.15.37:0/3074807231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba0003a50 0x7feba0003ec0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febac00c470 tx=0x7feba0005830 comp rx=0 tx=0).stop 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc1400640 1 -- 172.21.15.37:0/3074807231 reap_dead start 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc3400640 1 -- 172.21.15.37:0/3074807231 shutdown_connections 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc3400640 1 -- 172.21.15.37:0/3074807231 >> 172.21.15.37:0/3074807231 conn(0x7febbc0d4440 msgr2=0x7febbc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc3400640 1 -- 172.21.15.37:0/3074807231 shutdown_connections 2024-09-21T06:29:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.595+0000 7febc3400640 1 -- 172.21.15.37:0/3074807231 wait complete. 2024-09-21T06:29:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.597+0000 7febc3400640 1 Processor -- start 2024-09-21T06:29:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.597+0000 7febc3400640 1 -- start start 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.598+0000 7febc3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba0003a50 0x7febbc1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc3400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febbc0cb650 0x7febbc1f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc3400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febbc162b20 con 0x7febbc0cad00 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc3400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1629a0 con 0x7feba0003a50 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc3400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7febbc162ca0 con 0x7febbc0cb650 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49956/0 (socket says 172.21.15.37:49956) 2024-09-21T06:29:13.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 -- 172.21.15.37:0/4104527892 learned_addr learned my addr 172.21.15.37:0/4104527892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:13.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc1400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febbc0cb650 0x7febbc1f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febc0a00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba0003a50 0x7febbc1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 -- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febbc0cb650 msgr2=0x7febbc1f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febbc0cb650 0x7febbc1f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 -- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba0003a50 msgr2=0x7febbc1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba0003a50 0x7febbc1f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.599+0000 7febb3e00640 1 -- 172.21.15.37:0/4104527892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc1f5250 con 0x7febbc0cad00 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febb3e00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feba4011b10 tx=0x7feba4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba401e070 con 0x7febbc0cad00 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feba4004050 con 0x7febbc0cad00 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba400fea0 con 0x7febbc0cad00 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc161070 con 0x7febbc0cad00 2024-09-21T06:29:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.600+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc0ca500 con 0x7febbc0cad00 2024-09-21T06:29:13.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.601+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc160de0 con 0x7febbc0cad00 2024-09-21T06:29:13.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.603+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feba4021070 con 0x7febbc0cad00 2024-09-21T06:29:13.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.603+0000 7febb9600640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:13.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.604+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7feba409b970 con 0x7febbc0cad00 2024-09-21T06:29:13.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.604+0000 7febc0a00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:13.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.605+0000 7febc0a00640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7febac033000 tx=0x7febbc0d08e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.608+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feba40044a0 con 0x7febbc0cad00 2024-09-21T06:29:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.900+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7febbc0b3a70 con 0x7febbc0cad00 2024-09-21T06:29:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.903+0000 7febb9600640 1 -- 172.21.15.37:0/4104527892 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7feba4063680 con 0x7febbc0cad00 2024-09-21T06:29:13.905 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:29:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:29:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:29:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:29:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:29:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 msgr2=0x7feb9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7febac033000 tx=0x7febbc0d08e0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 msgr2=0x7febbc0cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feba4011b10 tx=0x7feba4009e90 comp rx=0 tx=0).stop 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 shutdown_connections 2024-09-21T06:29:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feb98078b90 0x7feb9807b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febbc0cad00 0x7febbc0cb110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febbc0cb650 0x7febbc1f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 --2- 172.21.15.37:0/4104527892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba0003a50 0x7febbc1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 >> 172.21.15.37:0/4104527892 conn(0x7febbc0d4440 msgr2=0x7febbc0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 shutdown_connections 2024-09-21T06:29:13.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:13.906+0000 7febc3400640 1 -- 172.21.15.37:0/4104527892 wait complete. 2024-09-21T06:29:13.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:29:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 -- 172.21.15.37:0/3551552594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c0cbc10 msgr2=0x7f0b7c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 --2- 172.21.15.37:0/3551552594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c0cc020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009830 tx=0x7f0b6802a510 comp rx=0 tx=0).stop 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 -- 172.21.15.37:0/3551552594 shutdown_connections 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 --2- 172.21.15.37:0/3551552594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c0cc020 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 --2- 172.21.15.37:0/3551552594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 0x7f0b7c073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 -- 172.21.15.37:0/3551552594 >> 172.21.15.37:0/3551552594 conn(0x7f0b7c0cca40 msgr2=0x7f0b7c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 -- 172.21.15.37:0/3551552594 shutdown_connections 2024-09-21T06:29:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.156+0000 7f0b84600640 1 -- 172.21.15.37:0/3551552594 wait complete. 2024-09-21T06:29:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.158+0000 7f0b84600640 1 Processor -- start 2024-09-21T06:29:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.158+0000 7f0b84600640 1 -- start start 2024-09-21T06:29:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c06ddf0 0x7f0b7c1f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 0x7f0b7c09f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c06f360 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c06f1e0 con 0x7f0b7c06ddf0 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b84600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c06f4e0 con 0x7f0b7c074cb0 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b82600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b82600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50398/0 (socket says 172.21.15.37:50398) 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b82600640 1 -- 172.21.15.37:0/127892846 learned_addr learned my addr 172.21.15.37:0/127892846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b81200640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 0x7f0b7c09f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:14.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.160+0000 7f0b81c00640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c06ddf0 0x7f0b7c1f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 -- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 msgr2=0x7f0b7c09f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 0x7f0b7c09f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 -- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c06ddf0 msgr2=0x7f0b7c1f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c06ddf0 0x7f0b7c1f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 -- 172.21.15.37:0/127892846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0b5bb0 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b82600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c730 tx=0x7f0b7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74003b70 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b74003d30 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c09bf60 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.161+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74005330 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.162+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c09c520 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.163+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c06fa00 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.164+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b740056e0 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.165+0000 7f0b79600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 0x7f0b5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.165+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0b7409ad80 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.165+0000 7f0b81c00640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 0x7f0b5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.166+0000 7f0b81c00640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 0x7f0b5c07b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009530 tx=0x7f0b70009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:14.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.169+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b740629e0 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.408+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7c15c660 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.408+0000 7f0b79600640 1 -- 172.21.15.37:0/127892846 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f0b74067410 con 0x7f0b7c0cbc10 2024-09-21T06:29:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:14.411 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"btime":"2024-09-21T06:29:11:331581+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:10.839150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":2}]} 2024-09-21T06:29:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.410+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 msgr2=0x7f0b5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.410+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 0x7f0b5c07b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009530 tx=0x7f0b70009260 comp rx=0 tx=0).stop 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.410+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 msgr2=0x7f0b7c09f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.410+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c730 tx=0x7f0b7400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 shutdown_connections 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0b5c078bc0 0x7f0b5c07b0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b7c0cbc10 0x7f0b7c09f700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b7c074cb0 0x7f0b7c09f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 --2- 172.21.15.37:0/127892846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c06ddf0 0x7f0b7c1f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:14.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 >> 172.21.15.37:0/127892846 conn(0x7f0b7c0cca40 msgr2=0x7f0b7c0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:14.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 shutdown_connections 2024-09-21T06:29:14.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:14.411+0000 7f0b84600640 1 -- 172.21.15.37:0/127892846 wait complete. 2024-09-21T06:29:14.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:29:07.816070+0000', 'modified': '2024-09-21T06:29:10.839150+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4984}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4984': {'gid': 4984, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/337579449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 337579449}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 337579449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4984, 'qdb_cluster': [4984]} 2024-09-21T06:29:14.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:29:14.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 -- 172.21.15.37:0/1193718665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaf0003a50 msgr2=0x7ffaf0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 --2- 172.21.15.37:0/1193718665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaf0003a50 0x7ffaf0003ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009b50 tx=0x7ffaf0005830 comp rx=0 tx=0).stop 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 -- 172.21.15.37:0/1193718665 shutdown_connections 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 --2- 172.21.15.37:0/1193718665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaf0003a50 0x7ffaf0003ec0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 -- 172.21.15.37:0/1193718665 >> 172.21.15.37:0/1193718665 conn(0x7ffb0c0b1b90 msgr2=0x7ffb0c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 -- 172.21.15.37:0/1193718665 shutdown_connections 2024-09-21T06:29:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.068+0000 7ffb11c00640 1 -- 172.21.15.37:0/1193718665 wait complete. 2024-09-21T06:29:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.071+0000 7ffb11c00640 1 Processor -- start 2024-09-21T06:29:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.071+0000 7ffb11c00640 1 -- start start 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 0x7ffb0c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb0c156430 0x7ffb0c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c0749b0 con 0x7ffaf0003a50 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c074830 con 0x7ffb0c156020 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb11c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0c06ddf0 con 0x7ffb0c156430 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb0aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb0aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50418/0 (socket says 172.21.15.37:50418) 2024-09-21T06:29:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.072+0000 7ffb0aa00640 1 -- 172.21.15.37:0/2075197824 learned_addr learned my addr 172.21.15.37:0/2075197824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb03e00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 0x7ffb0c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0b400640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb0c156430 0x7ffb0c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 -- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb0c156430 msgr2=0x7ffb0c158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb0c156430 0x7ffb0c158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 -- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 msgr2=0x7ffb0c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 0x7ffb0c1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 -- 172.21.15.37:0/2075197824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c1f2e80 con 0x7ffaf0003a50 2024-09-21T06:29:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb03e00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 0x7ffb0c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009a70 tx=0x7ffafc0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.073+0000 7ffb02a00640 1 -- 172.21.15.37:0/2075197824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc038040 con 0x7ffaf0003a50 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.074+0000 7ffb02a00640 1 -- 172.21.15.37:0/2075197824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffafc041070 con 0x7ffaf0003a50 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.074+0000 7ffb02a00640 1 -- 172.21.15.37:0/2075197824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc03c430 con 0x7ffaf0003a50 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.074+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c1f3130 con 0x7ffaf0003a50 2024-09-21T06:29:15.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.074+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c1f36f0 con 0x7ffaf0003a50 2024-09-21T06:29:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.076+0000 7ffb02a00640 1 -- 172.21.15.37:0/2075197824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffafc03c5d0 con 0x7ffaf0003a50 2024-09-21T06:29:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.077+0000 7ffb02a00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 0x7ffae807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.077+0000 7ffb02a00640 1 -- 172.21.15.37:0/2075197824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ffafc0bb9c0 con 0x7ffaf0003a50 2024-09-21T06:29:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.078+0000 7ffb03e00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 0x7ffae807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.078+0000 7ffb18c0a740 1 Processor -- start 2024-09-21T06:29:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.078+0000 7ffb18c0a740 1 -- start start 2024-09-21T06:29:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.078+0000 7ffb03e00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 0x7ffae807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c15c5a0 tx=0x7ffaf4006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 0x55a2f699f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f699feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2f68f9430 con 0x55a2f65ae590 2024-09-21T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2f68e95d0 con 0x55a2f65a63b0 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.079+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2f68d8a00 con 0x55a2f65cf270 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50420/0 (socket says 172.21.15.37:50420) 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 -- 172.21.15.37:0/701361316 learned_addr learned my addr 172.21.15.37:0/701361316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 0x55a2f699f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb03e00640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f699feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 -- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 msgr2=0x55a2f699f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 0x55a2f699f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 -- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 msgr2=0x55a2f699feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f699feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 -- 172.21.15.37:0/701361316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f68a1520 con 0x55a2f65ae590 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 0x55a2f699f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.080+0000 7ffb0b400640 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b986794af0f33c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffafb400640 1 -- 172.21.15.37:0/701361316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400c260 con 0x55a2f65ae590 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffafb400640 1 -- 172.21.15.37:0/701361316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb04016030 con 0x55a2f65ae590 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffafb400640 1 -- 172.21.15.37:0/701361316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04012440 con 0x55a2f65ae590 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 -- 172.21.15.37:0/701361316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 msgr2=0x55a2f696ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04003d40 comp rx=0 tx=0).stop 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 -- 172.21.15.37:0/701361316 shutdown_connections 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f696ff60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f699feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/701361316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65cf270 0x55a2f699f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.081+0000 7ffb18c0a740 1 -- 172.21.15.37:0/701361316 >> 172.21.15.37:0/701361316 conn(0x55a2f65c33c0 msgr2=0x55a2f6975970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.082+0000 7ffb18c0a740 1 -- 172.21.15.37:0/701361316 shutdown_connections 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.082+0000 7ffb18c0a740 1 -- 172.21.15.37:0/701361316 wait complete. 2024-09-21T06:29:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.083+0000 7ffb18c0a740 1 -- start start 2024-09-21T06:29:15.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.083+0000 7ffb18c0a740 1 Processor -- start 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f69baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2f687f330 con 0x55a2f65cf270 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2f68dc810 con 0x55a2f65a63b0 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2f6649620 con 0x55a2f65ae590 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.084+0000 7ffb18c0a740 10 client.0 _ll_register_callbacks cb 0x55a2f6884420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50436/0 (socket says 172.21.15.37:50436) 2024-09-21T06:29:15.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 -- 172.21.15.37:0/1278309594 learned_addr learned my addr 172.21.15.37:0/1278309594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 -- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 msgr2=0x55a2f69c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 -- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 msgr2=0x55a2f69baf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f69baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 -- 172.21.15.37:0/1278309594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f663a850 con 0x55a2f65cf270 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffb0b400640 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c075bd0 tx=0x7ffb04006610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d565a360b57aab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.085+0000 7ffaefe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04012850 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb04016030 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ffb0401dc70 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ffb0402c050 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 10 client.5127 fetch_fsmap learned FSMap version 17 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a2f66b16b0 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb04016030 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7ffb0402f020 con 0x55a2f65cf270 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 10 client.5127 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 10 client.5127 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 20 client.5127 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f658de40 0x55a2f6984e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 4 client.5127 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.086+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f68eb520 con 0x55a2f658de40 2024-09-21T06:29:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.087+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f658de40 0x55a2f6984e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.088+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f658de40 0x55a2f6984e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c5fa3d5c4b4761 server_cookie=cbd88153b6c67e38 in_seq=0 out_seq=0 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.088+0000 7ffaefe00640 10 client.5127 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffaefe00640 1 -- 172.21.15.37:0/1278309594 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2f68eb520 con 0x55a2f658de40 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffaefe00640 10 client.5127 handle_command_reply: tid=0 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 client.5127 shutdown 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 msgr2=0x55a2f69cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c075bd0 tx=0x7ffb04006610 comp rx=0 tx=0).stop 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 shutdown_connections 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f658de40 0x55a2f6984e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f69cd060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffaefe00640 0 client.5127 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/1278309594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f69baf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 >> 172.21.15.37:0/1278309594 conn(0x55a2f65c33c0 msgr2=0x55a2f69bb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 shutdown_connections 2024-09-21T06:29:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 1 -- 172.21.15.37:0/1278309594 wait complete. 2024-09-21T06:29:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.089+0000 7ffb18c0a740 20 client.5127 trim_cache size 0 max 16384 2024-09-21T06:29:15.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.109+0000 7ffb18c0a740 1 Processor -- start 2024-09-21T06:29:15.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.109+0000 7ffb18c0a740 1 -- start start 2024-09-21T06:29:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65a63b0 0x55a2f69757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65cf270 0x55a2f69de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1ff620 con 0x55a2f65ae590 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1fe980 con 0x55a2f65cf270 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0c1fefd0 con 0x55a2f65a63b0 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50444/0 (socket says 172.21.15.37:50444) 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.110+0000 7ffb03e00640 1 -- 172.21.15.37:0/4045358395 learned_addr learned my addr 172.21.15.37:0/4045358395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb0b400640 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65cf270 0x55a2f69de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65a63b0 0x55a2f69757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 -- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65a63b0 msgr2=0x55a2f69757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65a63b0 0x55a2f69757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 -- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65cf270 msgr2=0x55a2f69de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65cf270 0x55a2f69de510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 -- 172.21.15.37:0/4045358395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f69b6520 con 0x55a2f65ae590 2024-09-21T06:29:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffb03e00640 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4011910 tx=0x7ffaf4014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d498ea1a3402a5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffafaa00640 1 -- 172.21.15.37:0/4045358395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4012640 con 0x55a2f65ae590 2024-09-21T06:29:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffafaa00640 1 -- 172.21.15.37:0/4045358395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf40127e0 con 0x55a2f65ae590 2024-09-21T06:29:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.111+0000 7ffafaa00640 1 -- 172.21.15.37:0/4045358395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4012bf0 con 0x55a2f65ae590 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4045358395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 msgr2=0x55a2f6975d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4011910 tx=0x7ffaf4014070 comp rx=0 tx=0).stop 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4045358395 shutdown_connections 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65ae590 0x55a2f6975d30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65cf270 0x55a2f69de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4045358395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65a63b0 0x55a2f69757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4045358395 >> 172.21.15.37:0/4045358395 conn(0x55a2f65c33c0 msgr2=0x55a2f6998320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4045358395 shutdown_connections 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.112+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4045358395 wait complete. 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.113+0000 7ffb18c0a740 1 -- start start 2024-09-21T06:29:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.113+0000 7ffb18c0a740 1 Processor -- start 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f6a3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2f69b6220 con 0x55a2f65cf270 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2f69bb4b0 con 0x55a2f65a63b0 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2f6983ba0 con 0x55a2f65ae590 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb18c0a740 10 client.0 _ll_register_callbacks cb 0x7ffb04001770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50454/0 (socket says 172.21.15.37:50454) 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb0b400640 1 -- 172.21.15.37:0/4127465799 learned_addr learned my addr 172.21.15.37:0/4127465799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb03e00640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.115+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f6a3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 -- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 msgr2=0x55a2f69757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 -- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 msgr2=0x55a2f6a3d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f6a3d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 -- 172.21.15.37:0/4127465799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f6976650 con 0x55a2f65cf270 2024-09-21T06:29:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb0b400640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb04014dc0 tx=0x7ffb04014df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb6337d1d058287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffaefe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04012ce0 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ffb04026c40 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.116+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf40127e0 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ffb04006680 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 10 client.5133 fetch_fsmap learned FSMap version 17 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a2f69d4480 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf40127e0 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7ffb0402f030 con 0x55a2f65cf270 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 10 client.5133 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 10 client.5133 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 20 client.5133 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f6a3e8a0 0x55a2f6970c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 4 client.5133 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:29:15.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.117+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "status"}) -- 0x55a2f68e6fd0 con 0x55a2f6a3e8a0 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.118+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f6a3e8a0 0x55a2f6970c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.118+0000 7ffb0aa00640 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f6a3e8a0 0x55a2f6970c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a9f0824f345f33 server_cookie=39bdb9abc6db6e3f in_seq=0 out_seq=0 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.118+0000 7ffaefe00640 10 client.5133 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffaefe00640 1 -- 172.21.15.37:0/4127465799 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a2f68e6fd0 con 0x55a2f6a3e8a0 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffaefe00640 10 client.5133 handle_command_reply: tid=0 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 client.5133 shutdown 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 msgr2=0x55a2f6975d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb04014dc0 tx=0x7ffb04014df0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 shutdown_connections 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x55a2f6a3e8a0 0x55a2f6970c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2f65cf270 0x55a2f6975d30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2f65ae590 0x55a2f69757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 --2- 172.21.15.37:0/4127465799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2f65a63b0 0x55a2f6a3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffaefe00640 0 client.5133 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 >> 172.21.15.37:0/4127465799 conn(0x55a2f65c33c0 msgr2=0x55a2f6984dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 shutdown_connections 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 1 -- 172.21.15.37:0/4127465799 wait complete. 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.119+0000 7ffb18c0a740 20 client.5133 trim_cache size 0 max 16384 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4984, 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 6.8216796070000001, 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 14.558659925000001 2024-09-21T06:29:15.125 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 msgr2=0x7ffae807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 0x7ffae807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c15c5a0 tx=0x7ffaf4006b70 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 msgr2=0x7ffb0c0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009a70 tx=0x7ffafc0336f0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 shutdown_connections 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffae8078b90 0x7ffae807b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb0c156430 0x7ffb0c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0c156020 0x7ffb0c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 --2- 172.21.15.37:0/2075197824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf0003a50 0x7ffb0c0c9970 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 >> 172.21.15.37:0/2075197824 conn(0x7ffb0c0b1b90 msgr2=0x7ffb0c0b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 shutdown_connections 2024-09-21T06:29:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.120+0000 7ffb11c00640 1 -- 172.21.15.37:0/2075197824 wait complete. 2024-09-21T06:29:15.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.369+0000 7fdadd000640 1 -- 172.21.15.37:0/1062349725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad81627f0 msgr2=0x7fdad806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.369+0000 7fdadd000640 1 --2- 172.21.15.37:0/1062349725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad81627f0 0x7fdad806d2b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009830 tx=0x7fdac002a500 comp rx=0 tx=0).stop 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 -- 172.21.15.37:0/1062349725 shutdown_connections 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 --2- 172.21.15.37:0/1062349725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad81627f0 0x7fdad806d2b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 --2- 172.21.15.37:0/1062349725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad80b9e40 0x7fdad81622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 -- 172.21.15.37:0/1062349725 >> 172.21.15.37:0/1062349725 conn(0x7fdad80d1930 msgr2=0x7fdad80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 -- 172.21.15.37:0/1062349725 shutdown_connections 2024-09-21T06:29:15.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.370+0000 7fdadd000640 1 -- 172.21.15.37:0/1062349725 wait complete. 2024-09-21T06:29:15.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.372+0000 7fdadd000640 1 Processor -- start 2024-09-21T06:29:15.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.372+0000 7fdadd000640 1 -- start start 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad806d7f0 0x7fdad815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad80b9e40 0x7fdad80c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8155940 con 0x7fdad81627f0 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81557c0 con 0x7fdad80b9e40 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.373+0000 7fdadd000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdad8155ac0 con 0x7fdad806d7f0 2024-09-21T06:29:15.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50466/0 (socket says 172.21.15.37:50466) 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 -- 172.21.15.37:0/3144782224 learned_addr learned my addr 172.21.15.37:0/3144782224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7400640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad806d7f0 0x7fdad815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdacfe00640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad80b9e40 0x7fdad80c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 -- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad806d7f0 msgr2=0x7fdad815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad806d7f0 0x7fdad815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 -- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad80b9e40 msgr2=0x7fdad80c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad80b9e40 0x7fdad80c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 -- 172.21.15.37:0/3144782224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad81594b0 con 0x7fdad81627f0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.374+0000 7fdad7e00640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdad000caa0 tx=0x7fdad000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.375+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0016020 con 0x7fdad81627f0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.375+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdad000acf0 con 0x7fdad81627f0 2024-09-21T06:29:15.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.375+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0012680 con 0x7fdad81627f0 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.375+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad815c430 con 0x7fdad81627f0 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.375+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad815be70 con 0x7fdad81627f0 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.376+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad81622b0 con 0x7fdad81627f0 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.377+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdad0019020 con 0x7fdad81627f0 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.378+0000 7fdacf400640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 0x7fdab407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:15.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.378+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdad009b5f0 con 0x7fdad81627f0 2024-09-21T06:29:15.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.379+0000 7fdad7400640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 0x7fdab407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:15.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.379+0000 7fdad7400640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 0x7fdab407b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009ac0 tx=0x7fdac8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:15.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.382+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdad0020020 con 0x7fdad81627f0 2024-09-21T06:29:15.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.616+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdad8154f20 con 0x7fdad81627f0 2024-09-21T06:29:15.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.617+0000 7fdacf400640 1 -- 172.21.15.37:0/3144782224 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fdad8163170 con 0x7fdad81627f0 2024-09-21T06:29:15.619 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:29:15.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 msgr2=0x7fdab407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 0x7fdab407b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009ac0 tx=0x7fdac8005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 msgr2=0x7fdad81f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdad000caa0 tx=0x7fdad000cf60 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 shutdown_connections 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdab4078b90 0x7fdab407b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdad81627f0 0x7fdad81f9120 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad80b9e40 0x7fdad80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 --2- 172.21.15.37:0/3144782224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdad806d7f0 0x7fdad815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 >> 172.21.15.37:0/3144782224 conn(0x7fdad80d1930 msgr2=0x7fdad815cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 shutdown_connections 2024-09-21T06:29:15.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15.619+0000 7fdadd000640 1 -- 172.21.15.37:0/3144782224 wait complete. 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:29:15.644 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:29:15.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:15.645 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:29:15.680 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:29:15.680 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:29:15.680 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:29:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42636sec preferred_lft 42636sec 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:29:15.682 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:29:15.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:15.684 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:29:15.774 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:29:15.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:15.777 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:29:15.813 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:29:15.813 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:29:15.813 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:29:15.813 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:29:15.814 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:29:15.814 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:29:15.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:29:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:16.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:16.011 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:29:16.047 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:29:16.047 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:29:16.047 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:29:16.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:16.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:29:16.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:29:16.143 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:29:16.178 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:29:16.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:29:16.178 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:29:16.252 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:29:16.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:16.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:29:16.452 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:29:16.452 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:29:16.452 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:29:16.473 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:29:16.473 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:29:16.473 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:29:16.473 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:29:16.473 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:29:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:29:16.506 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:29:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:29:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:29:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:29:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:29:16.509 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:29:16.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:16.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:29:16.566 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:29:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:29:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:16.668 INFO:teuthology.orchestra.run.smithi037.stdout:client5155 2024-09-21T06:29:16.668 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:29:16.668 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:29:16.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:29:16.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 -- 172.21.15.37:0/4076608021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 msgr2=0x7f2db40ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:16.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 --2- 172.21.15.37:0/4076608021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db40ae970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009830 tx=0x7f2da802a480 comp rx=0 tx=0).stop 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 -- 172.21.15.37:0/4076608021 shutdown_connections 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 --2- 172.21.15.37:0/4076608021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40aeeb0 0x7f2db40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 --2- 172.21.15.37:0/4076608021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db40ae970 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 -- 172.21.15.37:0/4076608021 >> 172.21.15.37:0/4076608021 conn(0x7f2db40cdca0 msgr2=0x7f2db40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 -- 172.21.15.37:0/4076608021 shutdown_connections 2024-09-21T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.975+0000 7f2dbc200640 1 -- 172.21.15.37:0/4076608021 wait complete. 2024-09-21T06:29:16.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.977+0000 7f2dbc200640 1 Processor -- start 2024-09-21T06:29:16.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.977+0000 7f2dbc200640 1 -- start start 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2db40aeeb0 0x7f2db41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40b31e0 0x7f2db41f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4074730 con 0x7f2db415c990 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40745b0 con 0x7f2db40aeeb0 2024-09-21T06:29:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dbc200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40748b0 con 0x7f2db40b31e0 2024-09-21T06:29:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dba200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dba200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50480/0 (socket says 172.21.15.37:50480) 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2dba200640 1 -- 172.21.15.37:0/1940779362 learned_addr learned my addr 172.21.15.37:0/1940779362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2db8e00640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40b31e0 0x7f2db41f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.979+0000 7f2db9800640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2db40aeeb0 0x7f2db41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 -- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40b31e0 msgr2=0x7f2db41f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40b31e0 0x7f2db41f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 -- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2db40aeeb0 msgr2=0x7f2db41511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2db40aeeb0 0x7f2db41511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 -- 172.21.15.37:0/1940779362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4159c90 con 0x7f2db415c990 2024-09-21T06:29:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dba200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c760 tx=0x7f2dac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac003b70 con 0x7f2db415c990 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dac003d30 con 0x7f2db415c990 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0052d0 con 0x7f2db415c990 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db4151720 con 0x7f2db415c990 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.980+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db409adf0 con 0x7f2db415c990 2024-09-21T06:29:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.981+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db415f8d0 con 0x7f2db415c990 2024-09-21T06:29:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.983+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2dac005470 con 0x7f2db415c990 2024-09-21T06:29:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.984+0000 7f2db1600640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 0x7f2d9407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.984+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f2dac09ab60 con 0x7f2db415c990 2024-09-21T06:29:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.984+0000 7f2db9800640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 0x7f2d9407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.985+0000 7f2db9800640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 0x7f2d9407b0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2da8034000 tx=0x7f2da8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:16.989+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2db40b35f0 con 0x7f2db415c990 2024-09-21T06:29:17.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.229+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2db409aff0 con 0x7f2db415c990 2024-09-21T06:29:17.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.372+0000 7f2db1600640 1 -- 172.21.15.37:0/1940779362 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f2dac0627b0 con 0x7f2db415c990 2024-09-21T06:29:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 msgr2=0x7f2d9407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 0x7f2d9407b0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2da8034000 tx=0x7f2da8004730 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 msgr2=0x7f2db41f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c760 tx=0x7f2dac00cc20 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 shutdown_connections 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2d94078bc0 0x7f2d9407b0d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db415c990 0x7f2db41f7450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2db40b31e0 0x7f2db41f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 --2- 172.21.15.37:0/1940779362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2db40aeeb0 0x7f2db41511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 >> 172.21.15.37:0/1940779362 conn(0x7f2db40cdca0 msgr2=0x7f2db40b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 shutdown_connections 2024-09-21T06:29:17.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.375+0000 7f2dbc200640 1 -- 172.21.15.37:0/1940779362 wait complete. 2024-09-21T06:29:17.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:29:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 -- 172.21.15.37:0/3168390755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc15b5f0 msgr2=0x7fc1fc06ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 --2- 172.21.15.37:0/3168390755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc15b5f0 0x7fc1fc06ff00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0005fd0 tx=0x7fc1f002a540 comp rx=0 tx=0).stop 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 -- 172.21.15.37:0/3168390755 shutdown_connections 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 --2- 172.21.15.37:0/3168390755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc0745b0 0x7fc1fc06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 --2- 172.21.15.37:0/3168390755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc15b5f0 0x7fc1fc06ff00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 -- 172.21.15.37:0/3168390755 >> 172.21.15.37:0/3168390755 conn(0x7fc1fc0d1140 msgr2=0x7fc1fc0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 -- 172.21.15.37:0/3168390755 shutdown_connections 2024-09-21T06:29:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.638+0000 7fc203e00640 1 -- 172.21.15.37:0/3168390755 wait complete. 2024-09-21T06:29:17.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.641+0000 7fc203e00640 1 Processor -- start 2024-09-21T06:29:17.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.641+0000 7fc203e00640 1 -- start start 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc15b5f0 0x7fc1fc09e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 0x7fc1fc1f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc06ee10 con 0x7fc1fc15efb0 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc06ec90 con 0x7fc1fc1514e0 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc203e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1fc06ef90 con 0x7fc1fc15b5f0 2024-09-21T06:29:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.642+0000 7fc200a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50496/0 (socket says 172.21.15.37:50496) 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 -- 172.21.15.37:0/408881394 learned_addr learned my addr 172.21.15.37:0/408881394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc201400640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc15b5f0 0x7fc1fc09e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc201e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 0x7fc1fc1f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 -- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc15b5f0 msgr2=0x7fc1fc09e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc15b5f0 0x7fc1fc09e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 -- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 msgr2=0x7fc1fc1f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 0x7fc1fc1f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 -- 172.21.15.37:0/408881394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc1f7a80 con 0x7fc1fc15efb0 2024-09-21T06:29:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc201e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 0x7fc1fc1f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.643+0000 7fc200a00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c340 tx=0x7fc1e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.644+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80052d0 con 0x7fc1fc15efb0 2024-09-21T06:29:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.644+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1e8007020 con 0x7fc1fc15efb0 2024-09-21T06:29:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.644+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8007470 con 0x7fc1fc15efb0 2024-09-21T06:29:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.644+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc09cc50 con 0x7fc1fc15efb0 2024-09-21T06:29:17.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.644+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1f6d40 con 0x7fc1fc15efb0 2024-09-21T06:29:17.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.645+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc09c9c0 con 0x7fc1fc15efb0 2024-09-21T06:29:17.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.646+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1e80067d0 con 0x7fc1fc15efb0 2024-09-21T06:29:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.647+0000 7fc1f9600640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 0x7fc1dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.647+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc1e809a2f0 con 0x7fc1fc15efb0 2024-09-21T06:29:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.648+0000 7fc201400640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 0x7fc1dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:17.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.649+0000 7fc201400640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 0x7fc1dc07b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033000 tx=0x7fc1f0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:17.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.652+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1e8061f40 con 0x7fc1fc15efb0 2024-09-21T06:29:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.891+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1fc09d170 con 0x7fc1fc15efb0 2024-09-21T06:29:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.891+0000 7fc1f9600640 1 -- 172.21.15.37:0/408881394 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5242 (secure 0 0 0) 0x7fc1fc09d170 con 0x7fc1fc15efb0 2024-09-21T06:29:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:17.894 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"btime":"2024-09-21T06:29:17:379912+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:17.379892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4984,"mds_1":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:creating","state_seq":1,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":2}]} 2024-09-21T06:29:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-09-21T06:29:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 msgr2=0x7fc1dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 0x7fc1dc07b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033000 tx=0x7fc1f0004730 comp rx=0 tx=0).stop 2024-09-21T06:29:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 msgr2=0x7fc1fc1f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c340 tx=0x7fc1e800c800 comp rx=0 tx=0).stop 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 shutdown_connections 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1dc078b90 0x7fc1dc07b0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1fc15efb0 0x7fc1fc1f7540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1fc1514e0 0x7fc1fc1f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 --2- 172.21.15.37:0/408881394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1fc15b5f0 0x7fc1fc09e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 >> 172.21.15.37:0/408881394 conn(0x7fc1fc0d1140 msgr2=0x7fc1fc15a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 shutdown_connections 2024-09-21T06:29:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:17.894+0000 7fc203e00640 1 -- 172.21.15.37:0/408881394 wait complete. 2024-09-21T06:29:18.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:29:07.816070+0000', 'modified': '2024-09-21T06:29:17.379892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 4984, 'mds_1': 4977}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4977': {'gid': 4977, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.37:6835/2364291417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 2364291417}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2364291417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4984': {'gid': 4984, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/337579449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 337579449}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 337579449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4984, 'qdb_cluster': [4984]} 2024-09-21T06:29:18.321 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4977:up:creating 2024-09-21T06:29:19.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:29:19.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:29:19.321 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:29:19.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 -- 172.21.15.37:0/696120345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 msgr2=0x7f7dd0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 --2- 172.21.15.37:0/696120345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dd0004800 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7dec072bb0 tx=0x7f7dd0005830 comp rx=0 tx=0).stop 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7debe00640 1 -- 172.21.15.37:0/696120345 reap_dead start 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 -- 172.21.15.37:0/696120345 shutdown_connections 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 -- 172.21.15.37:0/696120345 >> 172.21.15.37:0/696120345 conn(0x7f7dec06f940 msgr2=0x7f7dec0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 -- 172.21.15.37:0/696120345 shutdown_connections 2024-09-21T06:29:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.550+0000 7f7df0e00640 1 -- 172.21.15.37:0/696120345 wait complete. 2024-09-21T06:29:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.552+0000 7f7df0e00640 1 Processor -- start 2024-09-21T06:29:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.553+0000 7f7df0e00640 1 -- start start 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dec145120 0x7f7dec145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 0x7f7dec140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec06e900 con 0x7f7dd0003f90 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec06e780 con 0x7f7dec145120 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7df0e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec0b4b90 con 0x7f7dec140b40 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7deb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7deb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50522/0 (socket says 172.21.15.37:50522) 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.554+0000 7f7deb400640 1 -- 172.21.15.37:0/3831934984 learned_addr learned my addr 172.21.15.37:0/3831934984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7de3e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dec145120 0x7f7dec145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7debe00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 0x7f7dec140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 -- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 msgr2=0x7f7dec140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 0x7f7dec140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 -- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dec145120 msgr2=0x7f7dec145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dec145120 0x7f7dec145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 -- 172.21.15.37:0/3831934984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec141490 con 0x7f7dd0003f90 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7debe00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 0x7f7dec140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7deb400640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc011b10 tx=0x7f7ddc009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.555+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc01e070 con 0x7f7dd0003f90 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.556+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ddc004050 con 0x7f7dd0003f90 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.556+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec1e70b0 con 0x7f7dd0003f90 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.556+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00fea0 con 0x7f7dd0003f90 2024-09-21T06:29:19.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.556+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1e7670 con 0x7f7dd0003f90 2024-09-21T06:29:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.557+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec160ea0 con 0x7f7dd0003f90 2024-09-21T06:29:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.558+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ddc021070 con 0x7f7dd0003f90 2024-09-21T06:29:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.560+0000 7f7de3400640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 0x7f7dc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:19.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.560+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7ddc09b960 con 0x7f7dd0003f90 2024-09-21T06:29:19.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.560+0000 7f7de3e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 0x7f7dc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:19.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.561+0000 7f7de3e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 0x7f7dc807b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7dec074e70 tx=0x7f7dd4001270 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:19.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.564+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7dec141490 con 0x7f7dd0003f90 2024-09-21T06:29:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.799+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dec0cc0e0 con 0x7f7dd0003f90 2024-09-21T06:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.800+0000 7f7de3400640 1 -- 172.21.15.37:0/3831934984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5264 (secure 0 0 0) 0x7f7ddc0044a0 con 0x7f7dd0003f90 2024-09-21T06:29:19.802 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:19.802 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"btime":"2024-09-21T06:29:19:392239+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:19.392228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4974,4977,4984]},"id":2}]} 2024-09-21T06:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-09-21T06:29:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 msgr2=0x7f7dc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 0x7f7dc807b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7dec074e70 tx=0x7f7dd4001270 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 msgr2=0x7f7dec13d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc011b10 tx=0x7f7ddc009e90 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 shutdown_connections 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7dc8078b90 0x7f7dc807b0a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dec140b40 0x7f7dec140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dec145120 0x7f7dec145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 --2- 172.21.15.37:0/3831934984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd0003f90 0x7f7dec13d8a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 >> 172.21.15.37:0/3831934984 conn(0x7f7dec06f940 msgr2=0x7f7dec160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 shutdown_connections 2024-09-21T06:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:19.802+0000 7f7df0e00640 1 -- 172.21.15.37:0/3831934984 wait complete. 2024-09-21T06:29:20.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:29:07.816070+0000', 'modified': '2024-09-21T06:29:19.392228+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4984, 'mds_1': 4977, 'mds_2': 4974}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4974': {'gid': 4974, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.37:6834/2753796335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 2753796335}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 2753796335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4977': {'gid': 4977, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.37:6835/2364291417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 2364291417}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2364291417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4984': {'gid': 4984, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/337579449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 337579449}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 337579449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4984, 'qdb_cluster': [4974, 4977, 4984]} 2024-09-21T06:29:20.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:29:20.231 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:29:20.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 -- 172.21.15.37:0/2724332134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00cd240 msgr2=0x7f76d015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 --2- 172.21.15.37:0/2724332134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00cd240 0x7f76d015d360 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76d00749c0 tx=0x7f76c002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 -- 172.21.15.37:0/2724332134 shutdown_connections 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 --2- 172.21.15.37:0/2724332134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00afde0 0x7f76d00cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 --2- 172.21.15.37:0/2724332134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00cd240 0x7f76d015d360 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 -- 172.21.15.37:0/2724332134 >> 172.21.15.37:0/2724332134 conn(0x7f76d00d1190 msgr2=0x7f76d0073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 -- 172.21.15.37:0/2724332134 shutdown_connections 2024-09-21T06:29:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.461+0000 7f76d4c00640 1 -- 172.21.15.37:0/2724332134 wait complete. 2024-09-21T06:29:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.463+0000 7f76d4c00640 1 Processor -- start 2024-09-21T06:29:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.464+0000 7f76d4c00640 1 -- start start 2024-09-21T06:29:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 0x7f76d0160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00cd240 0x7f76d00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76d015de30 con 0x7f76d00714b0 2024-09-21T06:29:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76d015dcb0 con 0x7f76d00afde0 2024-09-21T06:29:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76d4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f76d015dfb0 con 0x7f76d00cd240 2024-09-21T06:29:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50538/0 (socket says 172.21.15.37:50538) 2024-09-21T06:29:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.465+0000 7f76cf400640 1 -- 172.21.15.37:0/689984821 learned_addr learned my addr 172.21.15.37:0/689984821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76c7e00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 0x7f76d0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cfe00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00cd240 0x7f76d00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 -- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00cd240 msgr2=0x7f76d00b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00cd240 0x7f76d00b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 -- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 msgr2=0x7f76d0160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 0x7f76d0160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 -- 172.21.15.37:0/689984821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d01f6900 con 0x7f76d00714b0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76c7e00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 0x7f76d0160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76cf400640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c0003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76c7400640 1 -- 172.21.15.37:0/689984821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c003e030 con 0x7f76d00714b0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76c7400640 1 -- 172.21.15.37:0/689984821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c0003b80 con 0x7f76d00714b0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d01f6be0 con 0x7f76d00714b0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.466+0000 7f76c7400640 1 -- 172.21.15.37:0/689984821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0035040 con 0x7f76d00714b0 2024-09-21T06:29:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.467+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d01f71a0 con 0x7f76d00714b0 2024-09-21T06:29:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.469+0000 7f76c7400640 1 -- 172.21.15.37:0/689984821 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76c0041030 con 0x7f76d00714b0 2024-09-21T06:29:20.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.470+0000 7f76c7400640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 0x7f76ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.470+0000 7f76c7400640 1 -- 172.21.15.37:0/689984821 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f76c00bb400 con 0x7f76d00714b0 2024-09-21T06:29:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.471+0000 7f76c7e00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 0x7f76ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.471+0000 7f76dd27e740 1 Processor -- start 2024-09-21T06:29:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.471+0000 7f76dd27e740 1 -- start start 2024-09-21T06:29:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.471+0000 7f76c7e00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 0x7f76ac07b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76d01f6730 tx=0x7f76b8005f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.472+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b067270 0x55db9b452c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.472+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b03e3b0 0x55db9b408150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.472+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.472+0000 7f76dd27e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db9b391430 con 0x55db9b046590 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3815d0 con 0x55db9b03e3b0 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db9b370a00 con 0x55db9b067270 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50548/0 (socket says 172.21.15.37:50548) 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 -- 172.21.15.37:0/232611095 learned_addr learned my addr 172.21.15.37:0/232611095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76c7e00640 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b03e3b0 0x55db9b408150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cf400640 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b067270 0x55db9b452c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 -- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b067270 msgr2=0x55db9b452c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b067270 0x55db9b452c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 -- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b03e3b0 msgr2=0x55db9b408150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b03e3b0 0x55db9b408150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.473+0000 7f76cfe00640 1 -- 172.21.15.37:0/232611095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b339520 con 0x55db9b046590 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.474+0000 7f76cfe00640 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001bd0 tx=0x7f76c8002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7ccf10da0b55da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.474+0000 7f76bf400640 1 -- 172.21.15.37:0/232611095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8019070 con 0x55db9b046590 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.474+0000 7f76bf400640 1 -- 172.21.15.37:0/232611095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c8003c70 con 0x55db9b046590 2024-09-21T06:29:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.474+0000 7f76bf400640 1 -- 172.21.15.37:0/232611095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c800ea60 con 0x55db9b046590 2024-09-21T06:29:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 -- 172.21.15.37:0/232611095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 msgr2=0x55db9b408690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001bd0 tx=0x7f76c8002090 comp rx=0 tx=0).stop 2024-09-21T06:29:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 -- 172.21.15.37:0/232611095 shutdown_connections 2024-09-21T06:29:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b046590 0x55db9b408690 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b03e3b0 0x55db9b408150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 --2- 172.21.15.37:0/232611095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b067270 0x55db9b452c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 -- 172.21.15.37:0/232611095 >> 172.21.15.37:0/232611095 conn(0x55db9b05b3c0 msgr2=0x55db9b40db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 -- 172.21.15.37:0/232611095 shutdown_connections 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.475+0000 7f76dd27e740 1 -- 172.21.15.37:0/232611095 wait complete. 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.476+0000 7f76dd27e740 1 -- start start 2024-09-21T06:29:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.476+0000 7f76dd27e740 1 Processor -- start 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b44f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db9b317330 con 0x55db9b03e3b0 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db9b374810 con 0x55db9b067270 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db9b0e1620 con 0x55db9b046590 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76dd27e740 10 client.0 _ll_register_callbacks cb 0x55db9b31c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:20.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50552/0 (socket says 172.21.15.37:50552) 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 -- 172.21.15.37:0/1873472982 learned_addr learned my addr 172.21.15.37:0/1873472982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76c7e00640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cfe00640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b44f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 -- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 msgr2=0x55db9b42f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 -- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 msgr2=0x55db9b44f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b44f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.478+0000 7f76cf400640 1 -- 172.21.15.37:0/1873472982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b0d2850 con 0x55db9b03e3b0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76cf400640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76c0004010 tx=0x7f76c0082b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87f0bbcd7bde612 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0048040 con 0x55db9b03e3b0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f76c008c7d0 con 0x55db9b03e3b0 2024-09-21T06:29:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c8003c70 con 0x55db9b03e3b0 2024-09-21T06:29:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f76c00837d0 con 0x55db9b03e3b0 2024-09-21T06:29:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76dd27e740 10 client.5169 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55db9b1496b0 con 0x55db9b03e3b0 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.479+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c8003c70 con 0x55db9b03e3b0 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f76c009d140 con 0x55db9b03e3b0 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 10 client.5169 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 10 client.5169 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 20 client.5169 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b025e40 0x55db9b4697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 4 client.5169 mds_command: new command op to 4974 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 --> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9b383520 con 0x55db9b025e40 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.480+0000 7f76c7e00640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b025e40 0x55db9b4697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.481+0000 7f76c7e00640 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b025e40 0x55db9b4697a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3dcd78ac025d8fd server_cookie=1e3779301f0d3a2b in_seq=0 out_seq=0 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.481+0000 7f76b3e00640 10 client.5169 ms_handle_connect on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76b3e00640 1 -- 172.21.15.37:0/1873472982 <== mds.2 v2:172.21.15.37:6832/2753796335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db9b383520 con 0x55db9b025e40 2024-09-21T06:29:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76b3e00640 10 client.5169 handle_command_reply: tid=0 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 client.5169 shutdown 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 msgr2=0x55db9b466ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76c0004010 tx=0x7f76c0082b50 comp rx=0 tx=0).stop 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 shutdown_connections 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b025e40 0x55db9b4697a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b44f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 --2- 172.21.15.37:0/1873472982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b466ac0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76b3e00640 0 client.5169 ms_handle_reset on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 >> 172.21.15.37:0/1873472982 conn(0x55db9b05b3c0 msgr2=0x55db9b434770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 shutdown_connections 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 1 -- 172.21.15.37:0/1873472982 wait complete. 2024-09-21T06:29:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.482+0000 7f76dd27e740 20 client.5169 trim_cache size 0 max 16384 2024-09-21T06:29:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.502+0000 7f76dd27e740 1 Processor -- start 2024-09-21T06:29:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.502+0000 7f76dd27e740 1 -- start start 2024-09-21T06:29:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b408030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01fe1c0 con 0x55db9b03e3b0 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0070fb0 con 0x55db9b067270 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f76d01fdaf0 con 0x55db9b046590 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50564/0 (socket says 172.21.15.37:50564) 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.503+0000 7f76cf400640 1 -- 172.21.15.37:0/3552298204 learned_addr learned my addr 172.21.15.37:0/3552298204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76c7e00640 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cfe00640 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b408030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 -- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 msgr2=0x55db9b42f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 -- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 msgr2=0x55db9b408030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b408030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 -- 172.21.15.37:0/3552298204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b42f0a0 con 0x55db9b03e3b0 2024-09-21T06:29:20.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76cf400640 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f76c0003660 tx=0x7f76c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5bf00fbe6f0726 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76bea00640 1 -- 172.21.15.37:0/3552298204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0048020 con 0x55db9b03e3b0 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76bea00640 1 -- 172.21.15.37:0/3552298204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c008c7f0 con 0x55db9b03e3b0 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.504+0000 7f76bea00640 1 -- 172.21.15.37:0/3552298204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0087c00 con 0x55db9b03e3b0 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 -- 172.21.15.37:0/3552298204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 msgr2=0x55db9b42f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f76c0003660 tx=0x7f76c0009260 comp rx=0 tx=0).stop 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 -- 172.21.15.37:0/3552298204 shutdown_connections 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b067270 0x55db9b408030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b046590 0x55db9b42f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 --2- 172.21.15.37:0/3552298204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b03e3b0 0x55db9b42f290 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 -- 172.21.15.37:0/3552298204 >> 172.21.15.37:0/3552298204 conn(0x55db9b05b3c0 msgr2=0x55db9b47a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 -- 172.21.15.37:0/3552298204 shutdown_connections 2024-09-21T06:29:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.505+0000 7f76dd27e740 1 -- 172.21.15.37:0/3552298204 wait complete. 2024-09-21T06:29:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.506+0000 7f76dd27e740 1 -- start start 2024-09-21T06:29:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.506+0000 7f76dd27e740 1 Processor -- start 2024-09-21T06:29:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 0x55db9b40ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 0x55db9b4d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db9b44dfc0 con 0x55db9b067270 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0158110 con 0x55db9b046590 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db9b44d400 con 0x55db9b03e3b0 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76dd27e740 10 client.0 _ll_register_callbacks cb 0x7f76c0082f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50570/0 (socket says 172.21.15.37:50570) 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.508+0000 7f76cfe00640 1 -- 172.21.15.37:0/929057132 learned_addr learned my addr 172.21.15.37:0/929057132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cf400640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 0x55db9b40ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76c7e00640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 0x55db9b4d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 -- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 msgr2=0x55db9b40ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 0x55db9b40ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 -- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 msgr2=0x55db9b4d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 0x55db9b4d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 -- 172.21.15.37:0/929057132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b478ae0 con 0x55db9b067270 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cf400640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 0x55db9b40ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76c7e00640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 0x55db9b4d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76cfe00640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76c8013000 tx=0x7f76c8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218f5a280e932d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8024020 con 0x55db9b067270 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f76c800ea00 con 0x55db9b067270 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c008c7f0 con 0x55db9b067270 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f76c802e050 con 0x55db9b067270 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76dd27e740 10 client.5175 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.509+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55db9b40c720 con 0x55db9b067270 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c008c7f0 con 0x55db9b067270 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f76c8031020 con 0x55db9b067270 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 10 client.5175 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 10 client.5175 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 20 client.5175 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b42f990 0x55db9b4d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 4 client.5175 mds_command: new command op to 4974 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 --> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] -- command(tid 0: {"prefix": "status"}) -- 0x55db9b37efd0 con 0x55db9b42f990 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.510+0000 7f76cf400640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b42f990 0x55db9b4d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76b3e00640 10 client.5175 ms_handle_connect on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76cf400640 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b42f990 0x55db9b4d9e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4c0c6dae57e52e7 server_cookie=fb4e9d835f57a66d in_seq=0 out_seq=0 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76b3e00640 1 -- 172.21.15.37:0/929057132 <== mds.2 v2:172.21.15.37:6832/2753796335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55db9b37efd0 con 0x55db9b42f990 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76b3e00640 10 client.5175 handle_command_reply: tid=0 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 client.5175 shutdown 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 msgr2=0x55db9b438180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76c8013000 tx=0x7f76c8007f50 comp rx=0 tx=0).stop 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 shutdown_connections 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x55db9b42f990 0x55db9b4d9e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b067270 0x55db9b438180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db9b046590 0x55db9b4d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76b3e00640 0 client.5175 ms_handle_reset on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 --2- 172.21.15.37:0/929057132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db9b03e3b0 0x55db9b40ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 >> 172.21.15.37:0/929057132 conn(0x55db9b05b3c0 msgr2=0x55db9b47a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 shutdown_connections 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.511+0000 7f76dd27e740 1 -- 172.21.15.37:0/929057132 wait complete. 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76dd27e740 20 client.5175 trim_cache size 0 max 16384 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4974, 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:29:20.517 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1203646690000002, 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 19.952555347000001 2024-09-21T06:29:20.518 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 msgr2=0x7f76ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 0x7f76ac07b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76d01f6730 tx=0x7f76b8005f10 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 msgr2=0x7f76d0145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c0003690 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 shutdown_connections 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76ac078b90 0x7f76ac07b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76d00cd240 0x7f76d00b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d00afde0 0x7f76d0160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 --2- 172.21.15.37:0/689984821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d00714b0 0x7f76d0145150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 >> 172.21.15.37:0/689984821 conn(0x7f76d00d1190 msgr2=0x7f76d015afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 shutdown_connections 2024-09-21T06:29:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.512+0000 7f76d4c00640 1 -- 172.21.15.37:0/689984821 wait complete. 2024-09-21T06:29:20.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:29:20.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.762+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1351202672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b80d1450 msgr2=0x7fb5b8070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.762+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1351202672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b80d1450 0x7fb5b8070100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5b815a460 tx=0x7fb5a802a540 comp rx=0 tx=0).stop 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1351202672 shutdown_connections 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1351202672 >> v2:172.21.15.150:3300/0 conn(0x7fb5b80ccde0 0x7fb5b8070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1351202672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b80d1450 0x7fb5b8070100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1351202672 >> 172.21.15.37:0/1351202672 conn(0x7fb5b80b4010 msgr2=0x7fb5b80b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1351202672 shutdown_connections 2024-09-21T06:29:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.763+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1351202672 wait complete. 2024-09-21T06:29:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.765+0000 7fb5bcc00640 1 Processor -- start 2024-09-21T06:29:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.765+0000 7fb5bcc00640 1 -- start start 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5b80d1450 0x7fb5b81509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b80d1860 0x7fb5b81f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b815d540 con 0x7fb5b81515e0 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b815d3c0 con 0x7fb5b80d1860 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5bcc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b815d6c0 con 0x7fb5b80d1450 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50580/0 (socket says 172.21.15.37:50580) 2024-09-21T06:29:20.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7e00640 1 -- 172.21.15.37:0/1900398065 learned_addr learned my addr 172.21.15.37:0/1900398065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b80d1860 0x7fb5b81f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7400640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5b80d1450 0x7fb5b81509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7e00640 1 -- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5b80d1450 msgr2=0x7fb5b81509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.767+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5b80d1450 0x7fb5b81509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5b7e00640 1 -- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b80d1860 msgr2=0x7fb5b81f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b80d1860 0x7fb5b81f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5b7e00640 1 -- 172.21.15.37:0/1900398065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b80b60a0 con 0x7fb5b81515e0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000ea80 tx=0x7fb5b000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5af400640 1 -- 172.21.15.37:0/1900398065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0004b20 con 0x7fb5b81515e0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5af400640 1 -- 172.21.15.37:0/1900398065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b0004ce0 con 0x7fb5b81515e0 2024-09-21T06:29:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b80b6380 con 0x7fb5b81515e0 2024-09-21T06:29:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5af400640 1 -- 172.21.15.37:0/1900398065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0005280 con 0x7fb5b81515e0 2024-09-21T06:29:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.768+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b81546a0 con 0x7fb5b81515e0 2024-09-21T06:29:20.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.772+0000 7fb5af400640 1 -- 172.21.15.37:0/1900398065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5b0005420 con 0x7fb5b81515e0 2024-09-21T06:29:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.773+0000 7fb5af400640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 0x7fb59807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.773+0000 7fb5af400640 1 -- 172.21.15.37:0/1900398065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb5b009bca0 con 0x7fb5b81515e0 2024-09-21T06:29:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.773+0000 7fb5c5213740 1 Processor -- start 2024-09-21T06:29:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.773+0000 7fb5c5213740 1 -- start start 2024-09-21T06:29:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.773+0000 7fb5b7400640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 0x7fb59807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.774+0000 7fb5b7400640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 0x7fb59807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829e8270 0x55d382db8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382db90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d382d12430 con 0x55d3829c7590 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d382d025d0 con 0x55d3829e8270 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d382cf1a00 con 0x55d3829bf3b0 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50586/0 (socket says 172.21.15.37:50586) 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2864893609 learned_addr learned my addr 172.21.15.37:0/2864893609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382db90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829e8270 0x55d382db8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 msgr2=0x55d382db90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382db90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829e8270 msgr2=0x55d382db8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.775+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829e8270 0x55d382db8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.776+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2864893609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d382cba520 con 0x55d3829c7590 2024-09-21T06:29:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.776+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0099ee0 tx=0x7fb5b0063640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fddf24529594ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.776+0000 7fb5a3400640 1 -- 172.21.15.37:0/2864893609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00632c0 con 0x55d3829c7590 2024-09-21T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.776+0000 7fb5a3400640 1 -- 172.21.15.37:0/2864893609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b0063eb0 con 0x55d3829c7590 2024-09-21T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.776+0000 7fb5a3400640 1 -- 172.21.15.37:0/2864893609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b006cfa0 con 0x55d3829c7590 2024-09-21T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 -- 172.21.15.37:0/2864893609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 msgr2=0x55d382d89150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0099ee0 tx=0x7fb5b0063640 comp rx=0 tx=0).stop 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 -- 172.21.15.37:0/2864893609 shutdown_connections 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829c7590 0x55d382d89150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382db90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2864893609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829e8270 0x55d382db8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 -- 172.21.15.37:0/2864893609 >> 172.21.15.37:0/2864893609 conn(0x55d3829dc3c0 msgr2=0x55d382d8eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 -- 172.21.15.37:0/2864893609 shutdown_connections 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.777+0000 7fb5c5213740 1 -- 172.21.15.37:0/2864893609 wait complete. 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.778+0000 7fb5c5213740 1 -- start start 2024-09-21T06:29:20.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.778+0000 7fb5c5213740 1 Processor -- start 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382dce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382dcd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d382c98330 con 0x55d3829e8270 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d382cf5810 con 0x55d3829c7590 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d382a62620 con 0x55d3829bf3b0 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5c5213740 10 client.0 _ll_register_callbacks cb 0x55d382c9d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50588/0 (socket says 172.21.15.37:50588) 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2036689788 learned_addr learned my addr 172.21.15.37:0/2036689788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382dcd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382dce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 msgr2=0x55d382dce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382dce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 msgr2=0x55d382dcd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382dcd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.780+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2036689788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d382a53850 con 0x55d3829e8270 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0063740 tx=0x7fb5b006cff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291b695968d5edd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb597e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0082060 con 0x55d3829e8270 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb5b0062a10 con 0x55d3829e8270 2024-09-21T06:29:20.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b0063eb0 con 0x55d3829e8270 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5b00632c0 con 0x55d3829e8270 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb5c5213740 10 client.5190 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55d382aca6b0 con 0x55d3829e8270 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.781+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b0063eb0 con 0x55d3829e8270 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fb5b0064ae0 con 0x55d3829e8270 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 10 client.5190 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 10 client.5190 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 20 client.5190 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d3829a6e40 0x55d382db50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 4 client.5190 mds_command: new command op to 4977 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 --> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d382d04520 con 0x55d3829a6e40 2024-09-21T06:29:20.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.782+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d3829a6e40 0x55d382db50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.783+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d3829a6e40 0x55d382db50a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0477697e65b867b server_cookie=8c74d96f650d4459 in_seq=0 out_seq=0 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.783+0000 7fb597e00640 10 client.5190 ms_handle_connect on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.783+0000 7fb597e00640 1 -- 172.21.15.37:0/2036689788 <== mds.1 v2:172.21.15.37:6833/2364291417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d382d04520 con 0x55d3829a6e40 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.783+0000 7fb597e00640 10 client.5190 handle_command_reply: tid=0 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.783+0000 7fb5c5213740 1 client.5190 shutdown 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 msgr2=0x55d382de6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0063740 tx=0x7fb5b006cff0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 shutdown_connections 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d3829a6e40 0x55d382db50a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de6240 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382dcd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2036689788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382dce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb597e00640 0 client.5190 ms_handle_reset on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:20.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 >> 172.21.15.37:0/2036689788 conn(0x55d3829dc3c0 msgr2=0x55d382db8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 shutdown_connections 2024-09-21T06:29:20.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 1 -- 172.21.15.37:0/2036689788 wait complete. 2024-09-21T06:29:20.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.784+0000 7fb5c5213740 20 client.5190 trim_cache size 0 max 16384 2024-09-21T06:29:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.804+0000 7fb5c5213740 1 Processor -- start 2024-09-21T06:29:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.804+0000 7fb5c5213740 1 -- start start 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 0x55d382db6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 0x55d382d8e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8201630 con 0x55d3829e8270 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8201d00 con 0x55d3829bf3b0 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.805+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b80b58d0 con 0x55d3829c7590 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50590/0 (socket says 172.21.15.37:50590) 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2479270760 learned_addr learned my addr 172.21.15.37:0/2479270760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 0x55d382d8e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 msgr2=0x55d382d8e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 0x55d382d8e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 0x55d382db6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 msgr2=0x55d382db6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 0x55d382db6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 -- 172.21.15.37:0/2479270760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d382d8e350 con 0x55d3829e8270 2024-09-21T06:29:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 0x55d382d8e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7400640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 0x55d382db6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00634b0 tx=0x7fb5b009a020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c40924ab5870950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.806+0000 7fb5a2a00640 1 -- 172.21.15.37:0/2479270760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00682e0 con 0x55d3829e8270 2024-09-21T06:29:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5a2a00640 1 -- 172.21.15.37:0/2479270760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b0063eb0 con 0x55d3829e8270 2024-09-21T06:29:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5a2a00640 1 -- 172.21.15.37:0/2479270760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b005f650 con 0x55d3829e8270 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 -- 172.21.15.37:0/2479270760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 msgr2=0x55d382d8ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00634b0 tx=0x7fb5b009a020 comp rx=0 tx=0).stop 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 -- 172.21.15.37:0/2479270760 shutdown_connections 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382d8ef20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829c7590 0x55d382d8e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 --2- 172.21.15.37:0/2479270760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829bf3b0 0x55d382db6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 -- 172.21.15.37:0/2479270760 >> 172.21.15.37:0/2479270760 conn(0x55d3829dc3c0 msgr2=0x55d382db8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 -- 172.21.15.37:0/2479270760 shutdown_connections 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.807+0000 7fb5c5213740 1 -- 172.21.15.37:0/2479270760 wait complete. 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.808+0000 7fb5c5213740 1 -- start start 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.808+0000 7fb5c5213740 1 Processor -- start 2024-09-21T06:29:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382e525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382df6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d382c98330 con 0x55d3829e8270 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d382cf5810 con 0x55d3829c7590 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d382db3d50 con 0x55d3829bf3b0 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5c5213740 10 client.0 _ll_register_callbacks cb 0x7fb5b0099f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50604/0 (socket says 172.21.15.37:50604) 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.810+0000 7fb5b7e00640 1 -- 172.21.15.37:0/4047776820 learned_addr learned my addr 172.21.15.37:0/4047776820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b6a00640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382df6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7400640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382e525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 -- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 msgr2=0x55d382e525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382e525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 -- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 msgr2=0x55d382df6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382df6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 -- 172.21.15.37:0/4047776820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d382db5830 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5b7e00640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0004f50 tx=0x7fb5b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb3948fef13e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb597e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b005f650 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb5b005f100 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b0063eb0 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5b00682e0 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5c5213740 10 client.5196 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.811+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55d382db5830 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b0063eb0 con 0x55d3829e8270 2024-09-21T06:29:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fb5b0062480 con 0x55d3829e8270 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 10 client.5196 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 10 client.5196 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 20 client.5196 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d382d8e9e0 0x55d382d8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 4 client.5196 mds_command: new command op to 4977 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 --> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] -- command(tid 0: {"prefix": "status"}) -- 0x55d382cfffd0 con 0x55d382d8e9e0 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.812+0000 7fb5b7400640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d382d8e9e0 0x55d382d8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb597e00640 10 client.5196 ms_handle_connect on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5b7400640 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d382d8e9e0 0x55d382d8edf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3f7750ca671077f server_cookie=509f4301cd54bde0 in_seq=0 out_seq=0 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb597e00640 1 -- 172.21.15.37:0/4047776820 <== mds.1 v2:172.21.15.37:6833/2364291417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d382cfffd0 con 0x55d382d8e9e0 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb597e00640 10 client.5196 handle_command_reply: tid=0 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 client.5196 shutdown 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 msgr2=0x55d382de53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0004f50 tx=0x7fb5b000c8b0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 shutdown_connections 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55d382d8e9e0 0x55d382d8edf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3829e8270 0x55d382de53d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3829c7590 0x55d382df6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 --2- 172.21.15.37:0/4047776820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3829bf3b0 0x55d382e525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 >> 172.21.15.37:0/4047776820 conn(0x55d3829dc3c0 msgr2=0x55d382de2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb597e00640 0 client.5196 ms_handle_reset on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 shutdown_connections 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 1 -- 172.21.15.37:0/4047776820 wait complete. 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.813+0000 7fb5c5213740 20 client.5196 trim_cache size 0 max 16384 2024-09-21T06:29:20.821 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4977, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.4304772360000002, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T06:29:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 20.253986481999998 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 msgr2=0x7fb59807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 0x7fb59807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a8004730 comp rx=0 tx=0).stop 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 msgr2=0x7fb5b8154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000ea80 tx=0x7fb5b000ef40 comp rx=0 tx=0).stop 2024-09-21T06:29:20.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 shutdown_connections 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb598078b90 0x7fb59807b0a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b81515e0 0x7fb5b8154ea0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b80d1860 0x7fb5b81f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 --2- 172.21.15.37:0/1900398065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5b80d1450 0x7fb5b81509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 >> 172.21.15.37:0/1900398065 conn(0x7fb5b80b4010 msgr2=0x7fb5b8151f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 shutdown_connections 2024-09-21T06:29:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:20.814+0000 7fb5bcc00640 1 -- 172.21.15.37:0/1900398065 wait complete. 2024-09-21T06:29:20.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 -- 172.21.15.37:0/1594718826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c06e500 msgr2=0x7fdb1c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 --2- 172.21.15.37:0/1594718826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c06e500 0x7fdb1c0ce510 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c075240 tx=0x7fdb1402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 -- 172.21.15.37:0/1594718826 shutdown_connections 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 --2- 172.21.15.37:0/1594718826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c06e500 0x7fdb1c0ce510 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 --2- 172.21.15.37:0/1594718826 >> v2:172.21.15.150:3300/0 conn(0x7fdb1c0cc530 0x7fdb1c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.062+0000 7fdb23000640 1 -- 172.21.15.37:0/1594718826 >> 172.21.15.37:0/1594718826 conn(0x7fdb1c0b1b90 msgr2=0x7fdb1c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.063+0000 7fdb23000640 1 -- 172.21.15.37:0/1594718826 shutdown_connections 2024-09-21T06:29:21.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.063+0000 7fdb23000640 1 -- 172.21.15.37:0/1594718826 wait complete. 2024-09-21T06:29:21.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.065+0000 7fdb23000640 1 Processor -- start 2024-09-21T06:29:21.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.065+0000 7fdb23000640 1 -- start start 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1c0af5b0 0x7fdb1c0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb1c15faa0 0x7fdb1c0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0cec10 con 0x7fdb1c0cc530 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0cea90 con 0x7fdb1c15faa0 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.066+0000 7fdb23000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0ced90 con 0x7fdb1c0af5b0 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50626/0 (socket says 172.21.15.37:50626) 2024-09-21T06:29:21.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 -- 172.21.15.37:0/2283632319 learned_addr learned my addr 172.21.15.37:0/2283632319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1be00640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1c0af5b0 0x7fdb1c0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb21000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb1c15faa0 0x7fdb1c0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 -- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1c0af5b0 msgr2=0x7fdb1c0b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1c0af5b0 0x7fdb1c0b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 -- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb1c15faa0 msgr2=0x7fdb1c0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb1c15faa0 0x7fdb1c0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.067+0000 7fdb1b400640 1 -- 172.21.15.37:0/2283632319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c15ece0 con 0x7fdb1c0cc530 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb1b400640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c3c0 tx=0x7fdb0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb13e00640 1 -- 172.21.15.37:0/2283632319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08017070 con 0x7fdb1c0cc530 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb13e00640 1 -- 172.21.15.37:0/2283632319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb080035f0 con 0x7fdb1c0cc530 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb13e00640 1 -- 172.21.15.37:0/2283632319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08003a40 con 0x7fdb1c0cc530 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c15efc0 con 0x7fdb1c0cc530 2024-09-21T06:29:21.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.068+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c15f580 con 0x7fdb1c0cc530 2024-09-21T06:29:21.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.070+0000 7fdb13e00640 1 -- 172.21.15.37:0/2283632319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb0801a070 con 0x7fdb1c0cc530 2024-09-21T06:29:21.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.071+0000 7fdb13e00640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 0x7fdafc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.071+0000 7fdb13e00640 1 -- 172.21.15.37:0/2283632319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0809a190 con 0x7fdb1c0cc530 2024-09-21T06:29:21.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.072+0000 7fdb2a012740 1 Processor -- start 2024-09-21T06:29:21.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.072+0000 7fdb2a012740 1 -- start start 2024-09-21T06:29:21.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.072+0000 7fdb1be00640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 0x7fdafc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb1be00640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 0x7fdafc07b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009b30 tx=0x7fdb0c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fadb270 0x56290feab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56290fe05430 con 0x56290faba590 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56290fdf55d0 con 0x56290fab23b0 2024-09-21T06:29:21.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56290fde4a00 con 0x56290fadb270 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.073+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50634/0 (socket says 172.21.15.37:50634) 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 -- 172.21.15.37:0/742751466 learned_addr learned my addr 172.21.15.37:0/742751466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb1b400640 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb1be00640 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fadb270 0x56290feab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 -- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fadb270 msgr2=0x56290feab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fadb270 0x56290feab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 -- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 msgr2=0x56290feabeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 -- 172.21.15.37:0/742751466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290fdad520 con 0x56290faba590 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.074+0000 7fdb21000640 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402aea0 tx=0x7fdb14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962d7a682ecb414c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb10c00640 1 -- 172.21.15.37:0/742751466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14004040 con 0x56290faba590 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb10c00640 1 -- 172.21.15.37:0/742751466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb140042d0 con 0x56290faba590 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb10c00640 1 -- 172.21.15.37:0/742751466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14004700 con 0x56290faba590 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 -- 172.21.15.37:0/742751466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 msgr2=0x56290fe7bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402aea0 tx=0x7fdb14004a40 comp rx=0 tx=0).stop 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 -- 172.21.15.37:0/742751466 shutdown_connections 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290faba590 0x56290fe7bf60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 --2- 172.21.15.37:0/742751466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fadb270 0x56290feab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.075+0000 7fdb2a012740 1 -- 172.21.15.37:0/742751466 >> 172.21.15.37:0/742751466 conn(0x56290facf3c0 msgr2=0x56290fe81970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.076+0000 7fdb2a012740 1 -- 172.21.15.37:0/742751466 shutdown_connections 2024-09-21T06:29:21.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.076+0000 7fdb2a012740 1 -- 172.21.15.37:0/742751466 wait complete. 2024-09-21T06:29:21.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.077+0000 7fdb2a012740 1 -- start start 2024-09-21T06:29:21.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.077+0000 7fdb2a012740 1 Processor -- start 2024-09-21T06:29:21.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290fea6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290fec0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56290fd8b330 con 0x56290fadb270 2024-09-21T06:29:21.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56290fde8810 con 0x56290fab23b0 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56290fb55620 con 0x56290faba590 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.078+0000 7fdb2a012740 10 client.0 _ll_register_callbacks cb 0x56290fd90420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50642/0 (socket says 172.21.15.37:50642) 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 -- 172.21.15.37:0/4198103077 learned_addr learned my addr 172.21.15.37:0/4198103077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb1be00640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290fea6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb1b400640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290fec0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 -- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 msgr2=0x56290fec0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290fec0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 -- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 msgr2=0x56290fea6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290fea6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 -- 172.21.15.37:0/4198103077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290fb46850 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb21000640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009b30 tx=0x7fdb14009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afc8a7ae7df52aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.079+0000 7fdb05600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1404e020 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdb14045440 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb140042d0 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdb14058050 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 10 client.5211 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x56290fbbd6b0 con 0x56290fadb270 2024-09-21T06:29:21.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb140042d0 con 0x56290fadb270 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fdb1405b020 con 0x56290fadb270 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 10 client.5211 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 10 client.5211 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 20 client.5211 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fa99e40 0x56290fed5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 4 client.5211 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.080+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56290fdf7520 con 0x56290fa99e40 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.081+0000 7fdb1be00640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fa99e40 0x56290fed5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.082+0000 7fdb05600640 10 client.5211 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:21.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.081+0000 7fdb1be00640 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fa99e40 0x56290fed5da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b8cde6ccc9c84b server_cookie=b2855084d2c019c1 in_seq=0 out_seq=0 2024-09-21T06:29:21.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb05600640 1 -- 172.21.15.37:0/4198103077 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56290fdf7520 con 0x56290fa99e40 2024-09-21T06:29:21.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb05600640 10 client.5211 handle_command_reply: tid=0 2024-09-21T06:29:21.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 client.5211 shutdown 2024-09-21T06:29:21.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 msgr2=0x56290fea20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009b30 tx=0x7fdb14009280 comp rx=0 tx=0).stop 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 shutdown_connections 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fa99e40 0x56290fed5da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fea20d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290fec0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 --2- 172.21.15.37:0/4198103077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290fea6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb05600640 0 client.5211 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 >> 172.21.15.37:0/4198103077 conn(0x56290facf3c0 msgr2=0x56290fe7d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 shutdown_connections 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 1 -- 172.21.15.37:0/4198103077 wait complete. 2024-09-21T06:29:21.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.083+0000 7fdb2a012740 20 client.5211 trim_cache size 0 max 16384 2024-09-21T06:29:21.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.102+0000 7fdb2a012740 1 Processor -- start 2024-09-21T06:29:21.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.103+0000 7fdb2a012740 1 -- start start 2024-09-21T06:29:21.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fab23b0 0x56290fec6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 0x56290fee9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c201750 con 0x56290fadb270 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c201da0 con 0x56290faba590 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c1f5f10 con 0x56290fab23b0 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50652/0 (socket says 172.21.15.37:50652) 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb21000640 1 -- 172.21.15.37:0/1702711528 learned_addr learned my addr 172.21.15.37:0/1702711528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb1be00640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fab23b0 0x56290fec6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.104+0000 7fdb1b400640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 0x56290fee9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 -- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fab23b0 msgr2=0x56290fec6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fab23b0 0x56290fec6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 -- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 msgr2=0x56290fee9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 0x56290fee9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 -- 172.21.15.37:0/1702711528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290fec0ee0 con 0x56290fadb270 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb1b400640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 0x56290fee9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb21000640 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005c40 tx=0x7fdb14009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a19b84bd4c5c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb07e00640 1 -- 172.21.15.37:0/1702711528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1404e030 con 0x56290fadb270 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb07e00640 1 -- 172.21.15.37:0/1702711528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb14045550 con 0x56290fadb270 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.105+0000 7fdb07e00640 1 -- 172.21.15.37:0/1702711528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14045c30 con 0x56290fadb270 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 -- 172.21.15.37:0/1702711528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 msgr2=0x56290fee9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005c40 tx=0x7fdb14009a80 comp rx=0 tx=0).stop 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 -- 172.21.15.37:0/1702711528 shutdown_connections 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fee9fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290faba590 0x56290fee9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 --2- 172.21.15.37:0/1702711528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290fab23b0 0x56290fec6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 -- 172.21.15.37:0/1702711528 >> 172.21.15.37:0/1702711528 conn(0x56290facf3c0 msgr2=0x56290fec2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 -- 172.21.15.37:0/1702711528 shutdown_connections 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.106+0000 7fdb2a012740 1 -- 172.21.15.37:0/1702711528 wait complete. 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.107+0000 7fdb2a012740 1 -- start start 2024-09-21T06:29:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.107+0000 7fdb2a012740 1 Processor -- start 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290ff3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56290fea5600 con 0x56290fadb270 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56290fd8b330 con 0x56290fab23b0 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56290fea73a0 con 0x56290faba590 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb2a012740 10 client.0 _ll_register_callbacks cb 0x7fdb140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50656/0 (socket says 172.21.15.37:50656) 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 -- 172.21.15.37:0/972426496 learned_addr learned my addr 172.21.15.37:0/972426496 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb1be00640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb1b400640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290ff3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 -- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 msgr2=0x56290ff3d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290ff3d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 -- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 msgr2=0x56290feabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.109+0000 7fdb21000640 1 -- 172.21.15.37:0/972426496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290fea1d50 con 0x56290fadb270 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb21000640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb14002230 tx=0x7fdb14045870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c6bf928d7baab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb05600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14031640 con 0x56290fadb270 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdb14055880 con 0x56290fadb270 2024-09-21T06:29:21.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb14045550 con 0x56290fadb270 2024-09-21T06:29:21.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdb140042d0 con 0x56290fadb270 2024-09-21T06:29:21.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb2a012740 10 client.5217 fetch_fsmap learned FSMap version 22 2024-09-21T06:29:21.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.110+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x56290fe82650 con 0x56290fadb270 2024-09-21T06:29:21.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb14045550 con 0x56290fadb270 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fdb14065030 con 0x56290fadb270 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 10 client.5217 fetch_fsmap finished waiting for FSMap version 22 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 10 client.5217 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 20 client.5217 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fecf160 0x56290fecf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 4 client.5217 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "status"}) -- 0x56290fdf2fd0 con 0x56290fecf160 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.111+0000 7fdb1be00640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fecf160 0x56290fecf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb1be00640 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fecf160 0x56290fecf570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29580ff203c8b15 server_cookie=6199fe5103a8a2c6 in_seq=0 out_seq=0 2024-09-21T06:29:21.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb05600640 10 client.5217 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb05600640 1 -- 172.21.15.37:0/972426496 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56290fdf2fd0 con 0x56290fecf160 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb05600640 10 client.5217 handle_command_reply: tid=0 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 client.5217 shutdown 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 msgr2=0x56290fec6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb14002230 tx=0x7fdb14045870 comp rx=0 tx=0).stop 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 shutdown_connections 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56290fecf160 0x56290fecf570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56290fadb270 0x56290fec6c80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56290faba590 0x56290ff3d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 --2- 172.21.15.37:0/972426496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56290fab23b0 0x56290feabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb05600640 0 client.5217 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 >> 172.21.15.37:0/972426496 conn(0x56290facf3c0 msgr2=0x56290fec29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 shutdown_connections 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.112+0000 7fdb2a012740 1 -- 172.21.15.37:0/972426496 wait complete. 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb2a012740 20 client.5217 trim_cache size 0 max 16384 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4984, 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 12.815002773, 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 20.551983915000001 2024-09-21T06:29:21.120 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 msgr2=0x7fdafc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 0x7fdafc07b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009b30 tx=0x7fdb0c005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 msgr2=0x7fdb1c0b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c3c0 tx=0x7fdb0800c880 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 shutdown_connections 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdafc078b90 0x7fdafc07b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb1c0cc530 0x7fdb1c0b4270 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb1c15faa0 0x7fdb1c0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 --2- 172.21.15.37:0/2283632319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1c0af5b0 0x7fdb1c0b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 >> 172.21.15.37:0/2283632319 conn(0x7fdb1c0b1b90 msgr2=0x7fdb1c15d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.113+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 shutdown_connections 2024-09-21T06:29:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.114+0000 7fdb23000640 1 -- 172.21.15.37:0/2283632319 wait complete. 2024-09-21T06:29:21.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:29:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.365+0000 7f6a00a00640 1 -- 172.21.15.37:0/3210942298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc06e160 msgr2=0x7f69fc06e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.365+0000 7f6a00a00640 1 --2- 172.21.15.37:0/3210942298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc06e160 0x7f69fc06e570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69e4005fd0 tx=0x7f69e402a540 comp rx=0 tx=0).stop 2024-09-21T06:29:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 -- 172.21.15.37:0/3210942298 shutdown_connections 2024-09-21T06:29:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 --2- 172.21.15.37:0/3210942298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc06e160 0x7f69fc06e570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 --2- 172.21.15.37:0/3210942298 >> v2:172.21.15.150:3300/0 conn(0x7f69fc0cc8d0 0x7f69fc0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 -- 172.21.15.37:0/3210942298 >> 172.21.15.37:0/3210942298 conn(0x7f69fc06f940 msgr2=0x7f69fc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 -- 172.21.15.37:0/3210942298 shutdown_connections 2024-09-21T06:29:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.366+0000 7f6a00a00640 1 -- 172.21.15.37:0/3210942298 wait complete. 2024-09-21T06:29:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.368+0000 7f6a00a00640 1 Processor -- start 2024-09-21T06:29:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.368+0000 7f6a00a00640 1 -- start start 2024-09-21T06:29:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.369+0000 7f6a00a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 0x7f69fc1f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.369+0000 7f6a00a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f6a00a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc15dfe0 0x7f69fc1f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f6a00a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0b4c50 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f6a00a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0b4ad0 con 0x7f69fc06e160 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f6a00a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0b4dd0 con 0x7f69fc15dfe0 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50672/0 (socket says 172.21.15.37:50672) 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 -- 172.21.15.37:0/731675898 learned_addr learned my addr 172.21.15.37:0/731675898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69fb200640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc15dfe0 0x7f69fc1f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69fa800640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 0x7f69fc1f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 -- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc15dfe0 msgr2=0x7f69fc1f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc15dfe0 0x7f69fc1f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 -- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 msgr2=0x7f69fc1f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 0x7f69fc1f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69f3e00640 1 -- 172.21.15.37:0/731675898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0ad9f0 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.370+0000 7f69fa800640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 0x7f69fc1f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f69f3e00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69e4000c00 tx=0x7f69e40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e403e030 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69e4003ca0 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0adcd0 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4035040 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.371+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1f6f30 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.372+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc0b52f0 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.373+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69e4041030 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.375+0000 7f69f2a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 0x7f69d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.375+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f69e40bb600 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.376+0000 7f69fa800640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 0x7f69d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.376+0000 7f69fa800640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 0x7f69d807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0016c0 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:21.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.380+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69e4083140 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.622+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69fc0cc0e0 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.622+0000 7f69f2a00640 1 -- 172.21.15.37:0/731675898 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5264 (secure 0 0 0) 0x7f69e4087b70 con 0x7f69fc0cc8d0 2024-09-21T06:29:21.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:29:21.625 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"btime":"2024-09-21T06:29:19:392239+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:19.392228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4974,4977,4984]},"id":2}]} 2024-09-21T06:29:21.625 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-09-21T06:29:21.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 msgr2=0x7f69d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 0x7f69d807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0016c0 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 msgr2=0x7f69fc1f8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69e4000c00 tx=0x7f69e40038a0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 shutdown_connections 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69d8078b90 0x7f69d807b0a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69fc0cc8d0 0x7f69fc1f8f40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69fc15dfe0 0x7f69fc1f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 --2- 172.21.15.37:0/731675898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69fc06e160 0x7f69fc1f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 >> 172.21.15.37:0/731675898 conn(0x7f69fc06f940 msgr2=0x7f69fc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 shutdown_connections 2024-09-21T06:29:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:21.625+0000 7f6a00a00640 1 -- 172.21.15.37:0/731675898 wait complete. 2024-09-21T06:29:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:21.650 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:29:21.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:21.703 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:29:21.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:21.747 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:29:21.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:21.825 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:29:21.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:21.902 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:29:21.979 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:29:22.227 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:29:22.228 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:29:22.228 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:29:22.282 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:29:22.280+0000 7fdd99200640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:29:24.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:29:24.479 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:29:24.480 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:29:24.480 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:29:26.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:29:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:29:26.728 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:29:26.728 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:29:28.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:29:28.977 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:29:28.978 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:29:28.978 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:29:30.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:29:31.228 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:29:31.229 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:31.229 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:29:31.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:29:31.487 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:29:31.488 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:31.488 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:29:33.489 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:29:33.744 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:29:33.745 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:33.745 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:29:35.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:29:36.002 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:29:36.002 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:36.003 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:29:38.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:29:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:29:38.262 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:38.262 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:29:38.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:29:38.506 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:29:38.507 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:29:38.507 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:29:38.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2024-09-21T06:29:38.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 -- 172.21.15.37:0/3394987742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0cc530 msgr2=0x7efc3c15ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 --2- 172.21.15.37:0/3394987742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0cc530 0x7efc3c15ead0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009830 tx=0x7efc2c02a510 comp rx=0 tx=0).stop 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 -- 172.21.15.37:0/3394987742 shutdown_connections 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 --2- 172.21.15.37:0/3394987742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c06e500 0x7efc3c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 --2- 172.21.15.37:0/3394987742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0cc530 0x7efc3c15ead0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.732+0000 7efc42600640 1 -- 172.21.15.37:0/3394987742 >> 172.21.15.37:0/3394987742 conn(0x7efc3c0b1b90 msgr2=0x7efc3c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.733+0000 7efc42600640 1 -- 172.21.15.37:0/3394987742 shutdown_connections 2024-09-21T06:29:38.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.733+0000 7efc42600640 1 -- 172.21.15.37:0/3394987742 wait complete. 2024-09-21T06:29:38.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.735+0000 7efc42600640 1 Processor -- start 2024-09-21T06:29:38.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.735+0000 7efc42600640 1 -- start start 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 0x7efc3c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c09f060 0x7efc3c1f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c0cede0 con 0x7efc3c0af5b0 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c0cec60 con 0x7efc3c0cc530 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.736+0000 7efc42600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c0cef60 con 0x7efc3c09f060 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44588/0 (socket says 172.21.15.37:44588) 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 -- 172.21.15.37:0/661464716 learned_addr learned my addr 172.21.15.37:0/661464716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3be00640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c09f060 0x7efc3c1f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3aa00640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 0x7efc3c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 -- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c09f060 msgr2=0x7efc3c1f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c09f060 0x7efc3c1f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 -- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 msgr2=0x7efc3c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 0x7efc3c0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3b400640 1 -- 172.21.15.37:0/661464716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c09dfc0 con 0x7efc3c0af5b0 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.737+0000 7efc3aa00640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 0x7efc3c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc3b400640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000c00 tx=0x7efc2c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc33400640 1 -- 172.21.15.37:0/661464716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c005c90 con 0x7efc3c0af5b0 2024-09-21T06:29:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc33400640 1 -- 172.21.15.37:0/661464716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc2c031e50 con 0x7efc3c0af5b0 2024-09-21T06:29:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc33400640 1 -- 172.21.15.37:0/661464716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0332f0 con 0x7efc3c0af5b0 2024-09-21T06:29:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c09e2a0 con 0x7efc3c0af5b0 2024-09-21T06:29:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.738+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c09e860 con 0x7efc3c0af5b0 2024-09-21T06:29:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.741+0000 7efc33400640 1 -- 172.21.15.37:0/661464716 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc2c033490 con 0x7efc3c0af5b0 2024-09-21T06:29:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.743+0000 7efc33400640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 0x7efc1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.743+0000 7efc33400640 1 -- 172.21.15.37:0/661464716 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7efc2c0bbfc0 con 0x7efc3c0af5b0 2024-09-21T06:29:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.743+0000 7efc3aa00640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 0x7efc1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.743+0000 7efc49607740 1 Processor -- start 2024-09-21T06:29:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.743+0000 7efc49607740 1 -- start start 2024-09-21T06:29:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.744+0000 7efc3aa00640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 0x7efc1c07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af160 tx=0x7efc28009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d8e4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56248d7fd1c0 con 0x56248d4cd4d0 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56248d810870 con 0x56248d4d56b0 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56248d7ffca0 con 0x56248d4f6390 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc3aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc3aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44598/0 (socket says 172.21.15.37:44598) 2024-09-21T06:29:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.745+0000 7efc3aa00640 1 -- 172.21.15.37:0/3079576129 learned_addr learned my addr 172.21.15.37:0/3079576129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 -- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 msgr2=0x56248d8e4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d8e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 -- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 msgr2=0x56248d8c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3be00640 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 -- 172.21.15.37:0/3079576129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248d5d87d0 con 0x56248d4cd4d0 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3be00640 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc3aa00640 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc28005100 tx=0x7efc28011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d93396fa2f3acd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc27e00640 1 -- 172.21.15.37:0/3079576129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280035f0 con 0x56248d4cd4d0 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc27e00640 1 -- 172.21.15.37:0/3079576129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc28003880 con 0x56248d4cd4d0 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.746+0000 7efc27e00640 1 -- 172.21.15.37:0/3079576129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28003c30 con 0x56248d4cd4d0 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 -- 172.21.15.37:0/3079576129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 msgr2=0x56248d8c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc28005100 tx=0x7efc28011000 comp rx=0 tx=0).stop 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 -- 172.21.15.37:0/3079576129 shutdown_connections 2024-09-21T06:29:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8c7220 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 --2- 172.21.15.37:0/3079576129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d8e4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 -- 172.21.15.37:0/3079576129 >> 172.21.15.37:0/3079576129 conn(0x56248d4ea4e0 msgr2=0x56248d89d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 -- 172.21.15.37:0/3079576129 shutdown_connections 2024-09-21T06:29:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.747+0000 7efc49607740 1 -- 172.21.15.37:0/3079576129 wait complete. 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.748+0000 7efc49607740 1 -- start start 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.748+0000 7efc49607740 1 Processor -- start 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d89c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56248d7a6410 con 0x56248d4cd4d0 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56248d803ab0 con 0x56248d4d56b0 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56248d570740 con 0x56248d4f6390 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc49607740 10 client.0 _ll_register_callbacks cb 0x56248d783360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44600/0 (socket says 172.21.15.37:44600) 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.750+0000 7efc3b400640 1 -- 172.21.15.37:0/3433190054 learned_addr learned my addr 172.21.15.37:0/3433190054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 -- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 msgr2=0x56248d89c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d89c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 -- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 msgr2=0x56248d8df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 -- 172.21.15.37:0/3433190054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248d561970 con 0x56248d4cd4d0 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc3b400640 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc2c080d10 tx=0x7efc2c083ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36131b4968a80dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc24c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0317a0 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7efc2c0880c0 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc28003880 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efc2c0835b0 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc49607740 10 client.5235 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.751+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56248d53f6a0 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc28003880 con 0x56248d4cd4d0 2024-09-21T06:29:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7efc2c082d00 con 0x56248d4cd4d0 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 10 client.5235 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 10 client.5235 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 20 client.5235 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d4b4f60 0x56248d89c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 4 client.5235 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56248d8127c0 con 0x56248d4b4f60 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.752+0000 7efc3aa00640 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d4b4f60 0x56248d89c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.753+0000 7efc3aa00640 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d4b4f60 0x56248d89c970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab16378bd317f43 server_cookie=99d8bb77e097020a in_seq=0 out_seq=0 2024-09-21T06:29:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.753+0000 7efc24c00640 10 client.5235 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc24c00640 1 -- 172.21.15.37:0/3433190054 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56248d8127c0 con 0x56248d4b4f60 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc24c00640 10 client.5235 handle_command_reply: tid=0 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 client.5235 shutdown 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 msgr2=0x56248d8e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc2c080d10 tx=0x7efc2c083ff0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 shutdown_connections 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d4b4f60 0x56248d89c970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4f6390 0x56248d89c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc24c00640 0 client.5235 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 --2- 172.21.15.37:0/3433190054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4cd4d0 0x56248d8e0510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 >> 172.21.15.37:0/3433190054 conn(0x56248d4ea4e0 msgr2=0x56248d8dba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.754+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 shutdown_connections 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.755+0000 7efc49607740 1 -- 172.21.15.37:0/3433190054 wait complete. 2024-09-21T06:29:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.755+0000 7efc49607740 20 client.5235 trim_cache size 0 max 16384 2024-09-21T06:29:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.770+0000 7efc49607740 1 Processor -- start 2024-09-21T06:29:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.770+0000 7efc49607740 1 -- start start 2024-09-21T06:29:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c202700 con 0x56248d4f6390 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c201a10 con 0x56248d4d56b0 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c201fd0 con 0x56248d4cd4d0 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44612/0 (socket says 172.21.15.37:44612) 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 -- 172.21.15.37:0/807070734 learned_addr learned my addr 172.21.15.37:0/807070734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3b400640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3aa00640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 -- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 msgr2=0x56248d8e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 -- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 msgr2=0x56248d8b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3be00640 1 -- 172.21.15.37:0/807070734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248d899fa0 con 0x56248d4f6390 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.772+0000 7efc3aa00640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.773+0000 7efc3be00640 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efc34005fd0 tx=0x7efc340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba6dfc102f89f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.773+0000 7efc27400640 1 -- 172.21.15.37:0/807070734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34004d20 con 0x56248d4f6390 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.773+0000 7efc27400640 1 -- 172.21.15.37:0/807070734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc34016070 con 0x56248d4f6390 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.773+0000 7efc27400640 1 -- 172.21.15.37:0/807070734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34005260 con 0x56248d4f6390 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 -- 172.21.15.37:0/807070734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 msgr2=0x56248d8b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efc34005fd0 tx=0x7efc340117b0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 -- 172.21.15.37:0/807070734 shutdown_connections 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d8b6f00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 --2- 172.21.15.37:0/807070734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 -- 172.21.15.37:0/807070734 >> 172.21.15.37:0/807070734 conn(0x56248d4ea4e0 msgr2=0x56248d8decf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 -- 172.21.15.37:0/807070734 shutdown_connections 2024-09-21T06:29:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.774+0000 7efc49607740 1 -- 172.21.15.37:0/807070734 wait complete. 2024-09-21T06:29:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.775+0000 7efc49607740 1 -- start start 2024-09-21T06:29:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.775+0000 7efc49607740 1 Processor -- start 2024-09-21T06:29:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1f50d0 con 0x56248d4f6390 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56248d8daa70 con 0x56248d4d56b0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.776+0000 7efc49607740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56248d965320 con 0x56248d4cd4d0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc49607740 10 client.0 _ll_register_callbacks cb 0x7efc2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44620/0 (socket says 172.21.15.37:44620) 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 -- 172.21.15.37:0/4060140719 learned_addr learned my addr 172.21.15.37:0/4060140719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3b400640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3aa00640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 -- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 msgr2=0x56248d8be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8be3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 -- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 msgr2=0x56248d8da530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8da530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.777+0000 7efc3be00640 1 -- 172.21.15.37:0/4060140719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248d8dd6b0 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc3be00640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efc34017000 tx=0x7efc34004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff6495b64becf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc24c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34003970 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7efc34003b10 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc34016070 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efc34003d10 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc49607740 10 client.5241 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56248d898680 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.778+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc34016070 con 0x56248d4f6390 2024-09-21T06:29:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7efc34030080 con 0x56248d4f6390 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 10 client.5241 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 10 client.5241 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 20 client.5241 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d9606c0 0x56248d8dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 4 client.5241 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56248d80e270 con 0x56248d9606c0 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.779+0000 7efc3b400640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d9606c0 0x56248d8dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.780+0000 7efc3b400640 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d9606c0 0x56248d8dd880 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ff98d9131a967c server_cookie=1c8308244b0b66f3 in_seq=0 out_seq=0 2024-09-21T06:29:38.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.780+0000 7efc24c00640 10 client.5241 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc24c00640 1 -- 172.21.15.37:0/4060140719 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56248d80e270 con 0x56248d9606c0 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc24c00640 10 client.5241 handle_command_reply: tid=0 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 client.5241 shutdown 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 msgr2=0x56248d964de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efc34017000 tx=0x7efc34004ce0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 shutdown_connections 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56248d9606c0 0x56248d8dd880 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56248d4f6390 0x56248d964de0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56248d4d56b0 0x56248d8da530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 --2- 172.21.15.37:0/4060140719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56248d4cd4d0 0x56248d8be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc24c00640 0 client.5241 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 >> 172.21.15.37:0/4060140719 conn(0x56248d4ea4e0 msgr2=0x56248d8c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 shutdown_connections 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 1 -- 172.21.15.37:0/4060140719 wait complete. 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.805+0000 7efc49607740 20 client.5241 trim_cache size 0 max 16384 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:29:38.809 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 msgr2=0x7efc1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 0x7efc1c07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af160 tx=0x7efc28009040 comp rx=0 tx=0).stop 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 msgr2=0x7efc3c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000c00 tx=0x7efc2c033c90 comp rx=0 tx=0).stop 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 shutdown_connections 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efc1c078bc0 0x7efc1c07b0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc3c09f060 0x7efc3c1f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc3c0cc530 0x7efc3c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 --2- 172.21.15.37:0/661464716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc3c0af5b0 0x7efc3c0b59d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 >> 172.21.15.37:0/661464716 conn(0x7efc3c0b1b90 msgr2=0x7efc3c09d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 shutdown_connections 2024-09-21T06:29:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:38.806+0000 7efc42600640 1 -- 172.21.15.37:0/661464716 wait complete. 2024-09-21T06:29:38.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2024-09-21T06:29:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.060+0000 7ff809e00640 1 -- 172.21.15.37:0/2263142727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff80406e500 msgr2=0x7ff80415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.060+0000 7ff809e00640 1 --2- 172.21.15.37:0/2263142727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff80406e500 0x7ff80415ead0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009830 tx=0x7ff7ec02a5c0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.061+0000 7ff809e00640 1 -- 172.21.15.37:0/2263142727 shutdown_connections 2024-09-21T06:29:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.061+0000 7ff809e00640 1 --2- 172.21.15.37:0/2263142727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff80406e500 0x7ff80415ead0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.061+0000 7ff809e00640 1 -- 172.21.15.37:0/2263142727 >> 172.21.15.37:0/2263142727 conn(0x7ff8040b1b90 msgr2=0x7ff8040b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.061+0000 7ff809e00640 1 -- 172.21.15.37:0/2263142727 shutdown_connections 2024-09-21T06:29:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.061+0000 7ff809e00640 1 -- 172.21.15.37:0/2263142727 wait complete. 2024-09-21T06:29:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.063+0000 7ff809e00640 1 Processor -- start 2024-09-21T06:29:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.063+0000 7ff809e00640 1 -- start start 2024-09-21T06:29:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040af5b0 0x7ff80415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cc530 0x7ff8041f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040ceff0 con 0x7ff804159e60 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040cee70 con 0x7ff8040cc530 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.064+0000 7ff809e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff8040cf170 con 0x7ff8040af5b0 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44642/0 (socket says 172.21.15.37:44642) 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 -- 172.21.15.37:0/159131325 learned_addr learned my addr 172.21.15.37:0/159131325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cc530 0x7ff8041f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff802a00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040af5b0 0x7ff80415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 -- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040af5b0 msgr2=0x7ff80415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040af5b0 0x7ff80415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 -- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cc530 msgr2=0x7ff8041f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cc530 0x7ff8041f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.065+0000 7ff803400640 1 -- 172.21.15.37:0/159131325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80415b8d0 con 0x7ff804159e60 2024-09-21T06:29:39.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff803400640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00cb10 tx=0x7ff7fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff7fb400640 1 -- 172.21.15.37:0/159131325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc016020 con 0x7ff804159e60 2024-09-21T06:29:39.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff7fb400640 1 -- 172.21.15.37:0/159131325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7fc00aca0 con 0x7ff804159e60 2024-09-21T06:29:39.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff7fb400640 1 -- 172.21.15.37:0/159131325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc012680 con 0x7ff804159e60 2024-09-21T06:29:39.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041590a0 con 0x7ff804159e60 2024-09-21T06:29:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.066+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff804159660 con 0x7ff804159e60 2024-09-21T06:29:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.068+0000 7ff7fb400640 1 -- 172.21.15.37:0/159131325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7fc019020 con 0x7ff804159e60 2024-09-21T06:29:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.069+0000 7ff7fb400640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 0x7ff7e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.070+0000 7ff7fb400640 1 -- 172.21.15.37:0/159131325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7ff7fc09b050 con 0x7ff804159e60 2024-09-21T06:29:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.070+0000 7ff802a00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 0x7ff7e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.070+0000 7ff810e68740 1 Processor -- start 2024-09-21T06:29:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.070+0000 7ff810e68740 1 -- start start 2024-09-21T06:29:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.071+0000 7ff802a00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 0x7ff7e007b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001720 tx=0x7ff7f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc434550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc417760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0cc34d1c0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0cc360870 con 0x55b0cc046390 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0cc34fca0 con 0x55b0cc0256b0 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff802a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc434550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44646/0 (socket says 172.21.15.37:44646) 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 -- 172.21.15.37:0/2294297596 learned_addr learned my addr 172.21.15.37:0/2294297596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff803400640 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc417760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 -- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 msgr2=0x55b0cc417760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc417760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 -- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 msgr2=0x55b0cc434550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc434550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.072+0000 7ff7fbe00640 1 -- 172.21.15.37:0/2294297596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0cc1287d0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.073+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009920 tx=0x7ff7ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44c36ca1e13fa85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.073+0000 7ff7f3400640 1 -- 172.21.15.37:0/2294297596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec005220 con 0x55b0cc01d4d0 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.073+0000 7ff7f3400640 1 -- 172.21.15.37:0/2294297596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7ec0054b0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.073+0000 7ff7f3400640 1 -- 172.21.15.37:0/2294297596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004a00 con 0x55b0cc01d4d0 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 -- 172.21.15.37:0/2294297596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 msgr2=0x55b0cc417220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009920 tx=0x7ff7ec0046c0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 -- 172.21.15.37:0/2294297596 shutdown_connections 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc417220 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc417760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 --2- 172.21.15.37:0/2294297596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc434550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 -- 172.21.15.37:0/2294297596 >> 172.21.15.37:0/2294297596 conn(0x55b0cc03a4e0 msgr2=0x55b0cc3ed440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 -- 172.21.15.37:0/2294297596 shutdown_connections 2024-09-21T06:29:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.074+0000 7ff810e68740 1 -- 172.21.15.37:0/2294297596 wait complete. 2024-09-21T06:29:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.075+0000 7ff810e68740 1 -- start start 2024-09-21T06:29:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.075+0000 7ff810e68740 1 Processor -- start 2024-09-21T06:29:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc444ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc4451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0cc2f6410 con 0x55b0cc01d4d0 2024-09-21T06:29:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0cc353ab0 con 0x55b0cc046390 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0cc0c0740 con 0x55b0cc0256b0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff810e68740 10 client.0 _ll_register_callbacks cb 0x55b0cc2d3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff802a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff802a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44660/0 (socket says 172.21.15.37:44660) 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff802a00640 1 -- 172.21.15.37:0/1938727501 learned_addr learned my addr 172.21.15.37:0/1938727501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc444ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.077+0000 7ff803400640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc4451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 -- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 msgr2=0x55b0cc444ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc444ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 -- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 msgr2=0x55b0cc4451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc4451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 -- 172.21.15.37:0/1938727501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0cc0b1970 con 0x55b0cc01d4d0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc444ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff802a00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4014700 tx=0x7ff7f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910e8d9660ab5157 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4025020 con 0x55b0cc01d4d0 2024-09-21T06:29:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7ff7f4013740 con 0x55b0cc01d4d0 2024-09-21T06:29:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.078+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ec0054b0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff7f402f050 con 0x55b0cc01d4d0 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 10 client.5256 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b0cc08f6a0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7ec0054b0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7ff7f4032020 con 0x55b0cc01d4d0 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 10 client.5256 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 10 client.5256 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 20 client.5256 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc004f60 0x55b0cc42bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 4 client.5256 mds_command: new command op to 4977 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 --> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0cc3627c0 con 0x55b0cc004f60 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.079+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc004f60 0x55b0cc42bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.080+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc004f60 0x55b0cc42bad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3eeabe264f296e77 server_cookie=3bf1ff4aa997ed78 in_seq=0 out_seq=0 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.080+0000 7ff7e7e00640 10 client.5256 ms_handle_connect on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff7e7e00640 1 -- 172.21.15.37:0/1938727501 <== mds.1 v2:172.21.15.37:6833/2364291417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0cc3627c0 con 0x55b0cc004f60 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff7e7e00640 10 client.5256 handle_command_reply: tid=0 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 client.5256 shutdown 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 msgr2=0x55b0cc42a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4014700 tx=0x7ff7f4000ed0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 shutdown_connections 2024-09-21T06:29:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc4451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc444ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc004f60 0x55b0cc42bad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 --2- 172.21.15.37:0/1938727501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc42a2a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff7e7e00640 0 client.5256 ms_handle_reset on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 >> 172.21.15.37:0/1938727501 conn(0x55b0cc03a4e0 msgr2=0x55b0cc417220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 shutdown_connections 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 1 -- 172.21.15.37:0/1938727501 wait complete. 2024-09-21T06:29:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.081+0000 7ff810e68740 20 client.5256 trim_cache size 0 max 16384 2024-09-21T06:29:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.098+0000 7ff810e68740 1 Processor -- start 2024-09-21T06:29:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.098+0000 7ff810e68740 1 -- start start 2024-09-21T06:29:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.098+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc01d4d0 0x55b0cc3ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.098+0000 7ff810e68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.098+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc411a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff810e68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8042028f0 con 0x55b0cc0256b0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff804201ac0 con 0x55b0cc046390 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff8042021c0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44664/0 (socket says 172.21.15.37:44664) 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 -- 172.21.15.37:0/3286221137 learned_addr learned my addr 172.21.15.37:0/3286221137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff803400640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc411a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff802a00640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc01d4d0 0x55b0cc3ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 -- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc01d4d0 msgr2=0x55b0cc3ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc01d4d0 0x55b0cc3ec4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 -- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 msgr2=0x55b0cc411a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc411a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 -- 172.21.15.37:0/3286221137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0cc42be40 con 0x55b0cc0256b0 2024-09-21T06:29:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff803400640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc411a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.099+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009780 tx=0x7ff7ec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2205e9fedc738543 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.100+0000 7ff7f2a00640 1 -- 172.21.15.37:0/3286221137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0467b0 con 0x55b0cc0256b0 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.100+0000 7ff7f2a00640 1 -- 172.21.15.37:0/3286221137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7ec0054b0 con 0x55b0cc0256b0 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.100+0000 7ff7f2a00640 1 -- 172.21.15.37:0/3286221137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec046bf0 con 0x55b0cc0256b0 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.100+0000 7ff810e68740 1 -- 172.21.15.37:0/3286221137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 msgr2=0x55b0cc411530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.100+0000 7ff810e68740 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009780 tx=0x7ff7ec005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 -- 172.21.15.37:0/3286221137 shutdown_connections 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc0256b0 0x55b0cc411530 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc411a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 --2- 172.21.15.37:0/3286221137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc01d4d0 0x55b0cc3ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 -- 172.21.15.37:0/3286221137 >> 172.21.15.37:0/3286221137 conn(0x55b0cc03a4e0 msgr2=0x55b0cc3e68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 -- 172.21.15.37:0/3286221137 shutdown_connections 2024-09-21T06:29:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.101+0000 7ff810e68740 1 -- 172.21.15.37:0/3286221137 wait complete. 2024-09-21T06:29:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.102+0000 7ff810e68740 1 -- start start 2024-09-21T06:29:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.102+0000 7ff810e68740 1 Processor -- start 2024-09-21T06:29:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc4aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc40dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff804200b60 con 0x55b0cc01d4d0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0cc40edd0 con 0x55b0cc046390 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0cc413be0 con 0x55b0cc0256b0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.103+0000 7ff810e68740 10 client.0 _ll_register_callbacks cb 0x7ff7f4001400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44668/0 (socket says 172.21.15.37:44668) 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 -- 172.21.15.37:0/3635547222 learned_addr learned my addr 172.21.15.37:0/3635547222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff803400640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc40dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc4aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 -- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 msgr2=0x55b0cc4aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc4aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 -- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 msgr2=0x55b0cc40dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc40dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 -- 172.21.15.37:0/3635547222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0cc417a30 con 0x55b0cc01d4d0 2024-09-21T06:29:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff802a00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001eb0 tx=0x7ff7f4008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152b7e3b604e275 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.104+0000 7ff7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4025040 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7ff7f402a800 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ec0054b0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff7f4041050 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff810e68740 10 client.5262 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b0cc42c220 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7ec0054b0 con 0x55b0cc01d4d0 2024-09-21T06:29:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.105+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7ff7f4044020 con 0x55b0cc01d4d0 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 10 client.5262 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 10 client.5262 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 20 client.5262 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc40e510 0x55b0cc411530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 4 client.5262 mds_command: new command op to 4977 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 --> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0cc35e270 con 0x55b0cc40e510 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc40e510 0x55b0cc411530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff7fbe00640 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc40e510 0x55b0cc411530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74be8171bb5080f3 server_cookie=1bded0cc13415edb in_seq=0 out_seq=0 2024-09-21T06:29:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.106+0000 7ff7e7e00640 10 client.5262 ms_handle_connect on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff7e7e00640 1 -- 172.21.15.37:0/3635547222 <== mds.1 v2:172.21.15.37:6833/2364291417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0cc35e270 con 0x55b0cc40e510 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff7e7e00640 10 client.5262 handle_command_reply: tid=0 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 client.5262 shutdown 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 msgr2=0x55b0cc4172b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001eb0 tx=0x7ff7f4008f30 comp rx=0 tx=0).stop 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 shutdown_connections 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0cc046390 0x55b0cc40dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0cc0256b0 0x55b0cc4aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:6833/2364291417,v1:172.21.15.37:6835/2364291417] conn(0x55b0cc40e510 0x55b0cc411530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 --2- 172.21.15.37:0/3635547222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0cc01d4d0 0x55b0cc4172b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff7e7e00640 0 client.5262 ms_handle_reset on v2:172.21.15.37:6833/2364291417 2024-09-21T06:29:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 >> 172.21.15.37:0/3635547222 conn(0x55b0cc03a4e0 msgr2=0x55b0cc3ee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 shutdown_connections 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.118+0000 7ff810e68740 1 -- 172.21.15.37:0/3635547222 wait complete. 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff810e68740 20 client.5262 trim_cache size 0 max 16384 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:29:39.122 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 msgr2=0x7ff7e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 0x7ff7e007b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001720 tx=0x7ff7f4009040 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 msgr2=0x7ff8041f8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00cb10 tx=0x7ff7fc00a060 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 shutdown_connections 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff7e0078bc0 0x7ff7e007b0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff804159e60 0x7ff8041f8e40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cc530 0x7ff8041f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 --2- 172.21.15.37:0/159131325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040af5b0 0x7ff80415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 >> 172.21.15.37:0/159131325 conn(0x7ff8040b1b90 msgr2=0x7ff8040cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.119+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 shutdown_connections 2024-09-21T06:29:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.120+0000 7ff809e00640 1 -- 172.21.15.37:0/159131325 wait complete. 2024-09-21T06:29:39.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2024-09-21T06:29:39.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 -- 172.21.15.37:0/3990061363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40af5b0 msgr2=0x7f0ca406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/3990061363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40af5b0 0x7f0ca406ddf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4075240 tx=0x7f0c9002a540 comp rx=0 tx=0).stop 2024-09-21T06:29:39.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 -- 172.21.15.37:0/3990061363 shutdown_connections 2024-09-21T06:29:39.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/3990061363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40af5b0 0x7f0ca406ddf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/3990061363 >> v2:172.21.15.150:3300/0 conn(0x7f0ca40cc530 0x7f0ca415ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 -- 172.21.15.37:0/3990061363 >> 172.21.15.37:0/3990061363 conn(0x7f0ca40b1b90 msgr2=0x7f0ca40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 -- 172.21.15.37:0/3990061363 shutdown_connections 2024-09-21T06:29:39.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.376+0000 7f0ca9e00640 1 -- 172.21.15.37:0/3990061363 wait complete. 2024-09-21T06:29:39.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.379+0000 7f0ca9e00640 1 Processor -- start 2024-09-21T06:29:39.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.379+0000 7f0ca9e00640 1 -- start start 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca406e330 0x7f0ca409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca40af5b0 0x7f0ca4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40cae00 con 0x7f0ca40cc530 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40cbd60 con 0x7f0ca40af5b0 2024-09-21T06:29:39.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca9e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca40caf80 con 0x7f0ca406e330 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44688/0 (socket says 172.21.15.37:44688) 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.380+0000 7f0ca3400640 1 -- 172.21.15.37:0/930909683 learned_addr learned my addr 172.21.15.37:0/930909683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca2000640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca40af5b0 0x7f0ca4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca406e330 0x7f0ca409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 -- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca406e330 msgr2=0x7f0ca409eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca406e330 0x7f0ca409eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 -- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca40af5b0 msgr2=0x7f0ca4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca40af5b0 0x7f0ca4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 -- 172.21.15.37:0/930909683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca4159500 con 0x7f0ca40cc530 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0ca3400640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00cb10 tx=0x7f0c9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.381+0000 7f0c9b400640 1 -- 172.21.15.37:0/930909683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c016020 con 0x7f0ca40cc530 2024-09-21T06:29:39.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.382+0000 7f0c9b400640 1 -- 172.21.15.37:0/930909683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c00aca0 con 0x7f0ca40cc530 2024-09-21T06:29:39.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.382+0000 7f0c9b400640 1 -- 172.21.15.37:0/930909683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c012680 con 0x7f0ca40cc530 2024-09-21T06:29:39.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.382+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca41f9040 con 0x7f0ca40cc530 2024-09-21T06:29:39.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.382+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41f9600 con 0x7f0ca40cc530 2024-09-21T06:29:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.384+0000 7f0c9b400640 1 -- 172.21.15.37:0/930909683 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c9c019020 con 0x7f0ca40cc530 2024-09-21T06:29:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.385+0000 7f0c9b400640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 0x7f0c8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.385+0000 7f0c9b400640 1 -- 172.21.15.37:0/930909683 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0c9c09b090 con 0x7f0ca40cc530 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.385+0000 7f0cb0e8a740 1 Processor -- start 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.386+0000 7f0cb0e8a740 1 -- start start 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.386+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 0x7f0c8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.386+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 0x7f0c8407b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400bb50 tx=0x7f0c9400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5615843e41c0 con 0x5615840dd390 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5615843f7870 con 0x5615840bc6b0 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5615843e6ca0 con 0x5615840b44d0 2024-09-21T06:29:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44704/0 (socket says 172.21.15.37:44704) 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2151513366 learned_addr learned my addr 172.21.15.37:0/2151513366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.387+0000 7f0ca3400640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2000640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 msgr2=0x5615844ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 msgr2=0x5615844ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2151513366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615841bf7d0 con 0x5615840dd390 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2000640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c94010fe0 tx=0x7f0c94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275359dd8514fe2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0c8f400640 1 -- 172.21.15.37:0/2151513366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401e620 con 0x5615840dd390 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0c8f400640 1 -- 172.21.15.37:0/2151513366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9401e8b0 con 0x5615840dd390 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.388+0000 7f0c8f400640 1 -- 172.21.15.37:0/2151513366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401eca0 con 0x5615840dd390 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2151513366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 msgr2=0x5615844cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c94010fe0 tx=0x7f0c94013000 comp rx=0 tx=0).stop 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2151513366 shutdown_connections 2024-09-21T06:29:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2151513366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844cb550 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2151513366 >> 172.21.15.37:0/2151513366 conn(0x5615840d14e0 msgr2=0x561584484440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2151513366 shutdown_connections 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.389+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2151513366 wait complete. 2024-09-21T06:29:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.390+0000 7f0cb0e8a740 1 -- start start 2024-09-21T06:29:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.390+0000 7f0cb0e8a740 1 Processor -- start 2024-09-21T06:29:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56158438d410 con 0x5615840dd390 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5615843eaab0 con 0x5615840bc6b0 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561584157740 con 0x5615840b44d0 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 10 client.0 _ll_register_callbacks cb 0x56158436a360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44712/0 (socket says 172.21.15.37:44712) 2024-09-21T06:29:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 -- 172.21.15.37:0/3566703239 learned_addr learned my addr 172.21.15.37:0/3566703239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca2000640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 -- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 msgr2=0x5615844c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 -- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 msgr2=0x5615844d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 -- 172.21.15.37:0/3566703239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561584148970 con 0x5615840dd390 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca2000640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0ca3400640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0629b0 tx=0x7f0c9c062420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf547313054f0f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0676e0 con 0x5615840dd390 2024-09-21T06:29:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0c9c061f10 con 0x5615840dd390 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9401e8b0 con 0x5615840dd390 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c9c06c590 con 0x5615840dd390 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 10 client.5277 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.393+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5615841266a0 con 0x5615840dd390 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9401e8b0 con 0x5615840dd390 2024-09-21T06:29:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f0c9c083070 con 0x5615840dd390 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 10 client.5277 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 10 client.5277 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 20 client.5277 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x56158409bf60 0x5615844db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 4 client.5277 mds_command: new command op to 4974 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 --> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615843f97c0 con 0x56158409bf60 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.394+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x56158409bf60 0x5615844db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.395+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x56158409bf60 0x5615844db930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b01bbe7f9a831130 server_cookie=abce64b347337229 in_seq=0 out_seq=0 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.395+0000 7f0c83e00640 10 client.5277 ms_handle_connect on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.395+0000 7f0c83e00640 1 -- 172.21.15.37:0/3566703239 <== mds.2 v2:172.21.15.37:6832/2753796335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615843f97c0 con 0x56158409bf60 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.395+0000 7f0c83e00640 10 client.5277 handle_command_reply: tid=0 2024-09-21T06:29:39.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.395+0000 7f0cb0e8a740 1 client.5277 shutdown 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 msgr2=0x5615844d9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0629b0 tx=0x7f0c9c062420 comp rx=0 tx=0).stop 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 shutdown_connections 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x56158409bf60 0x5615844db930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840dd390 0x5615844d9a40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x5615844d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0c83e00640 0 client.5277 ms_handle_reset on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/3566703239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 >> 172.21.15.37:0/3566703239 conn(0x5615840d14e0 msgr2=0x56158449d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 shutdown_connections 2024-09-21T06:29:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/3566703239 wait complete. 2024-09-21T06:29:39.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.396+0000 7f0cb0e8a740 20 client.5277 trim_cache size 0 max 16384 2024-09-21T06:29:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.412+0000 7f0cb0e8a740 1 Processor -- start 2024-09-21T06:29:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.412+0000 7f0cb0e8a740 1 -- start start 2024-09-21T06:29:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 0x5615844aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840dd390 0x5615844ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4202670 con 0x5615840bc6b0 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4201840 con 0x5615840dd390 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.413+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca4201f40 con 0x5615840b44d0 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 0x5615844aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:47738/0 (socket says 172.21.15.37:47738) 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 0x5615844aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44718/0 (socket says 172.21.15.37:44718) 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 -- 172.21.15.37:0/1971087385 learned_addr learned my addr 172.21.15.37:0/1971087385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca3400640 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840dd390 0x5615844ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 -- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840dd390 msgr2=0x5615844ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840dd390 0x5615844ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 -- 172.21.15.37:0/1971087385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 msgr2=0x5615844aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 0x5615844aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 -- 172.21.15.37:0/1971087385 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615844cbf00 con 0x5615840b44d0 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.414+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c94010860 tx=0x7f0c94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358d51b6cbaafe21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.415+0000 7f0c8ea00640 1 -- 172.21.15.37:0/1971087385 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401e8b0 con 0x5615840b44d0 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.415+0000 7f0c8ea00640 1 -- 172.21.15.37:0/1971087385 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9401ea50 con 0x5615840b44d0 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.415+0000 7f0c8ea00640 1 -- 172.21.15.37:0/1971087385 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401ee40 con 0x5615840b44d0 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.415+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 msgr2=0x5615844c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.415+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c94010860 tx=0x7f0c94004730 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/1971087385 shutdown_connections 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840bc6b0 0x5615844aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840dd390 0x5615844ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/1971087385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840b44d0 0x5615844c4c30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/1971087385 >> 172.21.15.37:0/1971087385 conn(0x5615840d14e0 msgr2=0x5615844ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/1971087385 shutdown_connections 2024-09-21T06:29:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.416+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/1971087385 wait complete. 2024-09-21T06:29:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.417+0000 7f0cb0e8a740 1 -- start start 2024-09-21T06:29:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.417+0000 7f0cb0e8a740 1 Processor -- start 2024-09-21T06:29:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x56158449da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 0x5615844a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca415a0e0 con 0x5615840b44d0 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561584549790 con 0x5615840bc6b0 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5615844c2bd0 con 0x5615840dd390 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.418+0000 7f0cb0e8a740 10 client.0 _ll_register_callbacks cb 0x7f0c9c05f300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44734/0 (socket says 172.21.15.37:44734) 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2900551492 learned_addr learned my addr 172.21.15.37:0/2900551492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2000640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x56158449da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca3400640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 0x5615844a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 msgr2=0x5615844a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 0x5615844a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 msgr2=0x56158449da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x56158449da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 -- 172.21.15.37:0/2900551492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561584549580 con 0x5615840b44d0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca3400640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 0x5615844a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0ca2a00640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c94022000 tx=0x7f0c94005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44084fc66b3aad9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.419+0000 7f0c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401e8b0 con 0x5615840b44d0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0c94033620 con 0x5615840b44d0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9401ea50 con 0x5615840b44d0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c9404a050 con 0x5615840b44d0 2024-09-21T06:29:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 10 client.5283 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5615844e24e0 con 0x5615840b44d0 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9401ea50 con 0x5615840b44d0 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f0c9404d020 con 0x5615840b44d0 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 10 client.5283 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 10 client.5283 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 20 client.5283 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x561584547f20 0x5615844a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 4 client.5283 mds_command: new command op to 4974 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.420+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 --> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615843f5270 con 0x561584547f20 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.421+0000 7f0ca2000640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x561584547f20 0x5615844a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.421+0000 7f0ca2000640 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x561584547f20 0x5615844a4d40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b044c35323d6b0fd server_cookie=475d56bf646061a8 in_seq=0 out_seq=0 2024-09-21T06:29:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.421+0000 7f0c83e00640 10 client.5283 ms_handle_connect on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.434+0000 7f0c83e00640 1 -- 172.21.15.37:0/2900551492 <== mds.2 v2:172.21.15.37:6832/2753796335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615843f5270 con 0x561584547f20 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.434+0000 7f0c83e00640 10 client.5283 handle_command_reply: tid=0 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.434+0000 7f0cb0e8a740 1 client.5283 shutdown 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 msgr2=0x561584548e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c94022000 tx=0x7f0c94005c50 comp rx=0 tx=0).stop 2024-09-21T06:29:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 shutdown_connections 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:6832/2753796335,v1:172.21.15.37:6834/2753796335] conn(0x561584547f20 0x5615844a4d40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615840dd390 0x5615844a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615840bc6b0 0x56158449da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0c83e00640 0 client.5283 ms_handle_reset on v2:172.21.15.37:6832/2753796335 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 --2- 172.21.15.37:0/2900551492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615840b44d0 0x561584548e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 >> 172.21.15.37:0/2900551492 conn(0x5615840d14e0 msgr2=0x561584485350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 shutdown_connections 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 1 -- 172.21.15.37:0/2900551492 wait complete. 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.435+0000 7f0cb0e8a740 20 client.5283 trim_cache size 0 max 16384 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:29:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:29:39.439 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 msgr2=0x7f0c8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 0x7f0c8407b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400bb50 tx=0x7f0c9400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 msgr2=0x7f0ca409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00cb10 tx=0x7f0c9c00a060 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 shutdown_connections 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0c84078b90 0x7f0c8407b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ca40cc530 0x7f0ca409cbc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca40af5b0 0x7f0ca4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 --2- 172.21.15.37:0/930909683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca406e330 0x7f0ca409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 >> 172.21.15.37:0/930909683 conn(0x7f0ca40b1b90 msgr2=0x7f0ca415ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 shutdown_connections 2024-09-21T06:29:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39.436+0000 7f0ca9e00640 1 -- 172.21.15.37:0/930909683 wait complete. 2024-09-21T06:29:39.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:39.463 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-21T06:29:39.508 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/ 2024-09-21T06:29:39.508 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4 2024-09-21T06:29:39.508 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5 2024-09-21T06:29:39.508 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6 2024-09-21T06:29:39.508 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-21T06:29:39.509 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-21T06:29:39.510 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-09-21T06:29:39.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-21T06:29:39.661 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:39.802 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-09-21T06:29:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-21T06:29:39.813 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:39.953 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-09-21T06:29:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-21T06:29:39.965 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:40.105 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627781 2024-09-21T06:29:40.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:40.116 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:40.116 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:40.116 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:40.117 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:40.117 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-21T06:29:40.117 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627782 2024-09-21T06:29:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-21T06:29:40.269 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:29:40.412 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627783 2024-09-21T06:29:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:29:40.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.695+0000 7f8808e00640 1 -- 172.21.15.37:0/648620010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f880415f720 msgr2=0x7f8804155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.695+0000 7f8808e00640 1 --2- 172.21.15.37:0/648620010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f880415f720 0x7f8804155570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009860 tx=0x7f87f402a580 comp rx=0 tx=0).stop 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.696+0000 7f8808e00640 1 -- 172.21.15.37:0/648620010 shutdown_connections 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.696+0000 7f8808e00640 1 --2- 172.21.15.37:0/648620010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f880415f720 0x7f8804155570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.696+0000 7f8808e00640 1 -- 172.21.15.37:0/648620010 >> 172.21.15.37:0/648620010 conn(0x7f88040d0460 msgr2=0x7f880406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.696+0000 7f8808e00640 1 -- 172.21.15.37:0/648620010 shutdown_connections 2024-09-21T06:29:40.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.696+0000 7f8808e00640 1 -- 172.21.15.37:0/648620010 wait complete. 2024-09-21T06:29:40.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.698+0000 7f8808e00640 1 Processor -- start 2024-09-21T06:29:40.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.698+0000 7f8808e00640 1 -- start start 2024-09-21T06:29:40.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.699+0000 7f8808e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88040732a0 0x7f88041eb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8808e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8808e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415f720 0x7f88040af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8808e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88040af3a0 con 0x7f88040b49b0 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8808e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88040af220 con 0x7f88040732a0 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8808e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f88040af520 con 0x7f880415f720 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44744/0 (socket says 172.21.15.37:44744) 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 -- 172.21.15.37:0/128250235 learned_addr learned my addr 172.21.15.37:0/128250235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8803400640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88040732a0 0x7f88041eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f8803e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415f720 0x7f88040af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 -- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415f720 msgr2=0x7f88040af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415f720 0x7f88040af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 -- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88040732a0 msgr2=0x7f88041eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88040732a0 0x7f88041eb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.700+0000 7f87fbe00640 1 -- 172.21.15.37:0/128250235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88041eb740 con 0x7f88040b49b0 2024-09-21T06:29:40.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f87fbe00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c3a0 tx=0x7f87ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f87fb400640 1 -- 172.21.15.37:0/128250235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec017070 con 0x7f88040b49b0 2024-09-21T06:29:40.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f87fb400640 1 -- 172.21.15.37:0/128250235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ec0053c0 con 0x7f88040b49b0 2024-09-21T06:29:40.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88041f94a0 con 0x7f88040b49b0 2024-09-21T06:29:40.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f87fb400640 1 -- 172.21.15.37:0/128250235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003850 con 0x7f88040b49b0 2024-09-21T06:29:40.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.701+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8804141db0 con 0x7f88040b49b0 2024-09-21T06:29:40.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.704+0000 7f87fb400640 1 -- 172.21.15.37:0/128250235 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87ec01a070 con 0x7f88040b49b0 2024-09-21T06:29:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.705+0000 7f87fb400640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 0x7f87e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.705+0000 7f87fb400640 1 -- 172.21.15.37:0/128250235 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f87ec09a230 con 0x7f88040b49b0 2024-09-21T06:29:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.705+0000 7f8803400640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 0x7f87e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.705+0000 7f8811455740 1 Processor -- start 2024-09-21T06:29:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.705+0000 7f8811455740 1 -- start start 2024-09-21T06:29:40.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.706+0000 7f8803400640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 0x7f87e007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87f4034000 tx=0x7f87f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914598b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141836f0 0x56091456d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5609144c6970 con 0x56091418b8d0 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609144b5da0 con 0x5609141836f0 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5609144b9bb0 con 0x5609141ac5b0 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8803e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8803e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44748/0 (socket says 172.21.15.37:44748) 2024-09-21T06:29:40.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8803e00640 1 -- 172.21.15.37:0/2520524716 learned_addr learned my addr 172.21.15.37:0/2520524716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:40.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f8803400640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914598b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.707+0000 7f87fbe00640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141836f0 0x56091456d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 -- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 msgr2=0x560914598b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914598b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 -- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141836f0 msgr2=0x56091456d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141836f0 0x56091456d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 -- 172.21.15.37:0/2520524716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560914290a20 con 0x56091418b8d0 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803400640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914598b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f8803e00640 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00f330 tx=0x7f87fc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483be1cf8608153 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f87f3e00640 1 -- 172.21.15.37:0/2520524716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc018020 con 0x56091418b8d0 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f87f3e00640 1 -- 172.21.15.37:0/2520524716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87fc014680 con 0x56091418b8d0 2024-09-21T06:29:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.708+0000 7f87f3e00640 1 -- 172.21.15.37:0/2520524716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc014a70 con 0x56091418b8d0 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 -- 172.21.15.37:0/2520524716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 msgr2=0x56091456df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00f330 tx=0x7f87fc00f7f0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 -- 172.21.15.37:0/2520524716 shutdown_connections 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091456df00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141836f0 0x56091456d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 --2- 172.21.15.37:0/2520524716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914598b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 -- 172.21.15.37:0/2520524716 >> 172.21.15.37:0/2520524716 conn(0x5609141a0700 msgr2=0x560914553e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 -- 172.21.15.37:0/2520524716 shutdown_connections 2024-09-21T06:29:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.709+0000 7f8811455740 1 -- 172.21.15.37:0/2520524716 wait complete. 2024-09-21T06:29:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.710+0000 7f8811455740 1 -- start start 2024-09-21T06:29:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.711+0000 7f8811455740 1 Processor -- start 2024-09-21T06:29:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x56091456ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914594100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560914226960 con 0x5609141836f0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56091445b780 con 0x56091418b8d0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560914594f00 con 0x5609141ac5b0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8811455740 10 client.0 _ll_register_callbacks cb 0x5609144b24b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8803400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8803400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44758/0 (socket says 172.21.15.37:44758) 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.712+0000 7f8803400640 1 -- 172.21.15.37:0/3961006464 learned_addr learned my addr 172.21.15.37:0/3961006464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x56091456ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803e00640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914594100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 -- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 msgr2=0x560914594100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914594100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 -- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 msgr2=0x56091456ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x56091456ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 -- 172.21.15.37:0/3961006464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609141f58c0 con 0x5609141836f0 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803e00640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914594100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8803400640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009950 tx=0x7f87f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457ecdb0f9ea560c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f87f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4005020 con 0x5609141836f0 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f87f40051c0 con 0x5609141836f0 2024-09-21T06:29:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.713+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc014680 con 0x5609141836f0 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f87f4059050 con 0x5609141836f0 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 10 client.5295 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5609142f11b0 con 0x5609141836f0 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87fc014680 con 0x5609141836f0 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f87f4043040 con 0x5609141836f0 2024-09-21T06:29:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 10 client.5295 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 10 client.5295 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 20 client.5295 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56091416b180 0x5609145946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 4 client.5295 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.714+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609144c88c0 con 0x56091416b180 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.715+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56091416b180 0x5609145946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.715+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56091416b180 0x5609145946a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b073f8eed9eb0446 server_cookie=fc88f3471417be58 in_seq=0 out_seq=0 2024-09-21T06:29:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.715+0000 7f87f0c00640 10 client.5295 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.716+0000 7f87f0c00640 1 -- 172.21.15.37:0/3961006464 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609144c88c0 con 0x56091416b180 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.716+0000 7f87f0c00640 10 client.5295 handle_command_reply: tid=0 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 client.5295 shutdown 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 msgr2=0x560914592930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009950 tx=0x7f87f4005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 shutdown_connections 2024-09-21T06:29:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x56091416b180 0x5609145946a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141ac5b0 0x560914594100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x56091456ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 --2- 172.21.15.37:0/3961006464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141836f0 0x560914592930 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f87f0c00640 0 client.5295 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 >> 172.21.15.37:0/3961006464 conn(0x5609141a0700 msgr2=0x56091454eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 shutdown_connections 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 1 -- 172.21.15.37:0/3961006464 wait complete. 2024-09-21T06:29:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.717+0000 7f8811455740 20 client.5295 trim_cache size 0 max 16384 2024-09-21T06:29:40.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.737+0000 7f8811455740 1 Processor -- start 2024-09-21T06:29:40.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.737+0000 7f8811455740 1 -- start start 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141ac5b0 0x560914595090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8804202490 con 0x56091418b8d0 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8804201660 con 0x5609141ac5b0 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8804201d60 con 0x5609141836f0 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.738+0000 7f87fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44770/0 (socket says 172.21.15.37:44770) 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 -- 172.21.15.37:0/3011817274 learned_addr learned my addr 172.21.15.37:0/3011817274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f8803400640 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f8803e00640 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141ac5b0 0x560914595090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 -- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 msgr2=0x5609145bc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145bc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 -- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141ac5b0 msgr2=0x560914595090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141ac5b0 0x560914595090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 -- 172.21.15.37:0/3011817274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091428ed20 con 0x56091418b8d0 2024-09-21T06:29:40.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00cd60 tx=0x7f87ec01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cee3d52e640eaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87f3400640 1 -- 172.21.15.37:0/3011817274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec066800 con 0x56091418b8d0 2024-09-21T06:29:40.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87f3400640 1 -- 172.21.15.37:0/3011817274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ec06b6b0 con 0x56091418b8d0 2024-09-21T06:29:40.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.739+0000 7f87f3400640 1 -- 172.21.15.37:0/3011817274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0840b0 con 0x56091418b8d0 2024-09-21T06:29:40.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 -- 172.21.15.37:0/3011817274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 msgr2=0x56091454ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00cd60 tx=0x7f87ec01e040 comp rx=0 tx=0).stop 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 -- 172.21.15.37:0/3011817274 shutdown_connections 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56091418b8d0 0x56091454ed70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609141ac5b0 0x560914595090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 --2- 172.21.15.37:0/3011817274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145bc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 -- 172.21.15.37:0/3011817274 >> 172.21.15.37:0/3011817274 conn(0x5609141a0700 msgr2=0x5609145abbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 -- 172.21.15.37:0/3011817274 shutdown_connections 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.740+0000 7f8811455740 1 -- 172.21.15.37:0/3011817274 wait complete. 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.741+0000 7f8811455740 1 -- start start 2024-09-21T06:29:40.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.741+0000 7f8811455740 1 Processor -- start 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x5609145bbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560914598c50 con 0x5609141ac5b0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88041e7850 con 0x56091418b8d0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56091457e0c0 con 0x5609141836f0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8811455740 10 client.0 _ll_register_callbacks cb 0x7f87f40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44772/0 (socket says 172.21.15.37:44772) 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803e00640 1 -- 172.21.15.37:0/3534914436 learned_addr learned my addr 172.21.15.37:0/3534914436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803400640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f87fbe00640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x5609145bbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803e00640 1 -- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 msgr2=0x5609145abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.743+0000 7f8803e00640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8803e00640 1 -- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 msgr2=0x5609145bbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8803e00640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x5609145bbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8803e00640 1 -- 172.21.15.37:0/3534914436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560914593130 con 0x5609141ac5b0 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8803400640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8803e00640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f87fc016000 tx=0x7f87fc014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599b1ffa487006a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:40.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f87f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:40.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc018040 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f87fc029480 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec06b6b0 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f87fc031050 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8811455740 10 client.5301 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.744+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5609145ac150 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ec06b6b0 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f87fc034020 con 0x5609141ac5b0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 10 client.5301 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 10 client.5301 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 20 client.5301 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5609145938f0 0x5609145ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 4 client.5301 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) -- 0x5609144c4370 con 0x5609145938f0 2024-09-21T06:29:40.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.745+0000 7f8803400640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5609145938f0 0x5609145ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:40.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.746+0000 7f8803400640 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5609145938f0 0x5609145ba860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a287420831ff871 server_cookie=5e6a0f8d109f4ed9 in_seq=0 out_seq=0 2024-09-21T06:29:40.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.746+0000 7f87f0c00640 10 client.5301 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f87f0c00640 1 -- 172.21.15.37:0/3534914436 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5609144c4370 con 0x5609145938f0 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f87f0c00640 10 client.5301 handle_command_reply: tid=0 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 client.5301 shutdown 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 msgr2=0x5609145bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f87fc016000 tx=0x7f87fc014cb0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 shutdown_connections 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5609145938f0 0x5609145ba860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609141ac5b0 0x5609145bc0e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56091418b8d0 0x5609145bbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f87f0c00640 0 client.5301 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 --2- 172.21.15.37:0/3534914436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609141836f0 0x5609145abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.747+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 >> 172.21.15.37:0/3534914436 conn(0x5609141a0700 msgr2=0x5609145944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.748+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 shutdown_connections 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.748+0000 7f8811455740 1 -- 172.21.15.37:0/3534914436 wait complete. 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.748+0000 7f8811455740 20 client.5301 trim_cache size 0 max 16384 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "tag123", 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-21T06:29:40.751 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:40.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 msgr2=0x7f87e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 0x7f87e007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87f4034000 tx=0x7f87f4004730 comp rx=0 tx=0).stop 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 msgr2=0x7f88041f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c3a0 tx=0x7f87ec00c860 comp rx=0 tx=0).stop 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 shutdown_connections 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f87e0078bc0 0x7f87e007b0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88040b49b0 0x7f88041f8820 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415f720 0x7f88040af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 --2- 172.21.15.37:0/128250235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88040732a0 0x7f88041eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:40.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 >> 172.21.15.37:0/128250235 conn(0x7f88040d0460 msgr2=0x7f880415efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:40.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.749+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 shutdown_connections 2024-09-21T06:29:40.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:40.750+0000 7f8808e00640 1 -- 172.21.15.37:0/128250235 wait complete. 2024-09-21T06:29:40.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:29:40.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 -- 172.21.15.37:0/2001650420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2440ccef0 msgr2=0x7ff24406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 --2- 172.21.15.37:0/2001650420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2440ccef0 0x7ff24406cf90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff2440726b0 tx=0x7ff23c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 -- 172.21.15.37:0/2001650420 shutdown_connections 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 --2- 172.21.15.37:0/2001650420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2440ccef0 0x7ff24406cf90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 --2- 172.21.15.37:0/2001650420 >> v2:172.21.15.150:3300/0 conn(0x7ff24406e2c0 0x7ff2440cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 -- 172.21.15.37:0/2001650420 >> 172.21.15.37:0/2001650420 conn(0x7ff24406f920 msgr2=0x7ff2440b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 -- 172.21.15.37:0/2001650420 shutdown_connections 2024-09-21T06:29:41.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.005+0000 7ff248a00640 1 -- 172.21.15.37:0/2001650420 wait complete. 2024-09-21T06:29:41.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.007+0000 7ff248a00640 1 Processor -- start 2024-09-21T06:29:41.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.008+0000 7ff248a00640 1 -- start start 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff24406e2c0 0x7ff2441e7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2440b0fa0 0x7ff2440b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff24406d4d0 con 0x7ff244160a00 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff24406d650 con 0x7ff2440b0fa0 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff248a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff24406d7d0 con 0x7ff24406e2c0 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.009+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44794/0 (socket says 172.21.15.37:44794) 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 -- 172.21.15.37:0/4278413742 learned_addr learned my addr 172.21.15.37:0/4278413742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff242800640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff24406e2c0 0x7ff2441e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff243200640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2440b0fa0 0x7ff2440b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 -- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff24406e2c0 msgr2=0x7ff2441e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff24406e2c0 0x7ff2441e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 -- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2440b0fa0 msgr2=0x7ff2440b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2440b0fa0 0x7ff2440b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 -- 172.21.15.37:0/4278413742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2441e8000 con 0x7ff244160a00 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff239e00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c3f0 tx=0x7ff22c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.010+0000 7ff23b400640 1 -- 172.21.15.37:0/4278413742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c017070 con 0x7ff244160a00 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.011+0000 7ff23b400640 1 -- 172.21.15.37:0/4278413742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c0035f0 con 0x7ff244160a00 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.011+0000 7ff23b400640 1 -- 172.21.15.37:0/4278413742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c003a40 con 0x7ff244160a00 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.011+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2440b18f0 con 0x7ff244160a00 2024-09-21T06:29:41.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.011+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441f6e20 con 0x7ff244160a00 2024-09-21T06:29:41.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.013+0000 7ff23b400640 1 -- 172.21.15.37:0/4278413742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff22c01a070 con 0x7ff244160a00 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.014+0000 7ff23b400640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 0x7ff22007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.014+0000 7ff23b400640 1 -- 172.21.15.37:0/4278413742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff22c09a250 con 0x7ff244160a00 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.015+0000 7ff2505db740 1 Processor -- start 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.015+0000 7ff242800640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 0x7ff22007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.015+0000 7ff2505db740 1 -- start start 2024-09-21T06:29:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.015+0000 7ff242800640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 0x7ff22007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff234009b30 tx=0x7ff234005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b456b0 0x563e28f3bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563e28e6d1c0 con 0x563e28b3d4d0 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e28e80870 con 0x563e28b456b0 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563e28e6fca0 con 0x563e28b66390 2024-09-21T06:29:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44802/0 (socket says 172.21.15.37:44802) 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff242800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff239e00640 1 -- 172.21.15.37:0/1229789719 learned_addr learned my addr 172.21.15.37:0/1229789719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.016+0000 7ff243200640 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b456b0 0x563e28f3bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 -- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 msgr2=0x563e28f58b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f58b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 -- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b456b0 msgr2=0x563e28f3bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b456b0 0x563e28f3bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 -- 172.21.15.37:0/1229789719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e28c487d0 con 0x563e28b3d4d0 2024-09-21T06:29:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff239e00640 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0617f0 tx=0x7ff22c0618e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912f51d92e921431 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff233000640 1 -- 172.21.15.37:0/1229789719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0668a0 con 0x563e28b3d4d0 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff233000640 1 -- 172.21.15.37:0/1229789719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c06b750 con 0x563e28b3d4d0 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.017+0000 7ff233000640 1 -- 172.21.15.37:0/1229789719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c06b520 con 0x563e28b3d4d0 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 -- 172.21.15.37:0/1229789719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 msgr2=0x563e28f3b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0617f0 tx=0x7ff22c0618e0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 -- 172.21.15.37:0/1229789719 shutdown_connections 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28f3b810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b456b0 0x563e28f3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 --2- 172.21.15.37:0/1229789719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 -- 172.21.15.37:0/1229789719 >> 172.21.15.37:0/1229789719 conn(0x563e28b5a4e0 msgr2=0x563e28f11a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.018+0000 7ff2505db740 1 -- 172.21.15.37:0/1229789719 shutdown_connections 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.019+0000 7ff2505db740 1 -- 172.21.15.37:0/1229789719 wait complete. 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.019+0000 7ff2505db740 1 -- start start 2024-09-21T06:29:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.020+0000 7ff2505db740 1 Processor -- start 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f68ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563e28e16410 con 0x563e28b456b0 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e28e73ab0 con 0x563e28b3d4d0 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563e28be0740 con 0x563e28b66390 2024-09-21T06:29:41.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff2505db740 10 client.0 _ll_register_callbacks cb 0x563e28df3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.021+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44816/0 (socket says 172.21.15.37:44816) 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 -- 172.21.15.37:0/73583942 learned_addr learned my addr 172.21.15.37:0/73583942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff242800640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f68ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff243200640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 -- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 msgr2=0x563e28f57060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f57060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 -- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 msgr2=0x563e28f68ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f68ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 -- 172.21.15.37:0/73583942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e28bd1970 con 0x563e28b456b0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff239e00640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c3c0 tx=0x7ff22c06b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a2d9888e65675a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff227400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:41.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0668a0 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff22c0610d0 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.022+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22c06b750 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff22c070050 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 10 client.5316 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563e28baf6a0 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c06b750 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7ff22c05d7c0 con 0x563e28b456b0 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 10 client.5316 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 10 client.5316 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 20 client.5316 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28b24f60 0x563e28f11100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 4 client.5316 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:29:41.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e28e827c0 con 0x563e28b24f60 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.023+0000 7ff242800640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28b24f60 0x563e28f11100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.024+0000 7ff242800640 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28b24f60 0x563e28f11100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d5822de9589306 server_cookie=c147216a44ac3d62 in_seq=0 out_seq=0 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.024+0000 7ff227400640 10 client.5316 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff227400640 1 -- 172.21.15.37:0/73583942 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e28e827c0 con 0x563e28b24f60 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff227400640 10 client.5316 handle_command_reply: tid=0 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 client.5316 shutdown 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 msgr2=0x563e28f66170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c3c0 tx=0x7ff22c06b4f0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 shutdown_connections 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28b24f60 0x563e28f11100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f66170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f57060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff227400640 0 client.5316 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:41.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 --2- 172.21.15.37:0/73583942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f68ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 >> 172.21.15.37:0/73583942 conn(0x563e28b5a4e0 msgr2=0x563e28f3b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 shutdown_connections 2024-09-21T06:29:41.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.025+0000 7ff2505db740 1 -- 172.21.15.37:0/73583942 wait complete. 2024-09-21T06:29:41.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.026+0000 7ff2505db740 20 client.5316 trim_cache size 0 max 16384 2024-09-21T06:29:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.044+0000 7ff2505db740 1 Processor -- start 2024-09-21T06:29:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.045+0000 7ff2505db740 1 -- start start 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441fe020 con 0x563e28b456b0 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2440741b0 con 0x563e28b3d4d0 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff244074870 con 0x563e28b66390 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff239e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44830/0 (socket says 172.21.15.37:44830) 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff239e00640 1 -- 172.21.15.37:0/816948847 learned_addr learned my addr 172.21.15.37:0/816948847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff242800640 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.046+0000 7ff243200640 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 -- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 msgr2=0x563e28f6fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f6fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 -- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 msgr2=0x563e28f7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 -- 172.21.15.37:0/816948847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e28f0e590 con 0x563e28b456b0 2024-09-21T06:29:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff239e00640 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cdb0 tx=0x7ff22c061ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171d2cb21c4e5ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff232600640 1 -- 172.21.15.37:0/816948847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c06b750 con 0x563e28b456b0 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff232600640 1 -- 172.21.15.37:0/816948847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c07b030 con 0x563e28b456b0 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.047+0000 7ff232600640 1 -- 172.21.15.37:0/816948847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0668a0 con 0x563e28b456b0 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 -- 172.21.15.37:0/816948847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 msgr2=0x563e28f7acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cdb0 tx=0x7ff22c061ae0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 -- 172.21.15.37:0/816948847 shutdown_connections 2024-09-21T06:29:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b456b0 0x563e28f7acb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b66390 0x563e28f6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 --2- 172.21.15.37:0/816948847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b3d4d0 0x563e28f7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 -- 172.21.15.37:0/816948847 >> 172.21.15.37:0/816948847 conn(0x563e28b5a4e0 msgr2=0x563e28f0c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 -- 172.21.15.37:0/816948847 shutdown_connections 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.048+0000 7ff2505db740 1 -- 172.21.15.37:0/816948847 wait complete. 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.049+0000 7ff2505db740 1 -- start start 2024-09-21T06:29:41.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.049+0000 7ff2505db740 1 Processor -- start 2024-09-21T06:29:41.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b456b0 0x563e28f35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b66390 0x563e28fd53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff24414f320 con 0x563e28b3d4d0 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e28fd5fc0 con 0x563e28b66390 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563e28f65370 con 0x563e28b456b0 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff2505db740 10 client.0 _ll_register_callbacks cb 0x7ff23c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44846/0 (socket says 172.21.15.37:44846) 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 -- 172.21.15.37:0/2621355920 learned_addr learned my addr 172.21.15.37:0/2621355920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 -- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b456b0 msgr2=0x563e28f35060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b456b0 0x563e28f35060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 -- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b66390 msgr2=0x563e28fd53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b66390 0x563e28fd53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.051+0000 7ff242800640 1 -- 172.21.15.37:0/2621355920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e28f665b0 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff242800640 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x563e28f66950 tx=0x7ff2340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bb8c9e972e777b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234034020 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff234003b80 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22c07b030 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff23403e050 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff2505db740 10 client.5322 fetch_fsmap learned FSMap version 24 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563e28f355a0 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c07b030 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.052+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7ff234041020 con 0x563e28b3d4d0 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 10 client.5322 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T06:29:41.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 10 client.5322 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 20 client.5322 populate_metadata read hostname 'smithi037' 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28f4ddd0 0x563e28f4fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 4 client.5322 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e28e7e270 con 0x563e28f4ddd0 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.053+0000 7ff239e00640 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28f4ddd0 0x563e28f4fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff239e00640 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28f4ddd0 0x563e28f4fd60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d35db9e212c306 server_cookie=78259cd8fa242536 in_seq=0 out_seq=0 2024-09-21T06:29:41.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff227400640 10 client.5322 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff227400640 1 -- 172.21.15.37:0/2621355920 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+496 (crc 0 0 0) 0x563e28e7e270 con 0x563e28f4ddd0 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff227400640 10 client.5322 handle_command_reply: tid=0 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 client.5322 shutdown 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 msgr2=0x563e28fcbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x563e28f66950 tx=0x7ff2340046c0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 shutdown_connections 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x563e28f4ddd0 0x563e28f4fd60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e28b66390 0x563e28fd53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563e28b456b0 0x563e28f35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 --2- 172.21.15.37:0/2621355920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563e28b3d4d0 0x563e28fcbe90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 >> 172.21.15.37:0/2621355920 conn(0x563e28b5a4e0 msgr2=0x563e28f37950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff227400640 0 client.5322 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.054+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 shutdown_connections 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff2505db740 1 -- 172.21.15.37:0/2621355920 wait complete. 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff2505db740 20 client.5322 trim_cache size 0 max 16384 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:29:41.058 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tag123": { 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "tag123", 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tag123": { 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:29:40.746950+0000", 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:29:41.059 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:29:41.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:41.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:29:41.060 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:29:41.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 msgr2=0x7ff22007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 0x7ff22007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff234009b30 tx=0x7ff234005f90 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 msgr2=0x7ff244141970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c3f0 tx=0x7ff22c00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 shutdown_connections 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff220078b90 0x7ff22007b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244160a00 0x7ff244141970 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2440b0fa0 0x7ff2440b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 --2- 172.21.15.37:0/4278413742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff24406e2c0 0x7ff2441e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 >> 172.21.15.37:0/4278413742 conn(0x7ff24406f920 msgr2=0x7ff244156670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 shutdown_connections 2024-09-21T06:29:41.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:29:41.055+0000 7ff248a00640 1 -- 172.21.15.37:0/4278413742 wait complete. 2024-09-21T06:29:41.081 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2024-09-21T06:29:49.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:29:49.587 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:30:11.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:11.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-21T06:30:11.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.322+0000 7f6e6aa00640 1 -- 172.21.15.37:0/1559967131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640af5b0 msgr2=0x7f6e6406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/1559967131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640af5b0 0x7f6e6406ddf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e64072a30 tx=0x7f6e5002a560 comp rx=0 tx=0).stop 2024-09-21T06:30:11.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 -- 172.21.15.37:0/1559967131 shutdown_connections 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/1559967131 >> v2:172.21.15.150:3300/0 conn(0x7f6e6406e330 0x7f6e640ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/1559967131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640af5b0 0x7f6e6406ddf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/1559967131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640cc530 0x7f6e6415ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 -- 172.21.15.37:0/1559967131 >> 172.21.15.37:0/1559967131 conn(0x7f6e640b1b90 msgr2=0x7f6e640b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 -- 172.21.15.37:0/1559967131 shutdown_connections 2024-09-21T06:30:11.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.323+0000 7f6e6aa00640 1 -- 172.21.15.37:0/1559967131 wait complete. 2024-09-21T06:30:11.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.325+0000 7f6e6aa00640 1 Processor -- start 2024-09-21T06:30:11.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.326+0000 7f6e6aa00640 1 -- start start 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6406e330 0x7f6e64159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640af5b0 0x7f6e6415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640cadb0 con 0x7f6e640cc530 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640cac30 con 0x7f6e6406e330 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6e640caf30 con 0x7f6e640af5b0 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58388/0 (socket says 172.21.15.37:58388) 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e68a00640 1 -- 172.21.15.37:0/2096411919 learned_addr learned my addr 172.21.15.37:0/2096411919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6406e330 0x7f6e64159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.327+0000 7f6e63400640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640af5b0 0x7f6e6415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 -- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640af5b0 msgr2=0x7f6e6415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640af5b0 0x7f6e6415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 -- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6406e330 msgr2=0x7f6e64159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6406e330 0x7f6e64159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 -- 172.21.15.37:0/2096411919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e641f5530 con 0x7f6e640cc530 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e8e0 tx=0x7f6e5c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.328+0000 7f6e5be00640 1 -- 172.21.15.37:0/2096411919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c004d20 con 0x7f6e640cc530 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.329+0000 7f6e5be00640 1 -- 172.21.15.37:0/2096411919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5c00ca60 con 0x7f6e640cc530 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.329+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e641f5810 con 0x7f6e640cc530 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.329+0000 7f6e5be00640 1 -- 172.21.15.37:0/2096411919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c005020 con 0x7f6e640cc530 2024-09-21T06:30:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.329+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e641f5dd0 con 0x7f6e640cc530 2024-09-21T06:30:11.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.332+0000 7f6e5be00640 1 -- 172.21.15.37:0/2096411919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e5c0051c0 con 0x7f6e640cc530 2024-09-21T06:30:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.333+0000 7f6e5be00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 0x7f6e4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.334+0000 7f6e5be00640 1 -- 172.21.15.37:0/2096411919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6e5c09b860 con 0x7f6e640cc530 2024-09-21T06:30:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.334+0000 7f6e71a37740 1 Processor -- start 2024-09-21T06:30:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.334+0000 7f6e71a37740 1 -- start start 2024-09-21T06:30:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.334+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 0x7f6e4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 0x7f6e4407b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e54005fd0 tx=0x7f6e5400a250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d07220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561095c3d1c0 con 0x561095936390 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561095c50870 con 0x56109590d4d0 2024-09-21T06:30:11.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561095c3fca0 con 0x5610959156b0 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.335+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58398/0 (socket says 172.21.15.37:58398) 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 -- 172.21.15.37:0/435799795 learned_addr learned my addr 172.21.15.37:0/435799795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e68a00640 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63400640 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d07220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 -- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 msgr2=0x561095d07760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d07760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 -- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 msgr2=0x561095d07220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d07220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 -- 172.21.15.37:0/435799795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561095a187d0 con 0x561095936390 2024-09-21T06:30:11.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.336+0000 7f6e63e00640 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ff80 tx=0x7f6e54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d18ef7bef7e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.337+0000 7f6e4fe00640 1 -- 172.21.15.37:0/435799795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54016070 con 0x561095936390 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.337+0000 7f6e4fe00640 1 -- 172.21.15.37:0/435799795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e54025630 con 0x561095936390 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.337+0000 7f6e4fe00640 1 -- 172.21.15.37:0/435799795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54025a40 con 0x561095936390 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 -- 172.21.15.37:0/435799795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 msgr2=0x561095d24550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ff80 tx=0x7f6e54012000 comp rx=0 tx=0).stop 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 -- 172.21.15.37:0/435799795 shutdown_connections 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d07760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d07220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 --2- 172.21.15.37:0/435799795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d24550 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 -- 172.21.15.37:0/435799795 >> 172.21.15.37:0/435799795 conn(0x56109592a4e0 msgr2=0x561095cdd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 -- 172.21.15.37:0/435799795 shutdown_connections 2024-09-21T06:30:11.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.338+0000 7f6e71a37740 1 -- 172.21.15.37:0/435799795 wait complete. 2024-09-21T06:30:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.339+0000 7f6e71a37740 1 -- start start 2024-09-21T06:30:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.339+0000 7f6e71a37740 1 Processor -- start 2024-09-21T06:30:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d00e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561095be6410 con 0x561095936390 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561095c43ab0 con 0x56109590d4d0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5610959b0740 con 0x5610959156b0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e71a37740 10 client.0 _ll_register_callbacks cb 0x561095bc3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58410/0 (socket says 172.21.15.37:58410) 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e68a00640 1 -- 172.21.15.37:0/4925874 learned_addr learned my addr 172.21.15.37:0/4925874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e63400640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.341+0000 7f6e63e00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d00e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 -- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 msgr2=0x561095d1acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d1acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 -- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 msgr2=0x561095d00e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d00e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 -- 172.21.15.37:0/4925874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610959a1970 con 0x561095936390 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e68a00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c099aa0 tx=0x7f6e5c063b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffdc0f3f917fb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c01f040 con 0x561095936390 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6e5c04e9c0 con 0x561095936390 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e54025630 con 0x561095936390 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e5c063500 con 0x561095936390 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e71a37740 10 client.5334 fetch_fsmap learned FSMap version 25 2024-09-21T06:30:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.342+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x56109597f6a0 con 0x561095936390 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e54025630 con 0x561095936390 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 3558+0+0 (secure 0 0 0) 0x7f6e5c00f040 con 0x561095936390 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 10 client.5334 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 10 client.5334 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 20 client.5334 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5610958f4f60 0x561095d34700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 4 client.5334 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561095c527c0 con 0x5610958f4f60 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.343+0000 7f6e63e00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5610958f4f60 0x561095d34700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.344+0000 7f6e63e00640 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5610958f4f60 0x561095d34700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1497127d44df86 server_cookie=28f68db801cacc81 in_seq=0 out_seq=0 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.344+0000 7f6e4cc00640 10 client.5334 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e4cc00640 1 -- 172.21.15.37:0/4925874 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561095c527c0 con 0x5610958f4f60 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e4cc00640 10 client.5334 handle_command_reply: tid=0 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 client.5334 shutdown 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 msgr2=0x561095d1b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c099aa0 tx=0x7f6e5c063b20 comp rx=0 tx=0).stop 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 shutdown_connections 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x5610958f4f60 0x561095d34700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d1b210 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5610959156b0 0x561095d1acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e4cc00640 0 client.5334 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 --2- 172.21.15.37:0/4925874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56109590d4d0 0x561095d00e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.345+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 >> 172.21.15.37:0/4925874 conn(0x56109592a4e0 msgr2=0x561095cf69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.346+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 shutdown_connections 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.346+0000 7f6e71a37740 1 -- 172.21.15.37:0/4925874 wait complete. 2024-09-21T06:30:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.346+0000 7f6e71a37740 20 client.5334 trim_cache size 0 max 16384 2024-09-21T06:30:11.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.365+0000 7f6e71a37740 1 Processor -- start 2024-09-21T06:30:11.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.365+0000 7f6e71a37740 1 -- start start 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56109590d4d0 0x561095d02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d07220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64202770 con 0x561095936390 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64201910 con 0x5610959156b0 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6e64202040 con 0x56109590d4d0 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.366+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58424/0 (socket says 172.21.15.37:58424) 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 -- 172.21.15.37:0/185149626 learned_addr learned my addr 172.21.15.37:0/185149626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e63e00640 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56109590d4d0 0x561095d02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e63400640 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d07220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 -- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56109590d4d0 msgr2=0x561095d02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56109590d4d0 0x561095d02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 -- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 msgr2=0x561095d07220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d07220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 -- 172.21.15.37:0/185149626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561095cd8b40 con 0x561095936390 2024-09-21T06:30:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e68a00640 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c060770 tx=0x7f6e5c004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9b9f193b75ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e4f400640 1 -- 172.21.15.37:0/185149626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c01f020 con 0x561095936390 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e4f400640 1 -- 172.21.15.37:0/185149626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5c085c30 con 0x561095936390 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.367+0000 7f6e4f400640 1 -- 172.21.15.37:0/185149626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c05ee50 con 0x561095936390 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 -- 172.21.15.37:0/185149626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 msgr2=0x561095d07760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c060770 tx=0x7f6e5c004fb0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 -- 172.21.15.37:0/185149626 shutdown_connections 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561095936390 0x561095d07760 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d07220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 --2- 172.21.15.37:0/185149626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56109590d4d0 0x561095d02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 -- 172.21.15.37:0/185149626 >> 172.21.15.37:0/185149626 conn(0x56109592a4e0 msgr2=0x561095cdd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 -- 172.21.15.37:0/185149626 shutdown_connections 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.368+0000 7f6e71a37740 1 -- 172.21.15.37:0/185149626 wait complete. 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.369+0000 7f6e71a37740 1 -- start start 2024-09-21T06:30:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.369+0000 7f6e71a37740 1 Processor -- start 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d43410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561095936390 0x561095cd6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561095cf7850 con 0x56109590d4d0 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6409ed00 con 0x5610959156b0 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561095d43c00 con 0x561095936390 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e71a37740 10 client.0 _ll_register_callbacks cb 0x7f6e50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58426/0 (socket says 172.21.15.37:58426) 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.371+0000 7f6e63e00640 1 -- 172.21.15.37:0/2115595051 learned_addr learned my addr 172.21.15.37:0/2115595051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561095936390 0x561095cd6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63400640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d43410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 -- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561095936390 msgr2=0x561095cd6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561095936390 0x561095cd6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 -- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 msgr2=0x561095d43410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d43410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 -- 172.21.15.37:0/2115595051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561095cdb6c0 con 0x56109590d4d0 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e63e00640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e54010c90 tx=0x7f6e5400a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bbdcffb4664191 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5403d060 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6e54025530 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.372+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5c085c30 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e54043050 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 10 client.5340 fetch_fsmap learned FSMap version 25 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x561095cdbea0 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5c085c30 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 3558+0+0 (secure 0 0 0) 0x7f6e54046020 con 0x56109590d4d0 2024-09-21T06:30:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 10 client.5340 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 10 client.5340 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 20 client.5340 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x561095d30a50 0x561095d1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 4 client.5340 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.373+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 --> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561095c4e270 con 0x561095d30a50 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.374+0000 7f6e63400640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x561095d30a50 0x561095d1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e63400640 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x561095d30a50 0x561095d1ab40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970aa79e9cdcf5ca server_cookie=2f2e8813731603d5 in_seq=0 out_seq=0 2024-09-21T06:30:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e4cc00640 10 client.5340 ms_handle_connect on v2:172.21.15.150:6835/337579449 2024-09-21T06:30:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e4cc00640 1 -- 172.21.15.37:0/2115595051 <== mds.0 v2:172.21.15.150:6835/337579449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+352 (crc 0 0 0) 0x561095c4e270 con 0x561095d30a50 2024-09-21T06:30:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e4cc00640 10 client.5340 handle_command_reply: tid=0 2024-09-21T06:30:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e71a37740 1 client.5340 shutdown 2024-09-21T06:30:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 msgr2=0x561095da0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.375+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e54010c90 tx=0x7f6e5400a8c0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 shutdown_connections 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:6835/337579449,v1:172.21.15.150:6837/337579449] conn(0x561095d30a50 0x561095d1ab40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561095936390 0x561095cd6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5610959156b0 0x561095d43410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e4cc00640 0 client.5340 ms_handle_reset on v2:172.21.15.150:6835/337579449 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 --2- 172.21.15.37:0/2115595051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56109590d4d0 0x561095da0620 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 >> 172.21.15.37:0/2115595051 conn(0x56109592a4e0 msgr2=0x561095cf6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 shutdown_connections 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 1 -- 172.21.15.37:0/2115595051 wait complete. 2024-09-21T06:30:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e71a37740 20 client.5340 trim_cache size 0 max 16384 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {}, 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tag123": { 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:29:40.746950+0000", 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:30:11.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:11.382 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:11.382 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 msgr2=0x7f6e4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 0x7f6e4407b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e54005fd0 tx=0x7f6e5400a250 comp rx=0 tx=0).stop 2024-09-21T06:30:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 msgr2=0x7f6e6415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.376+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e8e0 tx=0x7f6e5c00eda0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 shutdown_connections 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e44078b90 0x7f6e4407b0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e640cc530 0x7f6e6415b290 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e640af5b0 0x7f6e6415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 --2- 172.21.15.37:0/2096411919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6406e330 0x7f6e64159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 >> 172.21.15.37:0/2096411919 conn(0x7f6e640b1b90 msgr2=0x7f6e640afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 shutdown_connections 2024-09-21T06:30:11.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.377+0000 7f6e6aa00640 1 -- 172.21.15.37:0/2096411919 wait complete. 2024-09-21T06:30:11.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:30:11.404 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2024-09-21T06:30:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.546+0000 7f6dade0ff40 1 Processor -- start 2024-09-21T06:30:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.546+0000 7f6dade0ff40 1 -- start start 2024-09-21T06:30:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98a2470 0x55b6b98cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98ea630 0x55b6b9a02410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6b98ea110 con 0x55b6b98e75a0 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6b97eec60 con 0x55b6b98a2470 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b6b98cdbc0 con 0x55b6b98ea630 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.547+0000 7f6da5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58434/0 (socket says 172.21.15.37:58434) 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 -- 172.21.15.37:0/3843378013 learned_addr learned my addr 172.21.15.37:0/3843378013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da6400640 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98a2470 0x55b6b98cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da6e00640 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98ea630 0x55b6b9a02410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 -- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98ea630 msgr2=0x55b6b9a02410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98ea630 0x55b6b9a02410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 -- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98a2470 msgr2=0x55b6b98cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98a2470 0x55b6b98cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 -- 172.21.15.37:0/3843378013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b97c6f10 con 0x55b6b98e75a0 2024-09-21T06:30:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c520 tx=0x7f6d9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aef6dd22e6a8ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.548+0000 7f6d9fe00640 1 -- 172.21.15.37:0/3843378013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400a960 con 0x55b6b98e75a0 2024-09-21T06:30:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6d9fe00640 1 -- 172.21.15.37:0/3843378013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d9400ab00 con 0x55b6b98e75a0 2024-09-21T06:30:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6d9fe00640 1 -- 172.21.15.37:0/3843378013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94015440 con 0x55b6b98e75a0 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3843378013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 msgr2=0x55b6b9a01ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c520 tx=0x7f6d9400c920 comp rx=0 tx=0).stop 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3843378013 shutdown_connections 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9a01ed0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98ea630 0x55b6b9a02410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3843378013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98a2470 0x55b6b98cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3843378013 >> 172.21.15.37:0/3843378013 conn(0x55b6b97bfac0 msgr2=0x55b6b98ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3843378013 shutdown_connections 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.549+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3843378013 wait complete. 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.550+0000 7f6dade0ff40 1 Processor -- start 2024-09-21T06:30:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.551+0000 7f6dade0ff40 1 -- start start 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98a2470 0x55b6b9957e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9973010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98ea630 0x55b6b98ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9a02e40 con 0x55b6b98e75a0 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6b989e870 con 0x55b6b98ea630 2024-09-21T06:30:11.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6dade0ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b6b995ab30 con 0x55b6b98a2470 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6da5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9973010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6da5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9973010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58448/0 (socket says 172.21.15.37:58448) 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6da5a00640 1 -- 172.21.15.37:0/3882114254 learned_addr learned my addr 172.21.15.37:0/3882114254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6da6e00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98ea630 0x55b6b98ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.552+0000 7f6da6400640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98a2470 0x55b6b9957e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 -- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98a2470 msgr2=0x55b6b9957e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98a2470 0x55b6b9957e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 -- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98ea630 msgr2=0x55b6b98ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98ea630 0x55b6b98ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 -- 172.21.15.37:0/3882114254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b99fe700 con 0x55b6b98e75a0 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da6e00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6b98ea630 0x55b6b98ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da6400640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6b98a2470 0x55b6b9957e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6da5a00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9973010 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c470 tx=0x7f6d940156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6d9ea00640 1 -- 172.21.15.37:0/3882114254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94024070 con 0x55b6b98e75a0 2024-09-21T06:30:11.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6d9ea00640 1 -- 172.21.15.37:0/3882114254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d94015de0 con 0x55b6b98e75a0 2024-09-21T06:30:11.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6d9ea00640 1 -- 172.21.15.37:0/3882114254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9401f6b0 con 0x55b6b98e75a0 2024-09-21T06:30:11.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b6b9958360 con 0x55b6b98e75a0 2024-09-21T06:30:11.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.553+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b9a01020 con 0x55b6b98e75a0 2024-09-21T06:30:11.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.556+0000 7f6d9ea00640 1 -- 172.21.15.37:0/3882114254 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d94027070 con 0x55b6b98e75a0 2024-09-21T06:30:11.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.556+0000 7f6d9ea00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6d880775a0 0x7f6d88079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.557+0000 7f6d9ea00640 1 -- 172.21.15.37:0/3882114254 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6d9409b760 con 0x55b6b98e75a0 2024-09-21T06:30:11.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.557+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55b6b9a11270 0x55b6b9a11680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.557+0000 7f6da6400640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6d880775a0 0x7f6d88079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.557+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55b6b9a19f40 con 0x55b6b9a11270 2024-09-21T06:30:11.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.557+0000 7f6da6e00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55b6b9a11270 0x55b6b9a11680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.558+0000 7f6da6400640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6d880775a0 0x7f6d88079ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d98055000 tx=0x7f6d98003020 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.558+0000 7f6da6e00640 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55b6b9a11270 0x55b6b9a11680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.558+0000 7f6da6e00640 1 -- 172.21.15.37:0/3882114254 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f6da000b040 con 0x55b6b9a11270 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55b6b9a11270 msgr2=0x55b6b9a11680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55b6b9a11270 0x55b6b9a11680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6d880775a0 msgr2=0x7f6d88079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6d880775a0 0x7f6d88079ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d98055000 tx=0x7f6d98003020 comp rx=0 tx=0).stop 2024-09-21T06:30:11.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 msgr2=0x55b6b9973010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 --2- 172.21.15.37:0/3882114254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6b98e75a0 0x55b6b9973010 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c470 tx=0x7f6d940156f0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6da6e00640 1 -- 172.21.15.37:0/3882114254 reap_dead start 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 shutdown_connections 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 >> 172.21.15.37:0/3882114254 conn(0x55b6b97bfac0 msgr2=0x55b6b989f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 shutdown_connections 2024-09-21T06:30:11.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.559+0000 7f6dade0ff40 1 -- 172.21.15.37:0/3882114254 wait complete. 2024-09-21T06:30:11.574 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:11.574 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2024-09-21T06:30:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.716+0000 7fc321d1ff40 1 Processor -- start 2024-09-21T06:30:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.716+0000 7fc321d1ff40 1 -- start start 2024-09-21T06:30:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772e04630 0x563772dfcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563772d08c60 con 0x563772dbc9b0 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563772d671d0 con 0x563772e04630 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563772e04110 con 0x563772e775c0 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc31a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc31a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58466/0 (socket says 172.21.15.37:58466) 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc31a400640 1 -- 172.21.15.37:0/809262411 learned_addr learned my addr 172.21.15.37:0/809262411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc319a00640 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772e04630 0x563772dfcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.718+0000 7fc31ae00640 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 -- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 msgr2=0x563772de91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 -- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772e04630 msgr2=0x563772dfcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772e04630 0x563772dfcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 -- 172.21.15.37:0/809262411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563772de5a20 con 0x563772dbc9b0 2024-09-21T06:30:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc31a400640 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc30c001d20 tx=0x7fc30c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72605d045f75ee5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc313e00640 1 -- 172.21.15.37:0/809262411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00bd80 con 0x563772dbc9b0 2024-09-21T06:30:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc313e00640 1 -- 172.21.15.37:0/809262411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc30c00f400 con 0x563772dbc9b0 2024-09-21T06:30:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.719+0000 7fc313e00640 1 -- 172.21.15.37:0/809262411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00f850 con 0x563772dbc9b0 2024-09-21T06:30:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 -- 172.21.15.37:0/809262411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 msgr2=0x563772de54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc30c001d20 tx=0x7fc30c00b5b0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 -- 172.21.15.37:0/809262411 shutdown_connections 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772e04630 0x563772dfcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/809262411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772dbc9b0 0x563772de54e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 -- 172.21.15.37:0/809262411 >> 172.21.15.37:0/809262411 conn(0x563772cd9ac0 msgr2=0x563772de6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 -- 172.21.15.37:0/809262411 shutdown_connections 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.720+0000 7fc321d1ff40 1 -- 172.21.15.37:0/809262411 wait complete. 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.721+0000 7fc321d1ff40 1 Processor -- start 2024-09-21T06:30:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.721+0000 7fc321d1ff40 1 -- start start 2024-09-21T06:30:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772dbc9b0 0x563772e00ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 0x563772e8d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563772e8f030 con 0x563772e04630 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563772e75470 con 0x563772dbc9b0 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc321d1ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563772de4fe0 con 0x563772e775c0 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 0x563772e8d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 0x563772e8d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58470/0 (socket says 172.21.15.37:58470) 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 -- 172.21.15.37:0/2155078242 learned_addr learned my addr 172.21.15.37:0/2155078242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc31a400640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772dbc9b0 0x563772e00ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc31ae00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 -- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 msgr2=0x563772de91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563772e775c0 0x563772de91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 -- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772dbc9b0 msgr2=0x563772e00ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563772dbc9b0 0x563772e00ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.723+0000 7fc319a00640 1 -- 172.21.15.37:0/2155078242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563772de3620 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc319a00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 0x563772e8d4d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc308025f00 tx=0x7fc308004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc312a00640 1 -- 172.21.15.37:0/2155078242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308068070 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc312a00640 1 -- 172.21.15.37:0/2155078242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc308058280 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc312a00640 1 -- 172.21.15.37:0/2155078242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308058870 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563772de3340 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.724+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563772e8da10 con 0x563772e04630 2024-09-21T06:30:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.726+0000 7fc312a00640 1 -- 172.21.15.37:0/2155078242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc30806b070 con 0x563772e04630 2024-09-21T06:30:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.727+0000 7fc312a00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc2fc0775a0 0x7fc2fc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.728+0000 7fc312a00640 1 -- 172.21.15.37:0/2155078242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fc3080dda00 con 0x563772e04630 2024-09-21T06:30:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.728+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x563772de7ec0 0x563772f31d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.728+0000 7fc31a400640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc2fc0775a0 0x7fc2fc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.728+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 5.12 5:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x563772de3e90 con 0x563772de7ec0 2024-09-21T06:30:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.728+0000 7fc31ae00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x563772de7ec0 0x563772f31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.729+0000 7fc31a400640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc2fc0775a0 0x7fc2fc079ab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0012b0 tx=0x7fc30c000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.729+0000 7fc31ae00640 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x563772de7ec0 0x563772f31d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc31ae00640 1 -- 172.21.15.37:0/2155078242 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fc314003040 con 0x563772de7ec0 2024-09-21T06:30:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x563772de7ec0 msgr2=0x563772f31d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x563772de7ec0 0x563772f31d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc2fc0775a0 msgr2=0x7fc2fc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc2fc0775a0 0x7fc2fc079ab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0012b0 tx=0x7fc30c000ed0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 msgr2=0x563772e8d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 --2- 172.21.15.37:0/2155078242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563772e04630 0x563772e8d4d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc308025f00 tx=0x7fc308004140 comp rx=0 tx=0).stop 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc31ae00640 1 -- 172.21.15.37:0/2155078242 reap_dead start 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 shutdown_connections 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 >> 172.21.15.37:0/2155078242 conn(0x563772cd9ac0 msgr2=0x563772de6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 shutdown_connections 2024-09-21T06:30:11.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.730+0000 7fc321d1ff40 1 -- 172.21.15.37:0/2155078242 wait complete. 2024-09-21T06:30:11.743 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:11.743 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2024-09-21T06:30:11.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.885+0000 7f480ce18f40 1 Processor -- start 2024-09-21T06:30:11.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.885+0000 7f480ce18f40 1 -- start start 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x560218070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 0x5602181aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560217f91c60 con 0x5602181187a0 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560217ff01d0 con 0x7f47ec002340 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f480ce18f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560218045b80 con 0x560218113dc0 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58494/0 (socket says 172.21.15.37:58494) 2024-09-21T06:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 -- 172.21.15.37:0/1216516778 learned_addr learned my addr 172.21.15.37:0/1216516778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4805e00640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 0x5602181aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4805400640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x560218070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 -- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 msgr2=0x5602181aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 0x5602181aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 -- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 msgr2=0x560218070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x560218070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4804a00640 1 -- 172.21.15.37:0/1216516778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560218116940 con 0x5602181187a0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4805e00640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 0x5602181aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.887+0000 7f4805400640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x560218070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.888+0000 7f4804a00640 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000fd0 tx=0x7f47f4001840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf4a6146fc786c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.888+0000 7f47ff400640 1 -- 172.21.15.37:0/1216516778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4014030 con 0x5602181187a0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.888+0000 7f47ff400640 1 -- 172.21.15.37:0/1216516778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f400f6d0 con 0x5602181187a0 2024-09-21T06:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.888+0000 7f47ff400640 1 -- 172.21.15.37:0/1216516778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f400fb00 con 0x5602181187a0 2024-09-21T06:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 -- 172.21.15.37:0/1216516778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 msgr2=0x5602181a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000fd0 tx=0x7f47f4001840 comp rx=0 tx=0).stop 2024-09-21T06:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 -- 172.21.15.37:0/1216516778 shutdown_connections 2024-09-21T06:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602181187a0 0x5602181a9c40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218113dc0 0x5602181aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 --2- 172.21.15.37:0/1216516778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x560218070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 -- 172.21.15.37:0/1216516778 >> 172.21.15.37:0/1216516778 conn(0x560217f62ac0 msgr2=0x560218089db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 -- 172.21.15.37:0/1216516778 shutdown_connections 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.889+0000 7f480ce18f40 1 -- 172.21.15.37:0/1216516778 wait complete. 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.890+0000 7f480ce18f40 1 Processor -- start 2024-09-21T06:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.890+0000 7f480ce18f40 1 -- start start 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 0x5602180f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x5602181a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218118710 0x5602181a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560218045470 con 0x560218113dc0 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560218114970 con 0x7f47ec002340 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.891+0000 7f480ce18f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5602180f8ad0 con 0x560218118710 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 0x5602180f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 0x5602180f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58498/0 (socket says 172.21.15.37:58498) 2024-09-21T06:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 -- 172.21.15.37:0/3569705969 learned_addr learned my addr 172.21.15.37:0/3569705969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805e00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218118710 0x5602181a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4804a00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x5602181a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 -- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218118710 msgr2=0x5602181a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560218118710 0x5602181a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 -- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 msgr2=0x5602181a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47ec002340 0x5602181a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 -- 172.21.15.37:0/3569705969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602181136f0 con 0x560218113dc0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.892+0000 7f4805400640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 0x5602180f74a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47f80263f0 tx=0x7f47f8055d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.893+0000 7f47fe000640 1 -- 172.21.15.37:0/3569705969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8065030 con 0x560218113dc0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.893+0000 7f47fe000640 1 -- 172.21.15.37:0/3569705969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f80553f0 con 0x560218113dc0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.893+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56021810dc60 con 0x560218113dc0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.893+0000 7f47fe000640 1 -- 172.21.15.37:0/3569705969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f800b440 con 0x560218113dc0 2024-09-21T06:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.893+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602181123a0 con 0x560218113dc0 2024-09-21T06:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.895+0000 7f47fe000640 1 -- 172.21.15.37:0/3569705969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47f805e050 con 0x560218113dc0 2024-09-21T06:30:11.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.896+0000 7f47fe000640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f47e80775a0 0x7f47e8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.896+0000 7f47fe000640 1 -- 172.21.15.37:0/3569705969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f47f80de560 con 0x560218113dc0 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.896+0000 7f480ce18f40 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x5602181b4230 0x5602181b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.897+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- osd_op(unknown.0.0:1 5.5 5:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x5602181bced0 con 0x5602181b4230 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.897+0000 7f4804a00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f47e80775a0 0x7f47e8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.897+0000 7f4805e00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x5602181b4230 0x5602181b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.897+0000 7f4804a00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f47e80775a0 0x7f47e8079ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001ac0 tx=0x7f47f40092a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.897+0000 7f4805e00640 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x5602181b4230 0x5602181b4640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f4805e00640 1 -- 172.21.15.37:0/3569705969 <== osd.7 v2:172.21.15.150:6826/1641077969 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f480000b040 con 0x5602181b4230 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x5602181b4230 msgr2=0x5602181b4640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x5602181b4230 0x5602181b4640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f47e80775a0 msgr2=0x7f47e8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f47e80775a0 0x7f47e8079ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001ac0 tx=0x7f47f40092a0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 msgr2=0x5602180f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 --2- 172.21.15.37:0/3569705969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560218113dc0 0x5602180f74a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47f80263f0 tx=0x7f47f8055d80 comp rx=0 tx=0).stop 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f4805e00640 1 -- 172.21.15.37:0/3569705969 reap_dead start 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 shutdown_connections 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.898+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 >> 172.21.15.37:0/3569705969 conn(0x560217f62ac0 msgr2=0x560218118c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.899+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 shutdown_connections 2024-09-21T06:30:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:11.899+0000 7f480ce18f40 1 -- 172.21.15.37:0/3569705969 wait complete. 2024-09-21T06:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:11.912 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2024-09-21T06:30:12.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.054+0000 7fe02371ef40 1 Processor -- start 2024-09-21T06:30:12.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.054+0000 7fe02371ef40 1 -- start start 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c37c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c4476b0 0x55dd5c447ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd5c3281d0 con 0x55dd5c44b280 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dd5c334f90 con 0x55dd5c4476b0 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.055+0000 7fe02371ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dd5c2c9c60 con 0x55dd5c37d9b0 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58510/0 (socket says 172.21.15.37:58510) 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 -- 172.21.15.37:0/997951172 learned_addr learned my addr 172.21.15.37:0/997951172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01be00640 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c37c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01c800640 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c4476b0 0x55dd5c447ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 -- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 msgr2=0x55dd5c37c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c37c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 -- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c4476b0 msgr2=0x55dd5c447ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c4476b0 0x55dd5c447ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.056+0000 7fe01b400640 1 -- 172.21.15.37:0/997951172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd5c44a1a0 con 0x55dd5c44b280 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe01b400640 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe008025cb0 tx=0x7fe008002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b15e5b0aa6ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe01a000640 1 -- 172.21.15.37:0/997951172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008066070 con 0x55dd5c44b280 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe01a000640 1 -- 172.21.15.37:0/997951172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe00804da90 con 0x55dd5c44b280 2024-09-21T06:30:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe01a000640 1 -- 172.21.15.37:0/997951172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008056470 con 0x55dd5c44b280 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe02371ef40 1 -- 172.21.15.37:0/997951172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 msgr2=0x55dd5c447170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.057+0000 7fe02371ef40 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe008025cb0 tx=0x7fe008002fb0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 -- 172.21.15.37:0/997951172 shutdown_connections 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c44b280 0x55dd5c447170 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c4476b0 0x55dd5c447ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 --2- 172.21.15.37:0/997951172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c37c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 -- 172.21.15.37:0/997951172 >> 172.21.15.37:0/997951172 conn(0x55dd5c29aac0 msgr2=0x55dd5c37d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 -- 172.21.15.37:0/997951172 shutdown_connections 2024-09-21T06:30:12.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.058+0000 7fe02371ef40 1 -- 172.21.15.37:0/997951172 wait complete. 2024-09-21T06:30:12.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.059+0000 7fe02371ef40 1 Processor -- start 2024-09-21T06:30:12.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.059+0000 7fe02371ef40 1 -- start start 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c43a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 0x55dd5c3c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c44b280 0x55dd5c3c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd5c448000 con 0x55dd5c4476b0 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dd5c4e8900 con 0x55dd5c44b280 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe02371ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dd5c43bb60 con 0x55dd5c37d9b0 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe01b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 0x55dd5c3c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.060+0000 7fe01b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 0x55dd5c3c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58522/0 (socket says 172.21.15.37:58522) 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 -- 172.21.15.37:0/2401395492 learned_addr learned my addr 172.21.15.37:0/2401395492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01c800640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c44b280 0x55dd5c3c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01be00640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c43a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 -- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 msgr2=0x55dd5c43a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dd5c37d9b0 0x55dd5c43a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 -- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c44b280 msgr2=0x55dd5c3c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd5c44b280 0x55dd5c3c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 -- 172.21.15.37:0/2401395492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd5c43bdb0 con 0x55dd5c4476b0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.061+0000 7fe01b400640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 0x55dd5c3c48c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe008059040 tx=0x7fe00804d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.062+0000 7fe018c00640 1 -- 172.21.15.37:0/2401395492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008071070 con 0x55dd5c4476b0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.062+0000 7fe018c00640 1 -- 172.21.15.37:0/2401395492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe008067040 con 0x55dd5c4476b0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.062+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dd5c43c090 con 0x55dd5c4476b0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.062+0000 7fe018c00640 1 -- 172.21.15.37:0/2401395492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008066070 con 0x55dd5c4476b0 2024-09-21T06:30:12.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.062+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd5c43a740 con 0x55dd5c4476b0 2024-09-21T06:30:12.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.064+0000 7fe018c00640 1 -- 172.21.15.37:0/2401395492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe00806d990 con 0x55dd5c4476b0 2024-09-21T06:30:12.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.065+0000 7fe018c00640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdffc0775a0 0x7fdffc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.065+0000 7fe018c00640 1 -- 172.21.15.37:0/2401395492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe0080df2d0 con 0x55dd5c4476b0 2024-09-21T06:30:12.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.065+0000 7fe02371ef40 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55dd5c440d80 0x55dd5c441190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.065+0000 7fe01be00640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdffc0775a0 0x7fdffc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.065+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 5.9 5:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55dd5c4416d0 con 0x55dd5c440d80 2024-09-21T06:30:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.066+0000 7fe01c800640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55dd5c440d80 0x55dd5c441190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.066+0000 7fe01be00640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdffc0775a0 0x7fdffc079ab0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe00c002fd0 tx=0x7fe00c002c80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.066+0000 7fe01c800640 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55dd5c440d80 0x55dd5c441190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe01c800640 1 -- 172.21.15.37:0/2401395492 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fe014002860 con 0x55dd5c440d80 2024-09-21T06:30:12.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55dd5c440d80 msgr2=0x55dd5c441190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55dd5c440d80 0x55dd5c441190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdffc0775a0 msgr2=0x7fdffc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdffc0775a0 0x7fdffc079ab0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe00c002fd0 tx=0x7fe00c002c80 comp rx=0 tx=0).stop 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 msgr2=0x55dd5c3c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 --2- 172.21.15.37:0/2401395492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd5c4476b0 0x55dd5c3c48c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe008059040 tx=0x7fe00804d690 comp rx=0 tx=0).stop 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe01c800640 1 -- 172.21.15.37:0/2401395492 reap_dead start 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 shutdown_connections 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 >> 172.21.15.37:0/2401395492 conn(0x55dd5c29aac0 msgr2=0x55dd5c37d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 shutdown_connections 2024-09-21T06:30:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.067+0000 7fe02371ef40 1 -- 172.21.15.37:0/2401395492 wait complete. 2024-09-21T06:30:12.080 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:12.080 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2024-09-21T06:30:12.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.222+0000 7f56c408cf40 1 Processor -- start 2024-09-21T06:30:12.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.223+0000 7f56c408cf40 1 -- start start 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c8883090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c8927280 0x55a2c892ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2c87a5c60 con 0x55a2c88599b0 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2c88041d0 con 0x55a2c88a1360 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56c408cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2c8859dc0 con 0x55a2c8927280 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56bd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56bd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58548/0 (socket says 172.21.15.37:58548) 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56bd200640 1 -- 172.21.15.37:0/727770172 learned_addr learned my addr 172.21.15.37:0/727770172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56bdc00640 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c8927280 0x55a2c892ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.224+0000 7f56b7e00640 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c8883090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 -- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c8927280 msgr2=0x55a2c892ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c8927280 0x55a2c892ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 -- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 msgr2=0x55a2c8883090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c8883090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 -- 172.21.15.37:0/727770172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c8884d40 con 0x55a2c88599b0 2024-09-21T06:30:12.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56bd200640 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56b0025e60 tx=0x7f56b0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1015041ae72d52f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56b6a00640 1 -- 172.21.15.37:0/727770172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b004d8c0 con 0x55a2c88599b0 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56b6a00640 1 -- 172.21.15.37:0/727770172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b004dd70 con 0x55a2c88599b0 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.225+0000 7f56b6a00640 1 -- 172.21.15.37:0/727770172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00566d0 con 0x55a2c88599b0 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 -- 172.21.15.37:0/727770172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 msgr2=0x55a2c89baaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56b0025e60 tx=0x7f56b0002fb0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 -- 172.21.15.37:0/727770172 shutdown_connections 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c8927280 0x55a2c892ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c8883090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 --2- 172.21.15.37:0/727770172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c88599b0 0x55a2c89baaf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 -- 172.21.15.37:0/727770172 >> 172.21.15.37:0/727770172 conn(0x55a2c8776ac0 msgr2=0x55a2c88a0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 -- 172.21.15.37:0/727770172 shutdown_connections 2024-09-21T06:30:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.226+0000 7f56c408cf40 1 -- 172.21.15.37:0/727770172 wait complete. 2024-09-21T06:30:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.227+0000 7f56c408cf40 1 Processor -- start 2024-09-21T06:30:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.227+0000 7f56c408cf40 1 -- start start 2024-09-21T06:30:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c88599b0 0x55a2c88570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c891cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 0x55a2c89ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2c8885000 con 0x55a2c8927280 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2c8927b40 con 0x55a2c88a1360 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56c408cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2c88835d0 con 0x55a2c88599b0 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 0x55a2c89ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 0x55a2c89ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58560/0 (socket says 172.21.15.37:58560) 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 -- 172.21.15.37:0/3788940310 learned_addr learned my addr 172.21.15.37:0/3788940310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56b7e00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c891cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bd200640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c88599b0 0x55a2c88570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 -- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c88599b0 msgr2=0x55a2c88570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c88599b0 0x55a2c88570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 -- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 msgr2=0x55a2c891cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2c88a1360 0x55a2c891cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bdc00640 1 -- 172.21.15.37:0/3788940310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c891fa00 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.229+0000 7f56bd200640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2c88599b0 0x55a2c88570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56bdc00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 0x55a2c89ba6b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b400 tx=0x7f56b800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56b5600640 1 -- 172.21.15.37:0/3788940310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8010560 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56b5600640 1 -- 172.21.15.37:0/3788940310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b8010810 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56b5600640 1 -- 172.21.15.37:0/3788940310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8010c60 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a2c891f720 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.230+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c89b9eb0 con 0x55a2c8927280 2024-09-21T06:30:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.232+0000 7f56b5600640 1 -- 172.21.15.37:0/3788940310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56b801b420 con 0x55a2c8927280 2024-09-21T06:30:12.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.233+0000 7f56b5600640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f569c0775a0 0x7f569c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.233+0000 7f56b5600640 1 -- 172.21.15.37:0/3788940310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f56b80999c0 con 0x55a2c8927280 2024-09-21T06:30:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.234+0000 7f56bd200640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f569c0775a0 0x7f569c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.234+0000 7f56c408cf40 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x55a2c89c6240 0x55a2c89c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.234+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- osd_op(unknown.0.0:1 5.5 5:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55a2c89c81f0 con 0x55a2c89c6240 2024-09-21T06:30:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.234+0000 7f56b7e00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x55a2c89c6240 0x55a2c89c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.234+0000 7f56bd200640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f569c0775a0 0x7f569c079ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56b0000c00 tx=0x7f56b0004940 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56b7e00640 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x55a2c89c6240 0x55a2c89c7cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56b7e00640 1 -- 172.21.15.37:0/3788940310 <== osd.7 v2:172.21.15.150:6826/1641077969 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f56a800c020 con 0x55a2c89c6240 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x55a2c89c6240 msgr2=0x55a2c89c7cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x55a2c89c6240 0x55a2c89c7cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f569c0775a0 msgr2=0x7f569c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f569c0775a0 0x7f569c079ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56b0000c00 tx=0x7f56b0004940 comp rx=0 tx=0).stop 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 msgr2=0x55a2c89ba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 --2- 172.21.15.37:0/3788940310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2c8927280 0x55a2c89ba6b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b400 tx=0x7f56b800b8c0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56bdc00640 1 -- 172.21.15.37:0/3788940310 reap_dead start 2024-09-21T06:30:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 shutdown_connections 2024-09-21T06:30:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.235+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 >> 172.21.15.37:0/3788940310 conn(0x55a2c8776ac0 msgr2=0x55a2c8881e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.236+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 shutdown_connections 2024-09-21T06:30:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.236+0000 7f56c408cf40 1 -- 172.21.15.37:0/3788940310 wait complete. 2024-09-21T06:30:12.248 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:12.248 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2024-09-21T06:30:12.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.393+0000 7f10b90f6f40 1 Processor -- start 2024-09-21T06:30:12.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.394+0000 7f10b90f6f40 1 -- start start 2024-09-21T06:30:12.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba86cbb0 0x55b7ba8524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba8f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba773c60 con 0x55b7ba826cf0 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba7d21d0 con 0x55b7ba8e1650 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b7ba86dd10 con 0x55b7ba86cbb0 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b2200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b2200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58574/0 (socket says 172.21.15.37:58574) 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b2200640 1 -- 172.21.15.37:0/2247178657 learned_addr learned my addr 172.21.15.37:0/2247178657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.395+0000 7f10b1800640 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba86cbb0 0x55b7ba8524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2c00640 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba8f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 -- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba86cbb0 msgr2=0x55b7ba8524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba86cbb0 0x55b7ba8524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 -- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 msgr2=0x55b7ba8f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba8f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 -- 172.21.15.37:0/2247178657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ba8dd270 con 0x55b7ba826cf0 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10b2200640 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10a4046520 tx=0x7f10a4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5093de18f5a152a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10abe00640 1 -- 172.21.15.37:0/2247178657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a404dd10 con 0x55b7ba826cf0 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10abe00640 1 -- 172.21.15.37:0/2247178657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a4056470 con 0x55b7ba826cf0 2024-09-21T06:30:12.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.396+0000 7f10abe00640 1 -- 172.21.15.37:0/2247178657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40568c0 con 0x55b7ba826cf0 2024-09-21T06:30:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 -- 172.21.15.37:0/2247178657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 msgr2=0x55b7ba852ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10a4046520 tx=0x7f10a4002fb0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 -- 172.21.15.37:0/2247178657 shutdown_connections 2024-09-21T06:30:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba8f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba86cbb0 0x55b7ba8524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/2247178657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba826cf0 0x55b7ba852ef0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 -- 172.21.15.37:0/2247178657 >> 172.21.15.37:0/2247178657 conn(0x55b7ba744ac0 msgr2=0x55b7ba84fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 -- 172.21.15.37:0/2247178657 shutdown_connections 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.397+0000 7f10b90f6f40 1 -- 172.21.15.37:0/2247178657 wait complete. 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.398+0000 7f10b90f6f40 1 Processor -- start 2024-09-21T06:30:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.398+0000 7f10b90f6f40 1 -- start start 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba826cf0 0x55b7ba8fab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 0x55b7ba821b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba98aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba8dd510 con 0x55b7ba86cbb0 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba852a00 con 0x55b7ba8e1650 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b90f6f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b7ba8fb420 con 0x55b7ba826cf0 2024-09-21T06:30:12.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 0x55b7ba821b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 0x55b7ba821b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58580/0 (socket says 172.21.15.37:58580) 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b1800640 1 -- 172.21.15.37:0/1328551782 learned_addr learned my addr 172.21.15.37:0/1328551782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b2200640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba826cf0 0x55b7ba8fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.400+0000 7f10b2c00640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba98aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 -- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba826cf0 msgr2=0x55b7ba8fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba826cf0 0x55b7ba8fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 -- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 msgr2=0x55b7ba98aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba98aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 -- 172.21.15.37:0/1328551782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ba8fbfe0 con 0x55b7ba86cbb0 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b2200640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba826cf0 0x55b7ba8fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:12.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b2c00640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba8e1650 0x55b7ba98aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b1800640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 0x55b7ba821b70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10a000a680 tx=0x7f10a000ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10aaa00640 1 -- 172.21.15.37:0/1328551782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0008990 con 0x55b7ba86cbb0 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10aaa00640 1 -- 172.21.15.37:0/1328551782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a0008b30 con 0x55b7ba86cbb0 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7ba98d9b0 con 0x55b7ba86cbb0 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10aaa00640 1 -- 172.21.15.37:0/1328551782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0016420 con 0x55b7ba86cbb0 2024-09-21T06:30:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.401+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ba822950 con 0x55b7ba86cbb0 2024-09-21T06:30:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.404+0000 7f10aaa00640 1 -- 172.21.15.37:0/1328551782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10a001f440 con 0x55b7ba86cbb0 2024-09-21T06:30:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.404+0000 7f10aaa00640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10940775a0 0x7f1094079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.405+0000 7f10aaa00640 1 -- 172.21.15.37:0/1328551782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f10a0099cb0 con 0x55b7ba86cbb0 2024-09-21T06:30:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.405+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x55b7ba8220b0 0x55b7ba8f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.405+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 --> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] -- osd_op(unknown.0.0:1 5.1d 5:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55b7ba99ef30 con 0x55b7ba8220b0 2024-09-21T06:30:12.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.405+0000 7f10b2200640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10940775a0 0x7f1094079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.405+0000 7f10b2c00640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x55b7ba8220b0 0x55b7ba8f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.406+0000 7f10b2200640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10940775a0 0x7f1094079ab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10a4046520 tx=0x7f10a4004880 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.406+0000 7f10b2c00640 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x55b7ba8220b0 0x55b7ba8f0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b2c00640 1 -- 172.21.15.37:0/1328551782 <== osd.4 v2:172.21.15.150:6818/2746357825 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f10ac00f580 con 0x55b7ba8220b0 2024-09-21T06:30:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x55b7ba8220b0 msgr2=0x55b7ba8f0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x55b7ba8220b0 0x55b7ba8f0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10940775a0 msgr2=0x7f1094079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10940775a0 0x7f1094079ab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10a4046520 tx=0x7f10a4004880 comp rx=0 tx=0).stop 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 msgr2=0x55b7ba821b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 --2- 172.21.15.37:0/1328551782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba86cbb0 0x55b7ba821b70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10a000a680 tx=0x7f10a000ab40 comp rx=0 tx=0).stop 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b2c00640 1 -- 172.21.15.37:0/1328551782 reap_dead start 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 shutdown_connections 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 >> 172.21.15.37:0/1328551782 conn(0x55b7ba744ac0 msgr2=0x55b7ba825e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 shutdown_connections 2024-09-21T06:30:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.407+0000 7f10b90f6f40 1 -- 172.21.15.37:0/1328551782 wait complete. 2024-09-21T06:30:12.419 INFO:teuthology.orchestra.run.smithi037.stdout:tag123 2024-09-21T06:30:12.420 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:30:12.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:30:12.510 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:30:12.606 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:30:12.606 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:30:12.606 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:12.606 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:12.641 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:12.642 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:12.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 -- 172.21.15.37:0/697384489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8145bf0 msgr2=0x7f95c80d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 --2- 172.21.15.37:0/697384489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8145bf0 0x7f95c80d05b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f95c8073770 tx=0x7f95b402a560 comp rx=0 tx=0).stop 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 -- 172.21.15.37:0/697384489 shutdown_connections 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 --2- 172.21.15.37:0/697384489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c8071160 0x7f95c8071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 --2- 172.21.15.37:0/697384489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8145bf0 0x7f95c80d05b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 --2- 172.21.15.37:0/697384489 >> v2:172.21.15.150:3300/0 conn(0x7f95c815d5e0 0x7f95c80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 -- 172.21.15.37:0/697384489 >> 172.21.15.37:0/697384489 conn(0x7f95c806fc80 msgr2=0x7f95c8070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 -- 172.21.15.37:0/697384489 shutdown_connections 2024-09-21T06:30:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.984+0000 7f95cee00640 1 -- 172.21.15.37:0/697384489 wait complete. 2024-09-21T06:30:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.986+0000 7f95cee00640 1 Processor -- start 2024-09-21T06:30:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.987+0000 7f95cee00640 1 -- start start 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 0x7f95c8143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c815d5e0 0x7f95c8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95c806cde0 con 0x7f95c8071160 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f95c80d0d00 con 0x7f95c8145bf0 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f95c806cf60 con 0x7f95c815d5e0 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58592/0 (socket says 172.21.15.37:58592) 2024-09-21T06:30:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.988+0000 7f95cda00640 1 -- 172.21.15.37:0/4280907650 learned_addr learned my addr 172.21.15.37:0/4280907650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cd000640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 0x7f95c8143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95ce400640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c815d5e0 0x7f95c8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 -- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c815d5e0 msgr2=0x7f95c8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c815d5e0 0x7f95c8155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 -- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 msgr2=0x7f95c8143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 0x7f95c8143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 -- 172.21.15.37:0/4280907650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c8143ff0 con 0x7f95c8071160 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cd000640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 0x7f95c8143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95cda00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d780 tx=0x7f95bc00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.989+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95c8071160 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.990+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95bc00de90 con 0x7f95c8071160 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.990+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0099d0 con 0x7f95c8071160 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.990+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c81f5e00 con 0x7f95c8071160 2024-09-21T06:30:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.990+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c81f63c0 con 0x7f95c8071160 2024-09-21T06:30:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.991+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c806d450 con 0x7f95c8071160 2024-09-21T06:30:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.992+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95bc009b70 con 0x7f95c8071160 2024-09-21T06:30:12.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.993+0000 7f95c5600640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 0x7f95a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:12.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.993+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f95bc09b380 con 0x7f95c8071160 2024-09-21T06:30:12.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.994+0000 7f95cd000640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 0x7f95a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.995+0000 7f95cd000640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 0x7f95a807b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95b4033000 tx=0x7f95b4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:12.998+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95bc020020 con 0x7f95c8071160 2024-09-21T06:30:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.210+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95c80b08a0 con 0x7f95c8071160 2024-09-21T06:30:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.210+0000 7f95c5600640 1 -- 172.21.15.37:0/4280907650 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7f95bc0635f0 con 0x7f95c8071160 2024-09-21T06:30:13.213 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:30:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 msgr2=0x7f95a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 0x7f95a807b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95b4033000 tx=0x7f95b4004730 comp rx=0 tx=0).stop 2024-09-21T06:30:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 msgr2=0x7f95c813f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d780 tx=0x7f95bc00dc40 comp rx=0 tx=0).stop 2024-09-21T06:30:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 shutdown_connections 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95a8078b90 0x7f95a807b0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95c815d5e0 0x7f95c8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95c8145bf0 0x7f95c8143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 --2- 172.21.15.37:0/4280907650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c8071160 0x7f95c813f3d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 >> 172.21.15.37:0/4280907650 conn(0x7f95c806fc80 msgr2=0x7f95c80b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 shutdown_connections 2024-09-21T06:30:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.213+0000 7f95cee00640 1 -- 172.21.15.37:0/4280907650 wait complete. 2024-09-21T06:30:13.236 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:30:13.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:13.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:30:13.302 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:30:13.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:13.302 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:30:13.339 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:30:13.339 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:30:13.339 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:13.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:30:13.379 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:30:13.379 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:13.379 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:13.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:30:13.415 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:30:13.416 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:30:13.416 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:13.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:30:13.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 -- 172.21.15.37:0/1572338563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe8073dd0 msgr2=0x7f4fe806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 --2- 172.21.15.37:0/1572338563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe8073dd0 0x7f4fe806ddf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4fe815d980 tx=0x7f4fd402a530 comp rx=0 tx=0).stop 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 -- 172.21.15.37:0/1572338563 shutdown_connections 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 --2- 172.21.15.37:0/1572338563 >> v2:172.21.15.150:3300/0 conn(0x7f4fe806e330 0x7f4fe806ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 --2- 172.21.15.37:0/1572338563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe8073dd0 0x7f4fe806ddf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 -- 172.21.15.37:0/1572338563 >> 172.21.15.37:0/1572338563 conn(0x7f4fe80cca40 msgr2=0x7f4fe80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 -- 172.21.15.37:0/1572338563 shutdown_connections 2024-09-21T06:30:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.681+0000 7f4fed000640 1 -- 172.21.15.37:0/1572338563 wait complete. 2024-09-21T06:30:13.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.683+0000 7f4fed000640 1 Processor -- start 2024-09-21T06:30:13.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.683+0000 7f4fed000640 1 -- start start 2024-09-21T06:30:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fe8073dd0 0x7f4fe81f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fe8074cb0 0x7f4fe80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe806f4a0 con 0x7f4fe806e330 2024-09-21T06:30:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe806f320 con 0x7f4fe8073dd0 2024-09-21T06:30:13.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fed000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe806f620 con 0x7f4fe8074cb0 2024-09-21T06:30:13.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fe5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:13.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fe5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58616/0 (socket says 172.21.15.37:58616) 2024-09-21T06:30:13.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fe5e00640 1 -- 172.21.15.37:0/621753572 learned_addr learned my addr 172.21.15.37:0/621753572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fe5400640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fe8073dd0 0x7f4fe81f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.685+0000 7f4fe6800640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fe8074cb0 0x7f4fe80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 -- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fe8074cb0 msgr2=0x7f4fe80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fe8074cb0 0x7f4fe80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 -- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fe8073dd0 msgr2=0x7f4fe81f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fe8073dd0 0x7f4fe81f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 -- 172.21.15.37:0/621753572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe80b1940 con 0x7f4fe806e330 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fe5e00640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c390 tx=0x7f4fd800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8017070 con 0x7f4fe806e330 2024-09-21T06:30:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fd80035f0 con 0x7f4fe806e330 2024-09-21T06:30:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8003a20 con 0x7f4fe806e330 2024-09-21T06:30:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.686+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe8152650 con 0x7f4fe806e330 2024-09-21T06:30:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.687+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe8152c10 con 0x7f4fe806e330 2024-09-21T06:30:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.688+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe8072860 con 0x7f4fe806e330 2024-09-21T06:30:13.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.689+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4fd801a070 con 0x7f4fe806e330 2024-09-21T06:30:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.690+0000 7f4fde000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 0x7f4fc807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.690+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4fd809a1e0 con 0x7f4fe806e330 2024-09-21T06:30:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.691+0000 7f4fe5400640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 0x7f4fc807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.692+0000 7f4fe5400640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 0x7f4fc807b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:13.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.695+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4fe8072860 con 0x7f4fe806e330 2024-09-21T06:30:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.929+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe80731a0 con 0x7f4fe806e330 2024-09-21T06:30:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.930+0000 7f4fde000640 1 -- 172.21.15.37:0/621753572 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5270 (secure 0 0 0) 0x7f4fe80731a0 con 0x7f4fe806e330 2024-09-21T06:30:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:13.933 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-21T06:29:41:227586+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:40.565163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984,4977,4974]},"id":2}]} 2024-09-21T06:30:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 msgr2=0x7f4fc807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 0x7f4fc807b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd4004730 comp rx=0 tx=0).stop 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 msgr2=0x7f4fe81f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c390 tx=0x7f4fd800c850 comp rx=0 tx=0).stop 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 shutdown_connections 2024-09-21T06:30:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4fc8078bc0 0x7f4fc807b0d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fe8074cb0 0x7f4fe80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fe8073dd0 0x7f4fe81f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.932+0000 7f4fed000640 1 --2- 172.21.15.37:0/621753572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe806e330 0x7f4fe81f71b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.933+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 >> 172.21.15.37:0/621753572 conn(0x7f4fe80cca40 msgr2=0x7f4fe80caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.933+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 shutdown_connections 2024-09-21T06:30:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:13.933+0000 7f4fed000640 1 -- 172.21.15.37:0/621753572 wait complete. 2024-09-21T06:30:13.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 -- 172.21.15.37:0/3619149461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa88003c30 msgr2=0x7faa88004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 --2- 172.21.15.37:0/3619149461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa88003c30 0x7faa88004080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faa94009b50 tx=0x7faa88005830 comp rx=0 tx=0).stop 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 -- 172.21.15.37:0/3619149461 shutdown_connections 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 --2- 172.21.15.37:0/3619149461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa88003c30 0x7faa88004080 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 -- 172.21.15.37:0/3619149461 >> 172.21.15.37:0/3619149461 conn(0x7faaa40d1020 msgr2=0x7faaa40d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 -- 172.21.15.37:0/3619149461 shutdown_connections 2024-09-21T06:30:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.183+0000 7faaa3e00640 1 -- 172.21.15.37:0/3619149461 wait complete. 2024-09-21T06:30:14.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.185+0000 7faaa3e00640 1 Processor -- start 2024-09-21T06:30:14.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.186+0000 7faaa3e00640 1 -- start start 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 0x7faaa4146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 0x7faaa415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faaa415e4f0 con 0x7faaa4156aa0 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faaa406e510 con 0x7faaa4153f20 2024-09-21T06:30:14.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faaa3e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faaa415e670 con 0x7faa88003c30 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faa9a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faa9a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60396/0 (socket says 172.21.15.37:60396) 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.187+0000 7faa9a000640 1 -- 172.21.15.37:0/4235721088 learned_addr learned my addr 172.21.15.37:0/4235721088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faaa2a00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 0x7faaa4146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faaa3400640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 0x7faaa415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 -- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 msgr2=0x7faaa4146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 0x7faaa4146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 -- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 msgr2=0x7faaa415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 0x7faaa415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 -- 172.21.15.37:0/4235721088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa41f6460 con 0x7faaa4156aa0 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faaa2a00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 0x7faaa4146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faaa3400640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 0x7faaa415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:14.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9a000640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa8c006fd0 tx=0x7faa8c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011640 con 0x7faaa4156aa0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa8c016020 con 0x7faaa4156aa0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011aa0 con 0x7faaa4156aa0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.188+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa415ac90 con 0x7faaa4156aa0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.189+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa415b250 con 0x7faaa4156aa0 2024-09-21T06:30:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.190+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa4157270 con 0x7faaa4156aa0 2024-09-21T06:30:14.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.191+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa8c011c40 con 0x7faaa4156aa0 2024-09-21T06:30:14.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.192+0000 7faa9be00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 0x7faa8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.192+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7faa8c09bb50 con 0x7faaa4156aa0 2024-09-21T06:30:14.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.193+0000 7faaa2a00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 0x7faa8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.193+0000 7faaa2a00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 0x7faa8007b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa94035000 tx=0x7faa940046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.197+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faa8c020020 con 0x7faaa4156aa0 2024-09-21T06:30:14.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.433+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaa4071160 con 0x7faaa4156aa0 2024-09-21T06:30:14.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.434+0000 7faa9be00640 1 -- 172.21.15.37:0/4235721088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5270 (secure 0 0 0) 0x7faa8c063dc0 con 0x7faaa4156aa0 2024-09-21T06:30:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:14.436 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-21T06:29:41:227586+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:40.565163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984,4977,4974]},"id":2}]} 2024-09-21T06:30:14.436 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 msgr2=0x7faa8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 0x7faa8007b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa94035000 tx=0x7faa940046c0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 msgr2=0x7faaa415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa8c006fd0 tx=0x7faa8c001800 comp rx=0 tx=0).stop 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 shutdown_connections 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa80078b90 0x7faa8007b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa4156aa0 0x7faaa415c950 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa4153f20 0x7faaa415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 --2- 172.21.15.37:0/4235721088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa88003c30 0x7faaa4146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.436+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 >> 172.21.15.37:0/4235721088 conn(0x7faaa40d1020 msgr2=0x7faaa40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:14.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.437+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 shutdown_connections 2024-09-21T06:30:14.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.437+0000 7faaa3e00640 1 -- 172.21.15.37:0/4235721088 wait complete. 2024-09-21T06:30:14.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:14.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 -- 172.21.15.37:0/1178961939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 msgr2=0x7fed900cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 --2- 172.21.15.37:0/1178961939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed900cf5c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fed78009830 tx=0x7fed7802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 -- 172.21.15.37:0/1178961939 shutdown_connections 2024-09-21T06:30:14.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 --2- 172.21.15.37:0/1178961939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed900cf5c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 -- 172.21.15.37:0/1178961939 >> 172.21.15.37:0/1178961939 conn(0x7fed900d4360 msgr2=0x7fed900d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:14.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 -- 172.21.15.37:0/1178961939 shutdown_connections 2024-09-21T06:30:14.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.688+0000 7fed95c00640 1 -- 172.21.15.37:0/1178961939 wait complete. 2024-09-21T06:30:14.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.690+0000 7fed95c00640 1 Processor -- start 2024-09-21T06:30:14.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.691+0000 7fed95c00640 1 -- start start 2024-09-21T06:30:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed900cfb00 0x7fed901f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed9015e780 0x7fed901624d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed9006cf60 con 0x7fed9006e220 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed9006cde0 con 0x7fed900cfb00 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed95c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fed900b80c0 con 0x7fed9015e780 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60426/0 (socket says 172.21.15.37:60426) 2024-09-21T06:30:14.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.692+0000 7fed8fe00640 1 -- 172.21.15.37:0/2711338701 learned_addr learned my addr 172.21.15.37:0/2711338701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed87400640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed900cfb00 0x7fed901f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed95200640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed9015e780 0x7fed901624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 -- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed9015e780 msgr2=0x7fed901624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed9015e780 0x7fed901624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 -- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed900cfb00 msgr2=0x7fed901f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed900cfb00 0x7fed901f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 -- 172.21.15.37:0/2711338701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9015aff0 con 0x7fed9006e220 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8fe00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c430 tx=0x7fed8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80017070 con 0x7fed9006e220 2024-09-21T06:30:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed800053c0 con 0x7fed9006e220 2024-09-21T06:30:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901f63c0 con 0x7fed9006e220 2024-09-21T06:30:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.693+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80003850 con 0x7fed9006e220 2024-09-21T06:30:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.694+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed900aac90 con 0x7fed9006e220 2024-09-21T06:30:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.694+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed900b85e0 con 0x7fed9006e220 2024-09-21T06:30:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.696+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed8001a070 con 0x7fed9006e220 2024-09-21T06:30:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.697+0000 7fed8cc00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 0x7fed6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.698+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fed8009a370 con 0x7fed9006e220 2024-09-21T06:30:14.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.698+0000 7fed87400640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 0x7fed6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:14.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.699+0000 7fed87400640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 0x7fed6c07b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed78004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:14.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.702+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed9015aff0 con 0x7fed9006e220 2024-09-21T06:30:14.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.899+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed900c7b10 con 0x7fed9006e220 2024-09-21T06:30:14.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.900+0000 7fed8cc00640 1 -- 172.21.15.37:0/2711338701 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17049 (secure 0 0 0) 0x7fed80062550 con 0x7fed9006e220 2024-09-21T06:30:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:14.903 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:29:39.217043+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:29:06.023459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T06:29:06.776946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.14","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 msgr2=0x7fed6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 0x7fed6c07b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed78004730 comp rx=0 tx=0).stop 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 msgr2=0x7fed9014e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c430 tx=0x7fed8000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 shutdown_connections 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed6c078bc0 0x7fed6c07b0d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed9015e780 0x7fed901624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed900cfb00 0x7fed901f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 --2- 172.21.15.37:0/2711338701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed9006e220 0x7fed9014e810 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:14.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 >> 172.21.15.37:0/2711338701 conn(0x7fed900d4360 msgr2=0x7fed900b9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:14.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 shutdown_connections 2024-09-21T06:30:14.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:14.903+0000 7fed95c00640 1 -- 172.21.15.37:0/2711338701 wait complete. 2024-09-21T06:30:14.928 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:30:14.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.153+0000 7f34c6400640 1 -- 172.21.15.37:0/1368948926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c015daf0 msgr2=0x7f34c015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.153+0000 7f34c6400640 1 --2- 172.21.15.37:0/1368948926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c015daf0 0x7f34c015a6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f34ac005fd0 tx=0x7f34ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 -- 172.21.15.37:0/1368948926 shutdown_connections 2024-09-21T06:30:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 --2- 172.21.15.37:0/1368948926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c00d0980 0x7f34c00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 --2- 172.21.15.37:0/1368948926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c015daf0 0x7f34c015a6f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 -- 172.21.15.37:0/1368948926 >> 172.21.15.37:0/1368948926 conn(0x7f34c0072990 msgr2=0x7f34c0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 -- 172.21.15.37:0/1368948926 shutdown_connections 2024-09-21T06:30:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.154+0000 7f34c6400640 1 -- 172.21.15.37:0/1368948926 wait complete. 2024-09-21T06:30:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.156+0000 7f34c6400640 1 Processor -- start 2024-09-21T06:30:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.156+0000 7f34c6400640 1 -- start start 2024-09-21T06:30:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.157+0000 7f34c6400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34c6400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34c00d0980 0x7f34c00a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34c6400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c015daf0 0x7f34c015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34c6400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34c00cd340 con 0x7f34c00d01f0 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34c6400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34c00cd1c0 con 0x7f34c00d0980 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34c6400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f34c00cd4c0 con 0x7f34c015daf0 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60456/0 (socket says 172.21.15.37:60456) 2024-09-21T06:30:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 -- 172.21.15.37:0/3790615234 learned_addr learned my addr 172.21.15.37:0/3790615234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bfe00640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c015daf0 0x7f34c015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bea00640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34c00d0980 0x7f34c00a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 -- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c015daf0 msgr2=0x7f34c015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c015daf0 0x7f34c015b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 -- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34c00d0980 msgr2=0x7f34c00a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34c00d0980 0x7f34c00a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.158+0000 7f34bf400640 1 -- 172.21.15.37:0/3790615234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c00c9380 con 0x7f34c00d01f0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34bf400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c3a0 tx=0x7f34b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0017070 con 0x7f34c00d01f0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34b00053c0 con 0x7f34c00d01f0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0003850 con 0x7f34c00d01f0 2024-09-21T06:30:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c00c9660 con 0x7f34c00d01f0 2024-09-21T06:30:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.159+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c00c9c20 con 0x7f34c00d01f0 2024-09-21T06:30:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.160+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c00cd970 con 0x7f34c00d01f0 2024-09-21T06:30:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.161+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34b001a070 con 0x7f34c00d01f0 2024-09-21T06:30:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.162+0000 7f34b7400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 0x7f34a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.162+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f34b009a2b0 con 0x7f34c00d01f0 2024-09-21T06:30:15.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.163+0000 7f34bea00640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 0x7f34a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.164+0000 7f34bea00640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 0x7f34a007b0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34ac033000 tx=0x7f34ac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.167+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34c00d12d0 con 0x7f34c00d01f0 2024-09-21T06:30:15.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.401+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34c015cb80 con 0x7f34c00d01f0 2024-09-21T06:30:15.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.401+0000 7f34b7400640 1 -- 172.21.15.37:0/3790615234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5270 (secure 0 0 0) 0x7f34b0062490 con 0x7f34c00d01f0 2024-09-21T06:30:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:15.404 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-21T06:29:41:227586+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:40.565163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984,4977,4974]},"id":2}]} 2024-09-21T06:30:15.404 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.403+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 msgr2=0x7f34a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.403+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 0x7f34a007b0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34ac033000 tx=0x7f34ac004730 comp rx=0 tx=0).stop 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.403+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 msgr2=0x7f34c0150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.403+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c3a0 tx=0x7f34b000c860 comp rx=0 tx=0).stop 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 shutdown_connections 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34a0078bc0 0x7f34a007b0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34c015daf0 0x7f34c015b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34c00d0980 0x7f34c00a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 --2- 172.21.15.37:0/3790615234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c00d01f0 0x7f34c0150570 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 >> 172.21.15.37:0/3790615234 conn(0x7f34c0072990 msgr2=0x7f34c0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:15.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 shutdown_connections 2024-09-21T06:30:15.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.404+0000 7f34c6400640 1 -- 172.21.15.37:0/3790615234 wait complete. 2024-09-21T06:30:15.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 -- 172.21.15.37:0/1507121106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe5806ddf0 msgr2=0x7efe5806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 --2- 172.21.15.37:0/1507121106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe5806ddf0 0x7efe5806e200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efe580cfe90 tx=0x7efe5002aa30 comp rx=0 tx=0).stop 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 -- 172.21.15.37:0/1507121106 shutdown_connections 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 --2- 172.21.15.37:0/1507121106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe5806ddf0 0x7efe5806e200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 --2- 172.21.15.37:0/1507121106 >> v2:172.21.15.150:3300/0 conn(0x7efe580cbc10 0x7efe580cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 --2- 172.21.15.37:0/1507121106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 0x7efe58073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 -- 172.21.15.37:0/1507121106 >> 172.21.15.37:0/1507121106 conn(0x7efe580cca40 msgr2=0x7efe580cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 -- 172.21.15.37:0/1507121106 shutdown_connections 2024-09-21T06:30:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.652+0000 7efe5de00640 1 -- 172.21.15.37:0/1507121106 wait complete. 2024-09-21T06:30:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.654+0000 7efe5de00640 1 Processor -- start 2024-09-21T06:30:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.655+0000 7efe5de00640 1 -- start start 2024-09-21T06:30:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5806ddf0 0x7efe581f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 0x7efe581570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe5806f170 con 0x7efe580cbc10 2024-09-21T06:30:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe5806eff0 con 0x7efe5806ddf0 2024-09-21T06:30:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe5de00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efe5806f2f0 con 0x7efe58074cb0 2024-09-21T06:30:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe57400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe57400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60482/0 (socket says 172.21.15.37:60482) 2024-09-21T06:30:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.656+0000 7efe57400640 1 -- 172.21.15.37:0/1185378750 learned_addr learned my addr 172.21.15.37:0/1185378750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe56a00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5806ddf0 0x7efe581f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe4fe00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 0x7efe581570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 -- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 msgr2=0x7efe581570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 0x7efe581570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 -- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5806ddf0 msgr2=0x7efe581f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5806ddf0 0x7efe581f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 -- 172.21.15.37:0/1185378750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe581f5d00 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe57400640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe5002b020 tx=0x7efe50003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50033050 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe50004540 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.657+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe580c9ed0 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.658+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50038450 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.658+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe580b5500 con 0x7efe580cbc10 2024-09-21T06:30:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.659+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe580b5320 con 0x7efe580cbc10 2024-09-21T06:30:15.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.660+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efe50036070 con 0x7efe580cbc10 2024-09-21T06:30:15.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.661+0000 7efe4f400640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 0x7efe3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.661+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7efe500bc290 con 0x7efe580cbc10 2024-09-21T06:30:15.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.661+0000 7efe56a00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 0x7efe3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.662+0000 7efe56a00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 0x7efe3407b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe5815a730 tx=0x7efe48005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.665+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efe580b5320 con 0x7efe580cbc10 2024-09-21T06:30:15.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.898+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5815c060 con 0x7efe580cbc10 2024-09-21T06:30:15.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.898+0000 7efe4f400640 1 -- 172.21.15.37:0/1185378750 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5270 (secure 0 0 0) 0x7efe581f5d00 con 0x7efe580cbc10 2024-09-21T06:30:15.901 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:15.901 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-21T06:29:41:227586+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:29:07.816070+0000","modified":"2024-09-21T06:29:40.565163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4984,"mds_1":4977,"mds_2":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6834/2753796335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2753796335},{"type":"v1","addr":"172.21.15.37:6834","nonce":2753796335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4977":{"gid":4977,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/2364291417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2364291417},{"type":"v1","addr":"172.21.15.37:6835","nonce":2364291417}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/337579449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":337579449},{"type":"v1","addr":"172.21.15.150:6837","nonce":337579449}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984,4977,4974]},"id":2}]} 2024-09-21T06:30:15.901 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 msgr2=0x7efe3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 0x7efe3407b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe5815a730 tx=0x7efe48005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 msgr2=0x7efe581575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe5002b020 tx=0x7efe50003f10 comp rx=0 tx=0).stop 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 shutdown_connections 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efe34078bc0 0x7efe3407b0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe580cbc10 0x7efe581575f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe58074cb0 0x7efe581570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 --2- 172.21.15.37:0/1185378750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5806ddf0 0x7efe581f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 >> 172.21.15.37:0/1185378750 conn(0x7efe580cca40 msgr2=0x7efe580b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 shutdown_connections 2024-09-21T06:30:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:15.901+0000 7efe5de00640 1 -- 172.21.15.37:0/1185378750 wait complete. 2024-09-21T06:30:15.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 -- 172.21.15.37:0/2386921892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 msgr2=0x7fe67006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 --2- 172.21.15.37:0/2386921892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67006ddf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe65c005fd0 tx=0x7fe65c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 -- 172.21.15.37:0/2386921892 shutdown_connections 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 --2- 172.21.15.37:0/2386921892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe67006ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 --2- 172.21.15.37:0/2386921892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67006ddf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 -- 172.21.15.37:0/2386921892 >> 172.21.15.37:0/2386921892 conn(0x7fe6700cca40 msgr2=0x7fe6700cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 -- 172.21.15.37:0/2386921892 shutdown_connections 2024-09-21T06:30:16.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.152+0000 7fe678200640 1 -- 172.21.15.37:0/2386921892 wait complete. 2024-09-21T06:30:16.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.155+0000 7fe678200640 1 Processor -- start 2024-09-21T06:30:16.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.155+0000 7fe678200640 1 -- start start 2024-09-21T06:30:16.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe670158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 0x7fe67015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe67006f4a0 con 0x7fe670073dd0 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe67006f320 con 0x7fe670074cb0 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.156+0000 7fe678200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe67006f620 con 0x7fe67006e330 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60500/0 (socket says 172.21.15.37:60500) 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 -- 172.21.15.37:0/1557798325 learned_addr learned my addr 172.21.15.37:0/1557798325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe676200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 0x7fe67015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe675800640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe670158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 -- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 msgr2=0x7fe670158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe670158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 -- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 msgr2=0x7fe67015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 0x7fe67015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 -- 172.21.15.37:0/1557798325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6701591f0 con 0x7fe670073dd0 2024-09-21T06:30:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe676200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 0x7fe67015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe675800640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe670158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.157+0000 7fe674e00640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034040 tx=0x7fe65c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.158+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c03e030 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.158+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c003ae0 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.158+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c035040 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.158+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701f5e70 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.158+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701f9bd0 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.159+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67006acf0 con 0x7fe670073dd0 2024-09-21T06:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.160+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe65c041030 con 0x7fe670073dd0 2024-09-21T06:30:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.161+0000 7fe66d600640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 0x7fe65007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.161+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe65c0bb910 con 0x7fe670073dd0 2024-09-21T06:30:16.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.162+0000 7fe675800640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 0x7fe65007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.162+0000 7fe675800640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 0x7fe65007b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67015a730 tx=0x7fe664009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:16.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.165+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe67006e740 con 0x7fe670073dd0 2024-09-21T06:30:16.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.364+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe670074440 con 0x7fe670073dd0 2024-09-21T06:30:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.365+0000 7fe66d600640 1 -- 172.21.15.37:0/1557798325 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17049 (secure 0 0 0) 0x7fe65c083af0 con 0x7fe670073dd0 2024-09-21T06:30:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:16.368 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:29:39.217043+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:29:06.023459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T06:29:06.776946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.14","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 msgr2=0x7fe65007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 0x7fe65007b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67015a730 tx=0x7fe664009040 comp rx=0 tx=0).stop 2024-09-21T06:30:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 msgr2=0x7fe67009d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034040 tx=0x7fe65c0035f0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 shutdown_connections 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe650078b90 0x7fe65007b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe670073dd0 0x7fe67009d920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe670074cb0 0x7fe67015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 --2- 172.21.15.37:0/1557798325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe67006e330 0x7fe670158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 >> 172.21.15.37:0/1557798325 conn(0x7fe6700cca40 msgr2=0x7fe67015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 shutdown_connections 2024-09-21T06:30:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.368+0000 7fe678200640 1 -- 172.21.15.37:0/1557798325 wait complete. 2024-09-21T06:30:16.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 -- 172.21.15.37:0/288175473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 msgr2=0x7fef6806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 --2- 172.21.15.37:0/288175473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef6806ddf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fef68075240 tx=0x7fef5402a540 comp rx=0 tx=0).stop 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 -- 172.21.15.37:0/288175473 shutdown_connections 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 --2- 172.21.15.37:0/288175473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef6806ddf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 --2- 172.21.15.37:0/288175473 >> v2:172.21.15.150:3300/0 conn(0x7fef680cc530 0x7fef6815ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.619+0000 7fef6d400640 1 -- 172.21.15.37:0/288175473 >> 172.21.15.37:0/288175473 conn(0x7fef680b1b90 msgr2=0x7fef680b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.620+0000 7fef6d400640 1 -- 172.21.15.37:0/288175473 shutdown_connections 2024-09-21T06:30:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.620+0000 7fef6d400640 1 -- 172.21.15.37:0/288175473 wait complete. 2024-09-21T06:30:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.622+0000 7fef6d400640 1 Processor -- start 2024-09-21T06:30:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.622+0000 7fef6d400640 1 -- start start 2024-09-21T06:30:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6806e330 0x7fef681e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef680cc530 0x7fef681f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef680cae00 con 0x7fef680af5b0 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef680cbd60 con 0x7fef6806e330 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.623+0000 7fef6d400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fef680caf80 con 0x7fef680cc530 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60506/0 (socket says 172.21.15.37:60506) 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 -- 172.21.15.37:0/792992218 learned_addr learned my addr 172.21.15.37:0/792992218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef66800640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6806e330 0x7fef681e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef67200640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef680cc530 0x7fef681f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 -- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef680cc530 msgr2=0x7fef681f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef680cc530 0x7fef681f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 -- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6806e330 msgr2=0x7fef681e48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6806e330 0x7fef681e48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 -- 172.21.15.37:0/792992218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef681f6060 con 0x7fef680af5b0 2024-09-21T06:30:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.624+0000 7fef65e00640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef54009920 tx=0x7fef54033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.625+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54031d60 con 0x7fef680af5b0 2024-09-21T06:30:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.625+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef54033e10 con 0x7fef680af5b0 2024-09-21T06:30:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.625+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54005260 con 0x7fef680af5b0 2024-09-21T06:30:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.625+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef681f6340 con 0x7fef680af5b0 2024-09-21T06:30:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.625+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef681e40d0 con 0x7fef680af5b0 2024-09-21T06:30:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.626+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef681f5db0 con 0x7fef680af5b0 2024-09-21T06:30:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.627+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef54005400 con 0x7fef680af5b0 2024-09-21T06:30:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.628+0000 7fef5ea00640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 0x7fef4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.628+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fef540bc100 con 0x7fef680af5b0 2024-09-21T06:30:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.629+0000 7fef66800640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 0x7fef4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.630+0000 7fef66800640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 0x7fef4807b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef58005fd0 tx=0x7fef5800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.633+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef5403f030 con 0x7fef680af5b0 2024-09-21T06:30:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.872+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fef680cb4a0 con 0x7fef680af5b0 2024-09-21T06:30:16.875 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:30:16.873+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:30:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.875+0000 7fef5ea00640 1 -- 172.21.15.37:0/792992218 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fef54083d20 con 0x7fef680af5b0 2024-09-21T06:30:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 msgr2=0x7fef4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 0x7fef4807b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef58005fd0 tx=0x7fef5800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 msgr2=0x7fef680b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef54009920 tx=0x7fef54033c20 comp rx=0 tx=0).stop 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 shutdown_connections 2024-09-21T06:30:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef48078b90 0x7fef4807b0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef680af5b0 0x7fef680b4410 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef680cc530 0x7fef681f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 --2- 172.21.15.37:0/792992218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6806e330 0x7fef681e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 >> 172.21.15.37:0/792992218 conn(0x7fef680b1b90 msgr2=0x7fef6806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 shutdown_connections 2024-09-21T06:30:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:16.879+0000 7fef6d400640 1 -- 172.21.15.37:0/792992218 wait complete. 2024-09-21T06:30:16.894 INFO:tasks.ceph.mds.b.smithi150.stderr: -8897> 2024-09-21T06:29:22.280+0000 7fdd99200640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:30:16.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:30:17.018 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:30:17.025 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:30:17.026 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.156+0000 7f1daca00640 1 -- 172.21.15.37:0/2701627660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da80cf850 msgr2=0x7f1da81556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.156+0000 7f1daca00640 1 --2- 172.21.15.37:0/2701627660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da80cf850 0x7f1da81556f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005fd0 tx=0x7f1d9802a670 comp rx=0 tx=0).stop 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.156+0000 7f1daca00640 1 -- 172.21.15.37:0/2701627660 shutdown_connections 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.156+0000 7f1daca00640 1 --2- 172.21.15.37:0/2701627660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da80cf850 0x7f1da81556f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.156+0000 7f1daca00640 1 -- 172.21.15.37:0/2701627660 >> 172.21.15.37:0/2701627660 conn(0x7f1da806f920 msgr2=0x7f1da80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.157+0000 7f1daca00640 1 -- 172.21.15.37:0/2701627660 shutdown_connections 2024-09-21T06:30:17.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.157+0000 7f1daca00640 1 -- 172.21.15.37:0/2701627660 wait complete. 2024-09-21T06:30:17.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.159+0000 7f1daca00640 1 Processor -- start 2024-09-21T06:30:17.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.159+0000 7f1daca00640 1 -- start start 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da80b1510 0x7f1da81e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8162b90 con 0x7f1da815f670 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1da806e930 con 0x7f1da80cf850 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.160+0000 7f1daca00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1da8162d10 con 0x7f1da80b1510 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1d9ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da80b1510 0x7f1da81e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:35372/0 (socket says 172.21.15.37:35372) 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 -- 172.21.15.37:0/469220279 learned_addr learned my addr 172.21.15.37:0/469220279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:17.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1d9ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60568/0 (socket says 172.21.15.37:60568) 2024-09-21T06:30:17.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 -- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da80b1510 msgr2=0x7f1da81e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:17.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da80b1510 0x7f1da81e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 -- 172.21.15.37:0/469220279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 msgr2=0x7f1da8154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1da7400640 1 -- 172.21.15.37:0/469220279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da81e7e80 con 0x7f1da80cf850 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.161+0000 7f1d9ea00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1da7400640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d98034040 tx=0x7f1d98003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9803e030 con 0x7f1da80cf850 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d980047a0 con 0x7f1da80cf850 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81e96e0 con 0x7f1da80cf850 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98035040 con 0x7f1da80cf850 2024-09-21T06:30:17.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.162+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81f6170 con 0x7f1da80cf850 2024-09-21T06:30:17.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.163+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da81556f0 con 0x7f1da80cf850 2024-09-21T06:30:17.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.164+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d98041030 con 0x7f1da80cf850 2024-09-21T06:30:17.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.165+0000 7f1d9fe00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 0x7f1d8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:17.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.165+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6417+0+0 (secure 0 0 0) 0x7f1d980bb850 con 0x7f1da80cf850 2024-09-21T06:30:17.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.166+0000 7f1d9ea00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 0x7f1d8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:17.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.167+0000 7f1d9ea00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 0x7f1d8407b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f60 tx=0x7f1d90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.171+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d98083360 con 0x7f1da80cf850 2024-09-21T06:30:17.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.407+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1da806e340 con 0x7f1da80cf850 2024-09-21T06:30:17.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.883+0000 7f1d9fe00640 1 -- 172.21.15.37:0/469220279 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f1d98087d90 con 0x7f1da80cf850 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 msgr2=0x7f1d8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 0x7f1d8407b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f60 tx=0x7f1d90009040 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 msgr2=0x7f1da81f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d98034040 tx=0x7f1d98003970 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 shutdown_connections 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1d84078bc0 0x7f1d8407b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da815f670 0x7f1da8154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da80b1510 0x7f1da81e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 --2- 172.21.15.37:0/469220279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da80cf850 0x7f1da81f58f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 >> 172.21.15.37:0/469220279 conn(0x7f1da806f920 msgr2=0x7f1da80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.886+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 shutdown_connections 2024-09-21T06:30:17.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:17.887+0000 7f1daca00640 1 -- 172.21.15.37:0/469220279 wait complete. 2024-09-21T06:30:17.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:30:18.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.149+0000 7ff5dc000640 1 -- 172.21.15.37:0/616591958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d4160c20 msgr2=0x7ff5d406d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:18.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 --2- 172.21.15.37:0/616591958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d4160c20 0x7ff5d406d820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005fd0 tx=0x7ff5cc02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:30:18.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 -- 172.21.15.37:0/616591958 shutdown_connections 2024-09-21T06:30:18.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 --2- 172.21.15.37:0/616591958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d4160c20 0x7ff5d406d820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:18.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 --2- 172.21.15.37:0/616591958 >> v2:172.21.15.150:3300/0 conn(0x7ff5d4160810 0x7ff5d406d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:18.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 -- 172.21.15.37:0/616591958 >> 172.21.15.37:0/616591958 conn(0x7ff5d40ccac0 msgr2=0x7ff5d40cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:18.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 -- 172.21.15.37:0/616591958 shutdown_connections 2024-09-21T06:30:18.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.150+0000 7ff5dc000640 1 -- 172.21.15.37:0/616591958 wait complete. 2024-09-21T06:30:18.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.152+0000 7ff5dc000640 1 Processor -- start 2024-09-21T06:30:18.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.152+0000 7ff5dc000640 1 -- start start 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d40ca690 0x7ff5d414f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d4160c20 0x7ff5d40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d406e0f0 con 0x7ff5d4160810 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d406df70 con 0x7ff5d4160c20 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5dc000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d406e270 con 0x7ff5d40ca690 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5d8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:18.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5d8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60572/0 (socket says 172.21.15.37:60572) 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5d8c00640 1 -- 172.21.15.37:0/2763829713 learned_addr learned my addr 172.21.15.37:0/2763829713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5d9600640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d40ca690 0x7ff5d414f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.154+0000 7ff5da000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d4160c20 0x7ff5d40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 -- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d40ca690 msgr2=0x7ff5d414f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d40ca690 0x7ff5d414f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 -- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d4160c20 msgr2=0x7ff5d40c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d4160c20 0x7ff5d40c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 -- 172.21.15.37:0/2763829713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d40c86b0 con 0x7ff5d4160810 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d8c00640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e7c0 tx=0x7ff5c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000cc60 con 0x7ff5d4160810 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c0010430 con 0x7ff5d4160810 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d40c8990 con 0x7ff5d4160810 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0010860 con 0x7ff5d4160810 2024-09-21T06:30:18.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.155+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d40c8f50 con 0x7ff5d4160810 2024-09-21T06:30:18.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.156+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d4160100 con 0x7ff5d4160810 2024-09-21T06:30:18.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.158+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5c0010a00 con 0x7ff5d4160810 2024-09-21T06:30:18.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.159+0000 7ff5d1600640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:18.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.159+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6417+0+0 (secure 0 0 0) 0x7ff5c009b9a0 con 0x7ff5d4160810 2024-09-21T06:30:18.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.160+0000 7ff5d9600640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:18.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.160+0000 7ff5d9600640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8007fd0 tx=0x7ff5c8001370 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:18.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.164+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5c00634b0 con 0x7ff5d4160810 2024-09-21T06:30:18.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:18.359+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5d40b5f20 con 0x7ff5d4160810 2024-09-21T06:30:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.227+0000 7ff5d1600640 1 -- 172.21.15.37:0/2763829713 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7ff5c0067ee0 con 0x7ff5d4160810 2024-09-21T06:30:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:30:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 msgr2=0x7ff5b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8007fd0 tx=0x7ff5c8001370 comp rx=0 tx=0).stop 2024-09-21T06:30:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 msgr2=0x7ff5d40c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e7c0 tx=0x7ff5c000ec80 comp rx=0 tx=0).stop 2024-09-21T06:30:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 shutdown_connections 2024-09-21T06:30:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d4160810 0x7ff5d40c3b30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d4160c20 0x7ff5d40c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 --2- 172.21.15.37:0/2763829713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d40ca690 0x7ff5d414f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.231+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 >> 172.21.15.37:0/2763829713 conn(0x7ff5d40ccac0 msgr2=0x7ff5d40afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.232+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 shutdown_connections 2024-09-21T06:30:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.232+0000 7ff5dc000640 1 -- 172.21.15.37:0/2763829713 wait complete. 2024-09-21T06:30:19.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 -- 172.21.15.37:0/3129010527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 msgr2=0x7f2dcc160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 --2- 172.21.15.37:0/3129010527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc160100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0c70c0 tx=0x7f2dc002a560 comp rx=0 tx=0).stop 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 -- 172.21.15.37:0/3129010527 shutdown_connections 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 --2- 172.21.15.37:0/3129010527 >> v2:172.21.15.150:3300/0 conn(0x7f2dcc06ef30 0x7f2dcc06f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 --2- 172.21.15.37:0/3129010527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 0x7f2dcc160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 --2- 172.21.15.37:0/3129010527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc160100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 -- 172.21.15.37:0/3129010527 >> 172.21.15.37:0/3129010527 conn(0x7f2dcc0ccac0 msgr2=0x7f2dcc0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 -- 172.21.15.37:0/3129010527 shutdown_connections 2024-09-21T06:30:19.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.500+0000 7f2dd3600640 1 -- 172.21.15.37:0/3129010527 wait complete. 2024-09-21T06:30:19.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.503+0000 7f2dd3600640 1 Processor -- start 2024-09-21T06:30:19.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.503+0000 7f2dd3600640 1 -- start start 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc06ef30 0x7f2dcc1efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 0x7f2dcc0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc06fc10 con 0x7f2dcc0ca690 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc06fa90 con 0x7f2dcc06ef30 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.504+0000 7f2dd3600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2dcc06fd90 con 0x7f2dcc160640 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60586/0 (socket says 172.21.15.37:60586) 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 -- 172.21.15.37:0/249641667 learned_addr learned my addr 172.21.15.37:0/249641667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dd0c00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc06ef30 0x7f2dcc1efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:19.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dd1600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 0x7f2dcc0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:19.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 -- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 msgr2=0x7f2dcc0b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:19.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 0x7f2dcc0b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 -- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc06ef30 msgr2=0x7f2dcc1efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc06ef30 0x7f2dcc1efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 -- 172.21.15.37:0/249641667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc0c6920 con 0x7f2dcc0ca690 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.505+0000 7f2dcbe00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2db800d740 tx=0x7f2db800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.506+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8017070 con 0x7f2dcc0ca690 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.506+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db8009d20 con 0x7f2dcc0ca690 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.506+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80106c0 con 0x7f2dcc0ca690 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.506+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc0b3700 con 0x7f2dcc0ca690 2024-09-21T06:30:19.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.506+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc0b3cc0 con 0x7f2dcc0ca690 2024-09-21T06:30:19.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.507+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc160100 con 0x7f2dcc0ca690 2024-09-21T06:30:19.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.508+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2db801a070 con 0x7f2dcc0ca690 2024-09-21T06:30:19.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.509+0000 7f2dc8c00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 0x7f2dac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:19.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.509+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5979+0+0 (secure 0 0 0) 0x7f2db809af80 con 0x7f2dcc0ca690 2024-09-21T06:30:19.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.510+0000 7f2dd0c00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 0x7f2dac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:19.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.510+0000 7f2dd0c00640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 0x7f2dac07b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0033000 tx=0x7f2dc0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:19.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.514+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2db8062c50 con 0x7f2dcc0ca690 2024-09-21T06:30:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:19.719+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2dcc0b5f20 con 0x7f2dcc0ca690 2024-09-21T06:30:19.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:30:19.728 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:30:20.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.229+0000 7f2dc8c00640 1 -- 172.21.15.37:0/249641667 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7f2db8067680 con 0x7f2dcc0ca690 2024-09-21T06:30:20.231 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:30:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 msgr2=0x7f2dac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 0x7f2dac07b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0033000 tx=0x7f2dc0004730 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 msgr2=0x7f2dcc0b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2db800d740 tx=0x7f2db800dc00 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 shutdown_connections 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2dac078b90 0x7f2dac07b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0ca690 0x7f2dcc0b44c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc160640 0x7f2dcc0b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 --2- 172.21.15.37:0/249641667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc06ef30 0x7f2dcc1efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 >> 172.21.15.37:0/249641667 conn(0x7f2dcc0ccac0 msgr2=0x7f2dcc0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 shutdown_connections 2024-09-21T06:30:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.232+0000 7f2dd3600640 1 -- 172.21.15.37:0/249641667 wait complete. 2024-09-21T06:30:20.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 -- 172.21.15.37:0/1333799416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 msgr2=0x7f6e84004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/1333799416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6e84004060 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e90006fb0 tx=0x7f6e84005830 comp rx=0 tx=0).stop 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea4e00640 1 -- 172.21.15.37:0/1333799416 reap_dead start 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 -- 172.21.15.37:0/1333799416 shutdown_connections 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 -- 172.21.15.37:0/1333799416 >> 172.21.15.37:0/1333799416 conn(0x7f6ea00b1b90 msgr2=0x7f6ea00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 -- 172.21.15.37:0/1333799416 shutdown_connections 2024-09-21T06:30:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.502+0000 7f6ea6e00640 1 -- 172.21.15.37:0/1333799416 wait complete. 2024-09-21T06:30:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.505+0000 7f6ea6e00640 1 Processor -- start 2024-09-21T06:30:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.505+0000 7f6ea6e00640 1 -- start start 2024-09-21T06:30:20.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea00c5e20 0x7f6ea00caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea00caf40 0x7f6ea00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea0073320 con 0x7f6e84003c10 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea00731a0 con 0x7f6ea00caf40 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6ea6e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea00734a0 con 0x7f6ea00c5e20 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60598/0 (socket says 172.21.15.37:60598) 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.506+0000 7f6e9fe00640 1 -- 172.21.15.37:0/4110657238 learned_addr learned my addr 172.21.15.37:0/4110657238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9f400640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea00c5e20 0x7f6ea00caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6ea4e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea00caf40 0x7f6ea00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 -- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea00c5e20 msgr2=0x7f6ea00caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea00c5e20 0x7f6ea00caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 -- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea00caf40 msgr2=0x7f6ea00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea00caf40 0x7f6ea00b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 -- 172.21.15.37:0/4110657238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea01f5b20 con 0x7f6e84003c10 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.507+0000 7f6e9fe00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e90031040 tx=0x7f6e90006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.508+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90003a00 con 0x7f6e84003c10 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.508+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e90003c90 con 0x7f6e84003c10 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.508+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90003e60 con 0x7f6e84003c10 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.508+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea01f5e00 con 0x7f6e84003c10 2024-09-21T06:30:20.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.508+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea01f63c0 con 0x7f6e84003c10 2024-09-21T06:30:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.509+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea00741f0 con 0x7f6e84003c10 2024-09-21T06:30:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.510+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e90035020 con 0x7f6e84003c10 2024-09-21T06:30:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.511+0000 7f6e97e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 0x7f6e8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.511+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f6e900bb300 con 0x7f6e84003c10 2024-09-21T06:30:20.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.511+0000 7f6e9f400640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 0x7f6e8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.512+0000 7f6e9f400640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 0x7f6e8007b0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:20.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.516+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e90047030 con 0x7f6e84003c10 2024-09-21T06:30:20.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.712+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f6ea00ce2d0 con 0x7f6e84003c10 2024-09-21T06:30:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.716+0000 7f6e97e00640 1 -- 172.21.15.37:0/4110657238 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f6e9003e070 con 0x7f6e84003c10 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 msgr2=0x7f6e8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 0x7f6e8007b0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c000ed0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 msgr2=0x7f6ea01f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e90031040 tx=0x7f6e90006c40 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 shutdown_connections 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6e80078b90 0x7f6e8007b0a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea00caf40 0x7f6ea00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea00c5e20 0x7f6ea00caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 --2- 172.21.15.37:0/4110657238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84003c10 0x7f6ea01f5510 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 >> 172.21.15.37:0/4110657238 conn(0x7f6ea00b1b90 msgr2=0x7f6ea015c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.719+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 shutdown_connections 2024-09-21T06:30:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.720+0000 7f6ea6e00640 1 -- 172.21.15.37:0/4110657238 wait complete. 2024-09-21T06:30:20.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 -- 172.21.15.37:0/1505020866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5368004800 msgr2=0x7f5368004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 --2- 172.21.15.37:0/1505020866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5368004800 0x7f5368004c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f53840cd8c0 tx=0x7f5368005830 comp rx=0 tx=0).stop 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 -- 172.21.15.37:0/1505020866 shutdown_connections 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 --2- 172.21.15.37:0/1505020866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5368004800 0x7f5368004c10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 -- 172.21.15.37:0/1505020866 >> 172.21.15.37:0/1505020866 conn(0x7f53840cca40 msgr2=0x7f53840cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 -- 172.21.15.37:0/1505020866 shutdown_connections 2024-09-21T06:30:20.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.972+0000 7f5389000640 1 -- 172.21.15.37:0/1505020866 wait complete. 2024-09-21T06:30:20.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.975+0000 7f5389000640 1 Processor -- start 2024-09-21T06:30:20.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.975+0000 7f5389000640 1 -- start start 2024-09-21T06:30:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5368004800 0x7f53841f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 0x7f53840b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f538406e180 con 0x7f538415eb90 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f538406e000 con 0x7f5384158e50 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.976+0000 7f5389000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f53840cbc10 con 0x7f5368004800 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5368004800 0x7f53841f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60614/0 (socket says 172.21.15.37:60614) 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 -- 172.21.15.37:0/4211121050 learned_addr learned my addr 172.21.15.37:0/4211121050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5382800640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 0x7f53840b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 -- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5368004800 msgr2=0x7f53841f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5368004800 0x7f53841f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 -- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 msgr2=0x7f53840b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 0x7f53840b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 -- 172.21.15.37:0/4211121050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384159260 con 0x7f538415eb90 2024-09-21T06:30:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5382800640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 0x7f53840b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.977+0000 7f5381400640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5370009dc0 tx=0x7f5370003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.978+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370038040 con 0x7f538415eb90 2024-09-21T06:30:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.978+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5370041070 con 0x7f538415eb90 2024-09-21T06:30:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.978+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370005c90 con 0x7f538415eb90 2024-09-21T06:30:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.978+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53840b2f40 con 0x7f538415eb90 2024-09-21T06:30:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.978+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53840b3500 con 0x7f538415eb90 2024-09-21T06:30:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.979+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53840b3740 con 0x7f538415eb90 2024-09-21T06:30:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.980+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5370003c70 con 0x7f538415eb90 2024-09-21T06:30:20.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.981+0000 7f537a000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 0x7f536407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:20.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.981+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f53700bb800 con 0x7f538415eb90 2024-09-21T06:30:20.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.982+0000 7f5381e00640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 0x7f536407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:20.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.982+0000 7f5381e00640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 0x7f536407b0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f5374001ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:20.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:20.986+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53840b3740 con 0x7f538415eb90 2024-09-21T06:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:21.264+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7f53840cc170 con 0x7f538415eb90 2024-09-21T06:30:22.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.242+0000 7f537a000640 1 -- 172.21.15.37:0/4211121050 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v144) ==== 136+0+0 (secure 0 0 0) 0x7f537004b020 con 0x7f538415eb90 2024-09-21T06:30:22.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 msgr2=0x7f536407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 0x7f536407b0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f5374001ab0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 msgr2=0x7f5384158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5370009dc0 tx=0x7f5370003820 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 shutdown_connections 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5364078bc0 0x7f536407b0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538415eb90 0x7f5384158910 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5384158e50 0x7f53840b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 --2- 172.21.15.37:0/4211121050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5368004800 0x7f53841f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 >> 172.21.15.37:0/4211121050 conn(0x7f53840cca40 msgr2=0x7f53840cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 shutdown_connections 2024-09-21T06:30:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.246+0000 7f5389000640 1 -- 172.21.15.37:0/4211121050 wait complete. 2024-09-21T06:30:22.274 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:30:22.274 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:30:22.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 -- 172.21.15.37:0/797801443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d7415e7c0 msgr2=0x7f7d7406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 --2- 172.21.15.37:0/797801443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d7415e7c0 0x7f7d7406ddf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7d60005fd0 tx=0x7f7d6002a530 comp rx=0 tx=0).stop 2024-09-21T06:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 -- 172.21.15.37:0/797801443 shutdown_connections 2024-09-21T06:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 --2- 172.21.15.37:0/797801443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d7415e7c0 0x7f7d7406ddf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 --2- 172.21.15.37:0/797801443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d740cb750 0x7f7d740af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.503+0000 7f7d7c000640 1 -- 172.21.15.37:0/797801443 >> 172.21.15.37:0/797801443 conn(0x7f7d740cca40 msgr2=0x7f7d740cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.504+0000 7f7d7c000640 1 -- 172.21.15.37:0/797801443 shutdown_connections 2024-09-21T06:30:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.504+0000 7f7d7c000640 1 -- 172.21.15.37:0/797801443 wait complete. 2024-09-21T06:30:22.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.506+0000 7f7d7c000640 1 Processor -- start 2024-09-21T06:30:22.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.506+0000 7f7d7c000640 1 -- start start 2024-09-21T06:30:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d7406e330 0x7f7d740c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d7415e7c0 0x7f7d740737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d74162ab0 con 0x7f7d740c97c0 2024-09-21T06:30:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d740cef20 con 0x7f7d7415e7c0 2024-09-21T06:30:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.507+0000 7f7d7c000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7d74162c30 con 0x7f7d7406e330 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d78c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d7415e7c0 0x7f7d740737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d79600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d7406e330 0x7f7d740c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60646/0 (socket says 172.21.15.37:60646) 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 -- 172.21.15.37:0/3500553215 learned_addr learned my addr 172.21.15.37:0/3500553215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 -- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d7406e330 msgr2=0x7f7d740c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d7406e330 0x7f7d740c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 -- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d7415e7c0 msgr2=0x7f7d740737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d7415e7c0 0x7f7d740737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.508+0000 7f7d7a000640 1 -- 172.21.15.37:0/3500553215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7409ed90 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d7a000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00b9e0 tx=0x7f7d6c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003b70 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d6c003e00 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00cb80 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7409f070 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.509+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d740a1060 con 0x7f7d740c97c0 2024-09-21T06:30:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.510+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d740b5ba0 con 0x7f7d740c97c0 2024-09-21T06:30:22.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.511+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d6c004000 con 0x7f7d740c97c0 2024-09-21T06:30:22.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.513+0000 7f7d71600640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 0x7f7d5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.513+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f7d6c09aa10 con 0x7f7d740c97c0 2024-09-21T06:30:22.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.514+0000 7f7d79600640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 0x7f7d5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.514+0000 7f7d79600640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 0x7f7d5407b0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d68004630 tx=0x7f7d6800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.518+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d6c0628b0 con 0x7f7d740c97c0 2024-09-21T06:30:22.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.718+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d74163120 con 0x7f7d740c97c0 2024-09-21T06:30:22.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.719+0000 7f7d71600640 1 -- 172.21.15.37:0/3500553215 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13577 (secure 0 0 0) 0x7f7d6c0672e0 con 0x7f7d740c97c0 2024-09-21T06:30:22.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:22.721 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:20.228191+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6834/2753796335":"2024-09-22T06:30:16.874226+0000","172.21.15.150:6835/337579449":"2024-09-22T06:30:16.874259+0000","172.21.15.37:6832/2753796335":"2024-09-22T06:30:16.874226+0000","172.21.15.37:6835/2364291417":"2024-09-22T06:30:16.874249+0000","172.21.15.150:6837/337579449":"2024-09-22T06:30:16.874259+0000","172.21.15.37:6833/2364291417":"2024-09-22T06:30:16.874249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 msgr2=0x7f7d5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 0x7f7d5407b0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d68004630 tx=0x7f7d6800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 msgr2=0x7f7d74073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00b9e0 tx=0x7f7d6c00bea0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 shutdown_connections 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d54078bc0 0x7f7d5407b0d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d740c97c0 0x7f7d74073d20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d7415e7c0 0x7f7d740737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 --2- 172.21.15.37:0/3500553215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d7406e330 0x7f7d740c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 >> 172.21.15.37:0/3500553215 conn(0x7f7d740cca40 msgr2=0x7f7d74153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 shutdown_connections 2024-09-21T06:30:22.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.721+0000 7f7d7c000640 1 -- 172.21.15.37:0/3500553215 wait complete. 2024-09-21T06:30:22.747 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:30:22.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:30:22.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.975+0000 7f4021600640 1 -- 172.21.15.37:0/3070948202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4000004d40 msgr2=0x7f40000051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.975+0000 7f4021600640 1 --2- 172.21.15.37:0/3070948202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4000004d40 0x7f40000051f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4014001dc0 tx=0x7f4000005830 comp rx=0 tx=0).stop 2024-09-21T06:30:22.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.975+0000 7f401b400640 1 -- 172.21.15.37:0/3070948202 reap_dead start 2024-09-21T06:30:22.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.976+0000 7f4021600640 1 -- 172.21.15.37:0/3070948202 shutdown_connections 2024-09-21T06:30:22.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.976+0000 7f4021600640 1 -- 172.21.15.37:0/3070948202 >> 172.21.15.37:0/3070948202 conn(0x7f401c0cca40 msgr2=0x7f401c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:22.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.976+0000 7f4021600640 1 -- 172.21.15.37:0/3070948202 shutdown_connections 2024-09-21T06:30:22.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.976+0000 7f4021600640 1 -- 172.21.15.37:0/3070948202 wait complete. 2024-09-21T06:30:22.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.978+0000 7f4021600640 1 Processor -- start 2024-09-21T06:30:22.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.978+0000 7f4021600640 1 -- start start 2024-09-21T06:30:22.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4000004d40 0x7f401c1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 0x7f401c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0cc1a0 con 0x7f401c0cf7f0 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0cc020 con 0x7f4000004d40 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.979+0000 7f4021600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0cc320 con 0x7f401c157690 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60658/0 (socket says 172.21.15.37:60658) 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 -- 172.21.15.37:0/2850206297 learned_addr learned my addr 172.21.15.37:0/2850206297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401aa00640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4000004d40 0x7f401c1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401b400640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 0x7f401c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 -- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 msgr2=0x7f401c157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 0x7f401c157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 -- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4000004d40 msgr2=0x7f401c1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4000004d40 0x7f401c1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401a000640 1 -- 172.21.15.37:0/2850206297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c1f9040 con 0x7f401c0cf7f0 2024-09-21T06:30:22.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.980+0000 7f401b400640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 0x7f401c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:22.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f401a000640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4008006680 tx=0x7f40080066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:22.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008015070 con 0x7f401c0cf7f0 2024-09-21T06:30:22.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4008011900 con 0x7f401c0cf7f0 2024-09-21T06:30:22.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008011d50 con 0x7f401c0cf7f0 2024-09-21T06:30:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c1f9320 con 0x7f401c0cf7f0 2024-09-21T06:30:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.981+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c09f4e0 con 0x7f401c0cf7f0 2024-09-21T06:30:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.982+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c156c80 con 0x7f401c0cf7f0 2024-09-21T06:30:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.983+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4008018070 con 0x7f401c0cf7f0 2024-09-21T06:30:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.984+0000 7f4012a00640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 0x7f3ffc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.984+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f400809bb30 con 0x7f401c0cf7f0 2024-09-21T06:30:22.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.985+0000 7f401aa00640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 0x7f3ffc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:22.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.985+0000 7f401aa00640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 0x7f3ffc07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f401c15a730 tx=0x7f400c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:22.989+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4008063f90 con 0x7f401c0cf7f0 2024-09-21T06:30:23.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.186+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f401c06f720 con 0x7f401c0cf7f0 2024-09-21T06:30:23.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.186+0000 7f4012a00640 1 -- 172.21.15.37:0/2850206297 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f4008063890 con 0x7f401c0cf7f0 2024-09-21T06:30:23.188 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 msgr2=0x7f3ffc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 0x7f3ffc07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f401c15a730 tx=0x7f400c005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 msgr2=0x7f401c09efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4008006680 tx=0x7f40080066b0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 shutdown_connections 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ffc078bc0 0x7f3ffc07b0d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cf7f0 0x7f401c09efa0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f401c157690 0x7f401c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 --2- 172.21.15.37:0/2850206297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4000004d40 0x7f401c1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.188+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 >> 172.21.15.37:0/2850206297 conn(0x7f401c0cca40 msgr2=0x7f401c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:23.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.189+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 shutdown_connections 2024-09-21T06:30:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.189+0000 7f4021600640 1 -- 172.21.15.37:0/2850206297 wait complete. 2024-09-21T06:30:23.214 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-21T06:30:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 -- 172.21.15.37:0/1039239311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53b4005280 msgr2=0x7f53b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1039239311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53b4005280 0x7f53b400b9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53b4005830 comp rx=0 tx=0).stop 2024-09-21T06:30:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 -- 172.21.15.37:0/1039239311 shutdown_connections 2024-09-21T06:30:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1039239311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53b4005280 0x7f53b400b9c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 -- 172.21.15.37:0/1039239311 >> 172.21.15.37:0/1039239311 conn(0x7f53d00cca40 msgr2=0x7f53d00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 -- 172.21.15.37:0/1039239311 shutdown_connections 2024-09-21T06:30:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.441+0000 7f53d5a00640 1 -- 172.21.15.37:0/1039239311 wait complete. 2024-09-21T06:30:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.443+0000 7f53d5a00640 1 Processor -- start 2024-09-21T06:30:23.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.443+0000 7f53d5a00640 1 -- start start 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53b4005280 0x7f53d01f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d00744a0 0x7f53d01f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0071000 con 0x7f53d0159d60 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0070e80 con 0x7f53b4005280 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53d5a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f53d006f710 con 0x7f53d00744a0 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60684/0 (socket says 172.21.15.37:60684) 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 -- 172.21.15.37:0/1964279024 learned_addr learned my addr 172.21.15.37:0/1964279024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53cf400640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d00744a0 0x7f53d01f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53cea00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53b4005280 0x7f53d01f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 -- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d00744a0 msgr2=0x7f53d01f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d00744a0 0x7f53d01f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 -- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53b4005280 msgr2=0x7f53d01f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53b4005280 0x7f53d01f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.445+0000 7f53c7e00640 1 -- 172.21.15.37:0/1964279024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d00748b0 con 0x7f53d0159d60 2024-09-21T06:30:23.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53c7e00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f53b8006600 tx=0x7f53b80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8015070 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53b8011900 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d00b5100 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8011d50 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.446+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d00b56c0 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.447+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d006ddf0 con 0x7f53d0159d60 2024-09-21T06:30:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.448+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53b8018070 con 0x7f53d0159d60 2024-09-21T06:30:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.449+0000 7f53c6a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 0x7f53ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.449+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f53b809b8f0 con 0x7f53d0159d60 2024-09-21T06:30:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.450+0000 7f53cea00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 0x7f53ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:23.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.450+0000 7f53cea00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 0x7f53ac07b0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c0006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:23.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.454+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53d00748b0 con 0x7f53d0159d60 2024-09-21T06:30:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:23.729+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7f53d00b5f40 con 0x7f53d0159d60 2024-09-21T06:30:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.250+0000 7f53c6a00640 1 -- 172.21.15.37:0/1964279024 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v146) ==== 146+0+0 (secure 0 0 0) 0x7f53b80637a0 con 0x7f53d0159d60 2024-09-21T06:30:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 msgr2=0x7f53ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 0x7f53ac07b0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c0006b10 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 msgr2=0x7f53d0073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f53b8006600 tx=0x7f53b80039c0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 shutdown_connections 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f53ac078bc0 0x7f53ac07b0d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d0159d60 0x7f53d0073f60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d00744a0 0x7f53d01f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 --2- 172.21.15.37:0/1964279024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53b4005280 0x7f53d01f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 >> 172.21.15.37:0/1964279024 conn(0x7f53d00cca40 msgr2=0x7f53d00aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 shutdown_connections 2024-09-21T06:30:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.253+0000 7f53d5a00640 1 -- 172.21.15.37:0/1964279024 wait complete. 2024-09-21T06:30:24.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 -- 172.21.15.37:0/197479465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 msgr2=0x7fe01406e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 --2- 172.21.15.37:0/197479465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe01406e570 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe00002a540 comp rx=0 tx=0).stop 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 -- 172.21.15.37:0/197479465 shutdown_connections 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 --2- 172.21.15.37:0/197479465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe01406e570 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 --2- 172.21.15.37:0/197479465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01415dfe0 0x7fe014160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.504+0000 7fe01a200640 1 -- 172.21.15.37:0/197479465 >> 172.21.15.37:0/197479465 conn(0x7fe01406f940 msgr2=0x7fe0140b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.505+0000 7fe01a200640 1 -- 172.21.15.37:0/197479465 shutdown_connections 2024-09-21T06:30:24.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.505+0000 7fe01a200640 1 -- 172.21.15.37:0/197479465 wait complete. 2024-09-21T06:30:24.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.507+0000 7fe01a200640 1 Processor -- start 2024-09-21T06:30:24.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.507+0000 7fe01a200640 1 -- start start 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 0x7fe0141386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 0x7fe0141f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0140b4c50 con 0x7fe01406e160 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0140b4ad0 con 0x7fe01415dfe0 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.508+0000 7fe01a200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe0140b4dd0 con 0x7fe0140cc8d0 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44224/0 (socket says 172.21.15.37:44224) 2024-09-21T06:30:24.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 -- 172.21.15.37:0/477116259 learned_addr learned my addr 172.21.15.37:0/477116259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:24.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe013e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 0x7fe0141386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe019800640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 0x7fe0141f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 -- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 msgr2=0x7fe0141386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 0x7fe0141386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 -- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 msgr2=0x7fe0141f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 0x7fe0141f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 -- 172.21.15.37:0/477116259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe014074430 con 0x7fe01406e160 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe013e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 0x7fe0141386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe019800640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 0x7fe0141f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.509+0000 7fe018e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c970 tx=0x7fe00800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.510+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007bb0 con 0x7fe01406e160 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.510+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe008007e40 con 0x7fe01406e160 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.510+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe014072e50 con 0x7fe01406e160 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.510+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00800f690 con 0x7fe01406e160 2024-09-21T06:30:24.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.510+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe014073410 con 0x7fe01406e160 2024-09-21T06:30:24.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.511+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0140b52f0 con 0x7fe01406e160 2024-09-21T06:30:24.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.512+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe00800f830 con 0x7fe01406e160 2024-09-21T06:30:24.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.513+0000 7fe010c00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 0x7fdff407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.513+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7fe00809b5a0 con 0x7fe01406e160 2024-09-21T06:30:24.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.513+0000 7fe013e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 0x7fdff407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.514+0000 7fe013e00640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 0x7fdff407b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe000033000 tx=0x7fe000004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.517+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe008014030 con 0x7fe01406e160 2024-09-21T06:30:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.713+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe0140cc0e0 con 0x7fe01406e160 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.714+0000 7fe010c00640 1 -- 172.21.15.37:0/477116259 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7fe008063a00 con 0x7fe01406e160 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:30:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:30:24.717 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:30:24.718 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:30:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 msgr2=0x7fdff407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 0x7fdff407b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe000033000 tx=0x7fe000004730 comp rx=0 tx=0).stop 2024-09-21T06:30:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 msgr2=0x7fe014073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c970 tx=0x7fe00800ce30 comp rx=0 tx=0).stop 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 shutdown_connections 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdff4078b90 0x7fdff407b0a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01415dfe0 0x7fe0141f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0140cc8d0 0x7fe0141386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.716+0000 7fe01a200640 1 --2- 172.21.15.37:0/477116259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe01406e160 0x7fe014073ef0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.717+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 >> 172.21.15.37:0/477116259 conn(0x7fe01406f940 msgr2=0x7fe0140cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.717+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 shutdown_connections 2024-09-21T06:30:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.717+0000 7fe01a200640 1 -- 172.21.15.37:0/477116259 wait complete. 2024-09-21T06:30:24.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 -- 172.21.15.37:0/3990992706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 msgr2=0x7fc6e4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 --2- 172.21.15.37:0/3990992706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e4074620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 -- 172.21.15.37:0/3990992706 shutdown_connections 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 --2- 172.21.15.37:0/3990992706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e406e500 0x7fc6e40cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 --2- 172.21.15.37:0/3990992706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e4074620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 -- 172.21.15.37:0/3990992706 >> 172.21.15.37:0/3990992706 conn(0x7fc6e40b1b90 msgr2=0x7fc6e40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 -- 172.21.15.37:0/3990992706 shutdown_connections 2024-09-21T06:30:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.971+0000 7fc6ec000640 1 -- 172.21.15.37:0/3990992706 wait complete. 2024-09-21T06:30:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.974+0000 7fc6ec000640 1 Processor -- start 2024-09-21T06:30:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.974+0000 7fc6ec000640 1 -- start start 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 0x7fc6e40c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e40cafd0 0x7fc6e415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e40cf7d0 con 0x7fc6e4156a50 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e40cf650 con 0x7fc6e4074b60 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.975+0000 7fc6ec000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e40cf950 con 0x7fc6e40cafd0 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 0x7fc6e40c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 0x7fc6e40c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:52310/0 (socket says 172.21.15.37:52310) 2024-09-21T06:30:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e9600640 1 -- 172.21.15.37:0/870745272 learned_addr learned my addr 172.21.15.37:0/870745272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44238/0 (socket says 172.21.15.37:44238) 2024-09-21T06:30:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6ea000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e40cafd0 0x7fc6e415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 -- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e40cafd0 msgr2=0x7fc6e415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e40cafd0 0x7fc6e415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 -- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 msgr2=0x7fc6e40c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 0x7fc6e40c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.976+0000 7fc6e8c00640 1 -- 172.21.15.37:0/870745272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e41f5030 con 0x7fc6e4156a50 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6e8c00640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c390 tx=0x7fc6d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0017070 con 0x7fc6e4156a50 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6d00053c0 con 0x7fc6e4156a50 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e40c6370 con 0x7fc6e4156a50 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0003850 con 0x7fc6e4156a50 2024-09-21T06:30:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.977+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e41f55e0 con 0x7fc6e4156a50 2024-09-21T06:30:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.978+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e40cef00 con 0x7fc6e4156a50 2024-09-21T06:30:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.979+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6d001a070 con 0x7fc6e4156a50 2024-09-21T06:30:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.980+0000 7fc6e1600640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 0x7fc6c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.980+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7fc6d0099df0 con 0x7fc6e4156a50 2024-09-21T06:30:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.981+0000 7fc6e9600640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 0x7fc6c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.982+0000 7fc6e9600640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 0x7fc6c407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009230 tx=0x7fc6d8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:24.985+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6d0061d20 con 0x7fc6e4156a50 2024-09-21T06:30:25.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.184+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc6e415a7c0 con 0x7fc6e4156a50 2024-09-21T06:30:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.188+0000 7fc6e1600640 1 -- 172.21.15.37:0/870745272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fc6d0066750 con 0x7fc6e4156a50 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 msgr2=0x7fc6c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 0x7fc6c407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009230 tx=0x7fc6d8004730 comp rx=0 tx=0).stop 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 msgr2=0x7fc6e40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c390 tx=0x7fc6d000c850 comp rx=0 tx=0).stop 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 shutdown_connections 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6c4078bc0 0x7fc6c407b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e4156a50 0x7fc6e40caa90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6e40cafd0 0x7fc6e415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 --2- 172.21.15.37:0/870745272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e4074b60 0x7fc6e40c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 >> 172.21.15.37:0/870745272 conn(0x7fc6e40b1b90 msgr2=0x7fc6e40b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 shutdown_connections 2024-09-21T06:30:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.191+0000 7fc6ec000640 1 -- 172.21.15.37:0/870745272 wait complete. 2024-09-21T06:30:25.215 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:30:25.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:30:25.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:30:25.386 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:30:25.386 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:25.386 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:25.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:30:25.421 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:30:25.422 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:25.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:30:25.458 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:25.458 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:25.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:30:25.497 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:30:25.497 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:25.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:30:25.535 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:30:25.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.760+0000 7f89ff200640 1 -- 172.21.15.37:0/658610165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a0015dfe0 msgr2=0x7f8a00160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.760+0000 7f89ff200640 1 --2- 172.21.15.37:0/658610165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a0015dfe0 0x7f8a00160a40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8a00074f80 tx=0x7f89f002a4a0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.760+0000 7f89ff200640 1 -- 172.21.15.37:0/658610165 shutdown_connections 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.760+0000 7f89ff200640 1 --2- 172.21.15.37:0/658610165 >> v2:172.21.15.150:3300/0 conn(0x7f8a000ccfe0 0x7f8a0006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.761+0000 7f89ff200640 1 --2- 172.21.15.37:0/658610165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a0015dfe0 0x7f8a00160a40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.761+0000 7f89ff200640 1 -- 172.21.15.37:0/658610165 >> 172.21.15.37:0/658610165 conn(0x7f8a0006f940 msgr2=0x7f8a000b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.761+0000 7f89ff200640 1 -- 172.21.15.37:0/658610165 shutdown_connections 2024-09-21T06:30:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.761+0000 7f89ff200640 1 -- 172.21.15.37:0/658610165 wait complete. 2024-09-21T06:30:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.763+0000 7f89ff200640 1 Processor -- start 2024-09-21T06:30:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.763+0000 7f89ff200640 1 -- start start 2024-09-21T06:30:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0006e160 0x7f8a0013ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0015dfe0 0x7f8a001431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b4b70 con 0x7f8a001f8bc0 2024-09-21T06:30:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b49f0 con 0x7f8a0006e160 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.764+0000 7f89ff200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b4cf0 con 0x7f8a0015dfe0 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44256/0 (socket says 172.21.15.37:44256) 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 -- 172.21.15.37:0/365380759 learned_addr learned my addr 172.21.15.37:0/365380759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fde00640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0006e160 0x7f8a0013ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fd400640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0015dfe0 0x7f8a001431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 -- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0015dfe0 msgr2=0x7f8a001431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0015dfe0 0x7f8a001431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 -- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0006e160 msgr2=0x7f8a0013ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0006e160 0x7f8a0013ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 -- 172.21.15.37:0/365380759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000cddc0 con 0x7f8a001f8bc0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.765+0000 7f89fe800640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f89f800cad0 tx=0x7f89f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.766+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800ae20 con 0x7f8a001f8bc0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.766+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f800f450 con 0x7f8a001f8bc0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.766+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800f8c0 con 0x7f8a001f8bc0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.766+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001f9310 con 0x7f8a001f8bc0 2024-09-21T06:30:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.766+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001f98d0 con 0x7f8a001f8bc0 2024-09-21T06:30:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.767+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000b5230 con 0x7f8a001f8bc0 2024-09-21T06:30:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.768+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89f800fa60 con 0x7f8a001f8bc0 2024-09-21T06:30:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.769+0000 7f89f6000640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 0x7f89e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.769+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7f89f809b5b0 con 0x7f8a001f8bc0 2024-09-21T06:30:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.770+0000 7f89fde00640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 0x7f89e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.771+0000 7f89fde00640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 0x7f89e007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f0033000 tx=0x7f89f0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:25.774+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89f8014030 con 0x7f8a001f8bc0 2024-09-21T06:30:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.007+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a001f90a0 con 0x7f8a001f8bc0 2024-09-21T06:30:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.008+0000 7f89f6000640 1 -- 172.21.15.37:0/365380759 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3917 (secure 0 0 0) 0x7f89f80634e0 con 0x7f8a001f8bc0 2024-09-21T06:30:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:26.010 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-21T06:30:17:877423+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/3563884417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3563884417},{"type":"v1","addr":"172.21.15.150:6836","nonce":3563884417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1985472783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1985472783},{"type":"v1","addr":"172.21.15.37:6833","nonce":1985472783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2359430257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2359430257},{"type":"v1","addr":"172.21.15.150:6837","nonce":2359430257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1708805924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1708805924},{"type":"v1","addr":"172.21.15.37:6835","nonce":1708805924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-09-21T06:30:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 msgr2=0x7f89e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 0x7f89e007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f0033000 tx=0x7f89f0004730 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 msgr2=0x7f8a00143720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f89f800cad0 tx=0x7f89f800cf90 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 shutdown_connections 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89e0078b90 0x7f89e007b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a001f8bc0 0x7f8a00143720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.010+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0015dfe0 0x7f8a001431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.011+0000 7f89ff200640 1 --2- 172.21.15.37:0/365380759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0006e160 0x7f8a0013ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.011+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 >> 172.21.15.37:0/365380759 conn(0x7f8a0006f940 msgr2=0x7f8a00073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.011+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 shutdown_connections 2024-09-21T06:30:26.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.011+0000 7f89ff200640 1 -- 172.21.15.37:0/365380759 wait complete. 2024-09-21T06:30:26.037 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:30:26.037 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:30:26.038 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:30:26.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:30:26.038 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:30:26.038 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:30:26.038 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:30:26.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:30:26.038 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:30:26.038 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:30:26.039 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:30:26.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:30:26.039 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:30:26.039 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:30:26.039 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:30:26.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:30:26.065 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:30:26.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:30:26.066 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:30:26.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:30:26.068 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:30:26.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:30:26.070 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:30:26.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:30:26.110 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:30:26.110 DEBUG:teuthology.parallel:result is None 2024-09-21T06:30:26.112 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:30:26.112 DEBUG:teuthology.parallel:result is None 2024-09-21T06:30:26.114 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:30:26.115 DEBUG:teuthology.parallel:result is None 2024-09-21T06:30:26.117 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:30:26.118 DEBUG:teuthology.parallel:result is None 2024-09-21T06:30:26.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:30:26.312 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:30:26.314 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:30:26.314 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:30:26.314 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:30:26.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 -- 172.21.15.37:0/2305630084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 msgr2=0x7fa91c0cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:26.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 --2- 172.21.15.37:0/2305630084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c0cf310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 -- 172.21.15.37:0/2305630084 shutdown_connections 2024-09-21T06:30:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 --2- 172.21.15.37:0/2305630084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c0cf310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 -- 172.21.15.37:0/2305630084 >> 172.21.15.37:0/2305630084 conn(0x7fa91c0d41e0 msgr2=0x7fa91c0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 -- 172.21.15.37:0/2305630084 shutdown_connections 2024-09-21T06:30:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.350+0000 7fa91b200640 1 -- 172.21.15.37:0/2305630084 wait complete. 2024-09-21T06:30:26.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.352+0000 7fa91b200640 1 Processor -- start 2024-09-21T06:30:26.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.352+0000 7fa91b200640 1 -- start start 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa91c0d2290 0x7fa91c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa91c15f6d0 0x7fa91c1e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0d2db0 con 0x7fa91c0cef00 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0d2f30 con 0x7fa91c15f6d0 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c0d30b0 con 0x7fa91c0d2290 2024-09-21T06:30:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa919e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa919e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44300/0 (socket says 172.21.15.37:44300) 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa919e00640 1 -- 172.21.15.37:0/4187055365 learned_addr learned my addr 172.21.15.37:0/4187055365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa911400640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa91c0d2290 0x7fa91c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.354+0000 7fa91a800640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa91c15f6d0 0x7fa91c1e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 -- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa91c0d2290 msgr2=0x7fa91c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa91c0d2290 0x7fa91c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 -- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa91c15f6d0 msgr2=0x7fa91c1e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa91c15f6d0 0x7fa91c1e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 -- 172.21.15.37:0/4187055365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1e89d0 con 0x7fa91c0cef00 2024-09-21T06:30:26.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa919e00640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c370 tx=0x7fa90c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c017070 con 0x7fa91c0cef00 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa90c0035f0 con 0x7fa91c0cef00 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003a40 con 0x7fa91c0cef00 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.355+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1e8cb0 con 0x7fa91c0cef00 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.356+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c0cb4e0 con 0x7fa91c0cef00 2024-09-21T06:30:26.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.356+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c0cf310 con 0x7fa91c0cef00 2024-09-21T06:30:26.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.358+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa90c01a070 con 0x7fa91c0cef00 2024-09-21T06:30:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.359+0000 7fa912a00640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 0x7fa8f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.359+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5507+0+0 (secure 0 0 0) 0x7fa90c099ed0 con 0x7fa91c0cef00 2024-09-21T06:30:26.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.360+0000 7fa911400640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 0x7fa8f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:26.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.360+0000 7fa911400640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 0x7fa8f807b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa904033000 tx=0x7fa904004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.363+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa91c15eae0 con 0x7fa91c0cef00 2024-09-21T06:30:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:26.569+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa91c0b8120 con 0x7fa91c0cef00 2024-09-21T06:30:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.270+0000 7fa912a00640 1 -- 172.21.15.37:0/4187055365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7fa90c061d70 con 0x7fa91c0cef00 2024-09-21T06:30:27.273 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 msgr2=0x7fa8f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 0x7fa8f807b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa904033000 tx=0x7fa904004730 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 msgr2=0x7fa91c1e8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c370 tx=0x7fa90c00c830 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 shutdown_connections 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa8f8078b90 0x7fa8f807b0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa91c15f6d0 0x7fa91c1e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa91c0d2290 0x7fa91c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.273+0000 7fa91b200640 1 --2- 172.21.15.37:0/4187055365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa91c0cef00 0x7fa91c1e8080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.274+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 >> 172.21.15.37:0/4187055365 conn(0x7fa91c0d41e0 msgr2=0x7fa91c0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:27.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.274+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 shutdown_connections 2024-09-21T06:30:27.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.274+0000 7fa91b200640 1 -- 172.21.15.37:0/4187055365 wait complete. 2024-09-21T06:30:27.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.537+0000 7fcd9ac00640 1 -- 172.21.15.37:0/1156518207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd94070030 msgr2=0x7fcd9406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.537+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/1156518207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd94070030 0x7fcd9406ddf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009830 tx=0x7fcd8002a520 comp rx=0 tx=0).stop 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 -- 172.21.15.37:0/1156518207 shutdown_connections 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/1156518207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9406e330 0x7fcd940cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/1156518207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd94070030 0x7fcd9406ddf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 -- 172.21.15.37:0/1156518207 >> 172.21.15.37:0/1156518207 conn(0x7fcd940cca40 msgr2=0x7fcd940cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 -- 172.21.15.37:0/1156518207 shutdown_connections 2024-09-21T06:30:27.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.538+0000 7fcd9ac00640 1 -- 172.21.15.37:0/1156518207 wait complete. 2024-09-21T06:30:27.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.540+0000 7fcd9ac00640 1 Processor -- start 2024-09-21T06:30:27.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.540+0000 7fcd9ac00640 1 -- start start 2024-09-21T06:30:27.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.541+0000 7fcd9ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:27.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.541+0000 7fcd9ac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd94070030 0x7fcd940b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:27.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd9ac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 0x7fcd940b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd9ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94070e80 con 0x7fcd9406e330 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd9ac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94071000 con 0x7fcd94074d90 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd9ac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcd940ce0d0 con 0x7fcd94070030 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44316/0 (socket says 172.21.15.37:44316) 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 -- 172.21.15.37:0/3487962782 learned_addr learned my addr 172.21.15.37:0/3487962782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93400640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd94070030 0x7fcd940b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd98c00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 0x7fcd940b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 -- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd94070030 msgr2=0x7fcd940b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd94070030 0x7fcd940b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 -- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 msgr2=0x7fcd940b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 0x7fcd940b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd93e00640 1 -- 172.21.15.37:0/3487962782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd941533c0 con 0x7fcd9406e330 2024-09-21T06:30:27.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.542+0000 7fcd98c00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 0x7fcd940b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd93e00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c410 tx=0x7fcd8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011790 con 0x7fcd9406e330 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd84017070 con 0x7fcd9406e330 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9415bfd0 con 0x7fcd9406e330 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011ca0 con 0x7fcd9406e330 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.543+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9415c2b0 con 0x7fcd9406e330 2024-09-21T06:30:27.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.544+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9415dda0 con 0x7fcd9406e330 2024-09-21T06:30:27.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.545+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd8401c460 con 0x7fcd9406e330 2024-09-21T06:30:27.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.546+0000 7fcd8be00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 0x7fcd7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:27.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.547+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fcd8409a880 con 0x7fcd9406e330 2024-09-21T06:30:27.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.547+0000 7fcd93400640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 0x7fcd7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:27.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.548+0000 7fcd93400640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 0x7fcd7407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd80034000 tx=0x7fcd80004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:27.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.551+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd9415dda0 con 0x7fcd9406e330 2024-09-21T06:30:27.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.820+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcd940ce660 con 0x7fcd9406e330 2024-09-21T06:30:27.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.821+0000 7fcd8be00640 1 -- 172.21.15.37:0/3487962782 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7fcd941533c0 con 0x7fcd9406e330 2024-09-21T06:30:27.823 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.824 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.825 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.827 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.828 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:30:27.829 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.831 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.832 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:30:27.833 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:30:27.834 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:27.835 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:30:27.836 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:27.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 msgr2=0x7fcd7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 0x7fcd7407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd80034000 tx=0x7fcd80004730 comp rx=0 tx=0).stop 2024-09-21T06:30:27.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 msgr2=0x7fcd94152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:27.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c410 tx=0x7fcd8400c8d0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 shutdown_connections 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd74078b90 0x7fcd7407b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd94074d90 0x7fcd940b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd94070030 0x7fcd940b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 --2- 172.21.15.37:0/3487962782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9406e330 0x7fcd94152e80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 >> 172.21.15.37:0/3487962782 conn(0x7fcd940cca40 msgr2=0x7fcd94068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 shutdown_connections 2024-09-21T06:30:27.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:27.823+0000 7fcd9ac00640 1 -- 172.21.15.37:0/3487962782 wait complete. 2024-09-21T06:30:27.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 -- 172.21.15.37:0/2333968966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0074cb0 msgr2=0x7f0ab0073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 --2- 172.21.15.37:0/2333968966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0074cb0 0x7f0ab0073dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0ab009e110 tx=0x7f0aa002a600 comp rx=0 tx=0).stop 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 -- 172.21.15.37:0/2333968966 shutdown_connections 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 --2- 172.21.15.37:0/2333968966 >> v2:172.21.15.150:3300/0 conn(0x7f0ab00cbc10 0x7f0ab00cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 --2- 172.21.15.37:0/2333968966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0074cb0 0x7f0ab0073dd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 -- 172.21.15.37:0/2333968966 >> 172.21.15.37:0/2333968966 conn(0x7f0ab00cca40 msgr2=0x7f0ab00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 -- 172.21.15.37:0/2333968966 shutdown_connections 2024-09-21T06:30:28.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.076+0000 7f0ab6800640 1 -- 172.21.15.37:0/2333968966 wait complete. 2024-09-21T06:30:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.079+0000 7f0ab6800640 1 Processor -- start 2024-09-21T06:30:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.079+0000 7f0ab6800640 1 -- start start 2024-09-21T06:30:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab0074cb0 0x7f0ab00cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab00cbc10 0x7f0ab00c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab006f360 con 0x7f0ab006ddf0 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab006f1e0 con 0x7f0ab00cbc10 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0ab6800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab006f4e0 con 0x7f0ab0074cb0 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44348/0 (socket says 172.21.15.37:44348) 2024-09-21T06:30:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.080+0000 7f0aaf400640 1 -- 172.21.15.37:0/458613858 learned_addr learned my addr 172.21.15.37:0/458613858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaea00640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab0074cb0 0x7f0ab00cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aafe00640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab00cbc10 0x7f0ab00c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 -- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab0074cb0 msgr2=0x7f0ab00cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab0074cb0 0x7f0ab00cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 -- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab00cbc10 msgr2=0x7f0ab00c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab00cbc10 0x7f0ab00c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 -- 172.21.15.37:0/458613858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab01f5580 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aaf400640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0034040 tx=0x7f0aa0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa003e030 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.081+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0aa0003e00 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.082+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0035040 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.082+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab00ca120 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.082+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab0158a90 con 0x7f0ab006ddf0 2024-09-21T06:30:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.083+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab006f990 con 0x7f0ab006ddf0 2024-09-21T06:30:28.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.084+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0aa0041030 con 0x7f0ab006ddf0 2024-09-21T06:30:28.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.085+0000 7f0aa7400640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 0x7f0a9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.085+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f0aa00bb410 con 0x7f0ab006ddf0 2024-09-21T06:30:28.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.085+0000 7f0aaea00640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 0x7f0a9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.086+0000 7f0aaea00640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 0x7f0a9007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005fd0 tx=0x7f0a9c005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.089+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0aa002ae20 con 0x7f0ab006ddf0 2024-09-21T06:30:28.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.290+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0ab0158880 con 0x7f0ab006ddf0 2024-09-21T06:30:28.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.290+0000 7f0aa7400640 1 -- 172.21.15.37:0/458613858 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7f0aa0083a00 con 0x7f0ab006ddf0 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:30:28.293 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 msgr2=0x7f0a9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 0x7f0a9007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005fd0 tx=0x7f0a9c005850 comp rx=0 tx=0).stop 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 msgr2=0x7f0ab01f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0034040 tx=0x7f0aa0003910 comp rx=0 tx=0).stop 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 shutdown_connections 2024-09-21T06:30:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a90078bc0 0x7f0a9007b0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab00cbc10 0x7f0ab00c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab0074cb0 0x7f0ab00cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 --2- 172.21.15.37:0/458613858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab006ddf0 0x7f0ab01f5040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 >> 172.21.15.37:0/458613858 conn(0x7f0ab00cca40 msgr2=0x7f0ab0159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 shutdown_connections 2024-09-21T06:30:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.293+0000 7f0ab6800640 1 -- 172.21.15.37:0/458613858 wait complete. 2024-09-21T06:30:28.317 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:30:28.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 -- 172.21.15.37:0/3161318390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2980ca690 msgr2=0x7fc298160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 --2- 172.21.15.37:0/3161318390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2980ca690 0x7fc298160100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc288005fd0 tx=0x7fc28802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 -- 172.21.15.37:0/3161318390 shutdown_connections 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 --2- 172.21.15.37:0/3161318390 >> v2:172.21.15.150:3300/0 conn(0x7fc29806e2f0 0x7fc298160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 --2- 172.21.15.37:0/3161318390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2980ca690 0x7fc298160100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 -- 172.21.15.37:0/3161318390 >> 172.21.15.37:0/3161318390 conn(0x7fc2980ccac0 msgr2=0x7fc2980cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 -- 172.21.15.37:0/3161318390 shutdown_connections 2024-09-21T06:30:28.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.545+0000 7fc29cc00640 1 -- 172.21.15.37:0/3161318390 wait complete. 2024-09-21T06:30:28.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.547+0000 7fc29cc00640 1 Processor -- start 2024-09-21T06:30:28.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.548+0000 7fc29cc00640 1 -- start start 2024-09-21T06:30:28.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc29806e2f0 0x7fc2981f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc298160640 0x7fc29815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc29806f9d0 con 0x7fc2980ca690 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc29806f850 con 0x7fc298160640 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc29cc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc29806fb50 con 0x7fc29806e2f0 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44358/0 (socket says 172.21.15.37:44358) 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc28fe00640 1 -- 172.21.15.37:0/698737845 learned_addr learned my addr 172.21.15.37:0/698737845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc295e00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc29806e2f0 0x7fc2981f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.549+0000 7fc296800640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc298160640 0x7fc29815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 -- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc29806e2f0 msgr2=0x7fc2981f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc29806e2f0 0x7fc2981f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 -- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc298160640 msgr2=0x7fc29815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc298160640 0x7fc29815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 -- 172.21.15.37:0/698737845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29815e360 con 0x7fc2980ca690 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28fe00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c340 tx=0x7fc28000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:28.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280017070 con 0x7fc2980ca690 2024-09-21T06:30:28.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.550+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2800053c0 con 0x7fc2980ca690 2024-09-21T06:30:28.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.551+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29815e630 con 0x7fc2980ca690 2024-09-21T06:30:28.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.551+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280003850 con 0x7fc2980ca690 2024-09-21T06:30:28.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.551+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29815ebf0 con 0x7fc2980ca690 2024-09-21T06:30:28.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.552+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298160a50 con 0x7fc2980ca690 2024-09-21T06:30:28.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.553+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc28001a070 con 0x7fc2980ca690 2024-09-21T06:30:28.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.554+0000 7fc28e000640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 0x7fc27407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:28.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.554+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fc280099e80 con 0x7fc2980ca690 2024-09-21T06:30:28.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.554+0000 7fc295e00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 0x7fc27407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:28.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.555+0000 7fc295e00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 0x7fc27407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc288033000 tx=0x7fc288004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:28.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.559+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc280061d90 con 0x7fc2980ca690 2024-09-21T06:30:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:28.753+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc2980b5f20 con 0x7fc2980ca690 2024-09-21T06:30:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.280+0000 7fc28e000640 1 -- 172.21.15.37:0/698737845 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7fc2800667c0 con 0x7fc2980ca690 2024-09-21T06:30:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 msgr2=0x7fc27407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 0x7fc27407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc288033000 tx=0x7fc288004730 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 msgr2=0x7fc2981585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c340 tx=0x7fc28000c800 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 shutdown_connections 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc274078b90 0x7fc27407b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2980ca690 0x7fc2981585d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc298160640 0x7fc29815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 --2- 172.21.15.37:0/698737845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc29806e2f0 0x7fc2981f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 >> 172.21.15.37:0/698737845 conn(0x7fc2980ccac0 msgr2=0x7fc298074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.283+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 shutdown_connections 2024-09-21T06:30:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.284+0000 7fc29cc00640 1 -- 172.21.15.37:0/698737845 wait complete. 2024-09-21T06:30:29.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 -- 172.21.15.37:0/3282612426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d40 msgr2=0x7fa138005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 --2- 172.21.15.37:0/3282612426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d40 0x7fa138005150 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa14c006fd0 tx=0x7fa138005830 comp rx=0 tx=0).stop 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa152800640 1 -- 172.21.15.37:0/3282612426 reap_dead start 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 -- 172.21.15.37:0/3282612426 shutdown_connections 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 -- 172.21.15.37:0/3282612426 >> 172.21.15.37:0/3282612426 conn(0x7fa15406f940 msgr2=0x7fa1540b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 -- 172.21.15.37:0/3282612426 shutdown_connections 2024-09-21T06:30:29.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.550+0000 7fa153200640 1 -- 172.21.15.37:0/3282612426 wait complete. 2024-09-21T06:30:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.553+0000 7fa153200640 1 Processor -- start 2024-09-21T06:30:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.553+0000 7fa153200640 1 -- start start 2024-09-21T06:30:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.554+0000 7fa153200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138004d40 0x7fa15415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa153200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa153200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa154139150 0x7fa1540a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa153200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540b4fb0 con 0x7fa154138800 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa153200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540b4e30 con 0x7fa138004d40 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa153200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa1540b5130 con 0x7fa154139150 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44372/0 (socket says 172.21.15.37:44372) 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 -- 172.21.15.37:0/3794020304 learned_addr learned my addr 172.21.15.37:0/3794020304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151e00640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138004d40 0x7fa15415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa152800640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa154139150 0x7fa1540a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 -- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa154139150 msgr2=0x7fa1540a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa154139150 0x7fa1540a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 -- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138004d40 msgr2=0x7fa15415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138004d40 0x7fa15415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.555+0000 7fa151400640 1 -- 172.21.15.37:0/3794020304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1540b0690 con 0x7fa154138800 2024-09-21T06:30:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa151400640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa140011ab0 tx=0x7fa140011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14001e070 con 0x7fa154138800 2024-09-21T06:30:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa140004460 con 0x7fa154138800 2024-09-21T06:30:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1540b0970 con 0x7fa154138800 2024-09-21T06:30:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14000fc00 con 0x7fa154138800 2024-09-21T06:30:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.556+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1540b0f30 con 0x7fa154138800 2024-09-21T06:30:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.557+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540cddc0 con 0x7fa154138800 2024-09-21T06:30:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.559+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa140021070 con 0x7fa154138800 2024-09-21T06:30:29.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.560+0000 7fa14a000640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:29.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.560+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7fa14009b690 con 0x7fa154138800 2024-09-21T06:30:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.561+0000 7fa151e00640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.562+0000 7fa151e00640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 0x7fa13407b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa144009cd0 tx=0x7fa144005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:29.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.565+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1400634c0 con 0x7fa154138800 2024-09-21T06:30:29.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:29.766+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa1540b0420 con 0x7fa154138800 2024-09-21T06:30:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.289+0000 7fa14a000640 1 -- 172.21.15.37:0/3794020304 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7fa140067ef0 con 0x7fa154138800 2024-09-21T06:30:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:30:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 msgr2=0x7fa13407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 0x7fa13407b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa144009cd0 tx=0x7fa144005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 msgr2=0x7fa154138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa140011ab0 tx=0x7fa140011eb0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 shutdown_connections 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa154138800 0x7fa154138c10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa154139150 0x7fa1540a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 --2- 172.21.15.37:0/3794020304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138004d40 0x7fa15415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 >> 172.21.15.37:0/3794020304 conn(0x7fa15406f940 msgr2=0x7fa154068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.293+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 shutdown_connections 2024-09-21T06:30:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.294+0000 7fa153200640 1 -- 172.21.15.37:0/3794020304 wait complete. 2024-09-21T06:30:30.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:30:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 -- 172.21.15.37:0/566858523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100156a50 msgr2=0x7f5100074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 --2- 172.21.15.37:0/566858523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100156a50 0x7f5100074620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5100074160 tx=0x7f50f002a540 comp rx=0 tx=0).stop 2024-09-21T06:30:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 -- 172.21.15.37:0/566858523 shutdown_connections 2024-09-21T06:30:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 --2- 172.21.15.37:0/566858523 >> v2:172.21.15.150:3300/0 conn(0x7f5100074b60 0x7f510006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 --2- 172.21.15.37:0/566858523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100156a50 0x7f5100074620 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 -- 172.21.15.37:0/566858523 >> 172.21.15.37:0/566858523 conn(0x7f51000b1b90 msgr2=0x7f51000b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 -- 172.21.15.37:0/566858523 shutdown_connections 2024-09-21T06:30:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.563+0000 7f5105200640 1 -- 172.21.15.37:0/566858523 wait complete. 2024-09-21T06:30:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.565+0000 7f5105200640 1 Processor -- start 2024-09-21T06:30:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.566+0000 7f5105200640 1 -- start start 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5100074b60 0x7f51001f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5100156a50 0x7f51000a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51000ce500 con 0x7f510006e330 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f51000ce380 con 0x7f5100156a50 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f5105200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f51000ce680 con 0x7f5100074b60 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f50fde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.567+0000 7f50fde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44400/0 (socket says 172.21.15.37:44400) 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 -- 172.21.15.37:0/2956620538 learned_addr learned my addr 172.21.15.37:0/2956620538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 -- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5100074b60 msgr2=0x7f51001f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5100074b60 0x7f51001f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 -- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5100156a50 msgr2=0x7f51000a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5100156a50 0x7f51000a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 -- 172.21.15.37:0/2956620538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5100158150 con 0x7f510006e330 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50fde00640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009920 tx=0x7f50f0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0031d60 con 0x7f510006e330 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50f0033e10 con 0x7f510006e330 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0005260 con 0x7f510006e330 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.568+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5100158430 con 0x7f510006e330 2024-09-21T06:30:30.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.569+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51001f8fe0 con 0x7f510006e330 2024-09-21T06:30:30.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.570+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510006ddf0 con 0x7f510006e330 2024-09-21T06:30:30.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.571+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50f0005400 con 0x7f510006e330 2024-09-21T06:30:30.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.573+0000 7f50f6000640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 0x7f50e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:30.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.573+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7f50f00bc090 con 0x7f510006e330 2024-09-21T06:30:30.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.573+0000 7f50fd400640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 0x7f50e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:30.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.574+0000 7f50fd400640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 0x7f50e007b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5100070f80 tx=0x7f50ec00b370 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:30.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.578+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50f003f030 con 0x7f510006e330 2024-09-21T06:30:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:30.817+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f51000ce160 con 0x7f510006e330 2024-09-21T06:30:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.304+0000 7f50f6000640 1 -- 172.21.15.37:0/2956620538 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v30) ==== 172+0+0 (secure 0 0 0) 0x7f50f0084420 con 0x7f510006e330 2024-09-21T06:30:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 7 and data pool 8 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.311+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 msgr2=0x7f50e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.311+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 0x7f50e007b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5100070f80 tx=0x7f50ec00b370 comp rx=0 tx=0).stop 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.311+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 msgr2=0x7f510015a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.311+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009920 tx=0x7f50f0033c20 comp rx=0 tx=0).stop 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 shutdown_connections 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f50e0078b90 0x7f50e007b0a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5100156a50 0x7f51000a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5100074b60 0x7f51001f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 --2- 172.21.15.37:0/2956620538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f510006e330 0x7f510015a080 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.312+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 >> 172.21.15.37:0/2956620538 conn(0x7f51000b1b90 msgr2=0x7f510015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:31.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.313+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 shutdown_connections 2024-09-21T06:30:31.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.314+0000 7f5105200640 1 -- 172.21.15.37:0/2956620538 wait complete. 2024-09-21T06:30:31.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:31.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 -- 172.21.15.37:0/2377005210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 msgr2=0x7f0b181622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 --2- 172.21.15.37:0/2377005210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b181622b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b18162210 tx=0x7f0b0802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 -- 172.21.15.37:0/2377005210 shutdown_connections 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 --2- 172.21.15.37:0/2377005210 >> v2:172.21.15.150:3300/0 conn(0x7f0b181627f0 0x7f0b1806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 --2- 172.21.15.37:0/2377005210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b181622b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 -- 172.21.15.37:0/2377005210 >> 172.21.15.37:0/2377005210 conn(0x7f0b180d1930 msgr2=0x7f0b180d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 -- 172.21.15.37:0/2377005210 shutdown_connections 2024-09-21T06:30:31.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.576+0000 7f0b17200640 1 -- 172.21.15.37:0/2377005210 wait complete. 2024-09-21T06:30:31.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.578+0000 7f0b17200640 1 Processor -- start 2024-09-21T06:30:31.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.578+0000 7f0b17200640 1 -- start start 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b1806d7f0 0x7f0b18157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b181627f0 0x7f0b18156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18155940 con 0x7f0b180b9e40 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181557c0 con 0x7f0b1806d7f0 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b17200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0b18155ac0 con 0x7f0b181627f0 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44428/0 (socket says 172.21.15.37:44428) 2024-09-21T06:30:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 -- 172.21.15.37:0/59465296 learned_addr learned my addr 172.21.15.37:0/59465296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15e00640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b1806d7f0 0x7f0b18157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b16800640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b181627f0 0x7f0b18156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 -- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b181627f0 msgr2=0x7f0b18156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b181627f0 0x7f0b18156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 -- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b1806d7f0 msgr2=0x7f0b18157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b1806d7f0 0x7f0b18157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.580+0000 7f0b15400640 1 -- 172.21.15.37:0/59465296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b181f9030 con 0x7f0b180b9e40 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b15400640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d770 tx=0x7f0b0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017070 con 0x7f0b180b9e40 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b04009d20 con 0x7f0b180b9e40 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181f9310 con 0x7f0b180b9e40 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040106c0 con 0x7f0b180b9e40 2024-09-21T06:30:31.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.581+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b18151c50 con 0x7f0b180b9e40 2024-09-21T06:30:31.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.582+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b181f8da0 con 0x7f0b180b9e40 2024-09-21T06:30:31.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.584+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b0401a070 con 0x7f0b180b9e40 2024-09-21T06:30:31.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.585+0000 7f0b0e000640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 0x7f0af807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:31.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.585+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0b0409afe0 con 0x7f0b180b9e40 2024-09-21T06:30:31.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.585+0000 7f0b15e00640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 0x7f0af807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:31.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.586+0000 7f0b15e00640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 0x7f0af807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b08033000 tx=0x7f0b08004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:31.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.589+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b04062be0 con 0x7f0b180b9e40 2024-09-21T06:30:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.785+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b18155cd0 con 0x7f0b180b9e40 2024-09-21T06:30:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.787+0000 7f0b0e000640 1 -- 172.21.15.37:0/59465296 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16967 (secure 0 0 0) 0x7f0b04067610 con 0x7f0b180b9e40 2024-09-21T06:30:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:31.789 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":53,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:30.819011+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:30:28.767088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T06:30:29.779185+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 msgr2=0x7f0af807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 0x7f0af807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b08033000 tx=0x7f0b08004730 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 msgr2=0x7f0b18151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d770 tx=0x7f0b0400dc30 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 shutdown_connections 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0af8078b90 0x7f0af807b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b180b9e40 0x7f0b18151710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b181627f0 0x7f0b18156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 --2- 172.21.15.37:0/59465296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b1806d7f0 0x7f0b18157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 >> 172.21.15.37:0/59465296 conn(0x7f0b180d1930 msgr2=0x7f0b180b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 shutdown_connections 2024-09-21T06:30:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:31.789+0000 7f0b17200640 1 -- 172.21.15.37:0/59465296 wait complete. 2024-09-21T06:30:31.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.044+0000 7fe1af200640 1 -- 172.21.15.37:0/1827726467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe190005300 msgr2=0x7fe19000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.044+0000 7fe1af200640 1 --2- 172.21.15.37:0/1827726467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe190005300 0x7fe19000b9c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80d0c70 tx=0x7fe190005830 comp rx=0 tx=0).stop 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.044+0000 7fe1af200640 1 -- 172.21.15.37:0/1827726467 shutdown_connections 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.044+0000 7fe1af200640 1 --2- 172.21.15.37:0/1827726467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe190005300 0x7fe19000b9c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.044+0000 7fe1af200640 1 -- 172.21.15.37:0/1827726467 >> 172.21.15.37:0/1827726467 conn(0x7fe1a80d4360 msgr2=0x7fe1a80d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.045+0000 7fe1af200640 1 -- 172.21.15.37:0/1827726467 shutdown_connections 2024-09-21T06:30:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.045+0000 7fe1af200640 1 -- 172.21.15.37:0/1827726467 wait complete. 2024-09-21T06:30:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.047+0000 7fe1af200640 1 Processor -- start 2024-09-21T06:30:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.047+0000 7fe1af200640 1 -- start start 2024-09-21T06:30:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190005300 0x7fe1a80cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 0x7fe1a80ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80cf630 con 0x7fe1a81f8150 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80cf4b0 con 0x7fe190005300 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.048+0000 7fe1af200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a80cf7b0 con 0x7fe1a80acbf0 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44450/0 (socket says 172.21.15.37:44450) 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 -- 172.21.15.37:0/3110185180 learned_addr learned my addr 172.21.15.37:0/3110185180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ade00640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190005300 0x7fe1a80cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ae800640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 0x7fe1a80ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 -- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 msgr2=0x7fe1a80ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 0x7fe1a80ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 -- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190005300 msgr2=0x7fe1a80cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190005300 0x7fe1a80cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 -- 172.21.15.37:0/3110185180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a80cbe10 con 0x7fe1a81f8150 2024-09-21T06:30:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ae800640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 0x7fe1a80ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.049+0000 7fe1ad400640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe198004d60 tx=0x7fe198001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.050+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198015020 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.050+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe198004e50 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.050+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198011c20 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.050+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a80cc0f0 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.050+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a80cc6b0 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.051+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a80cfd10 con 0x7fe1a81f8150 2024-09-21T06:30:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.052+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe198018020 con 0x7fe1a81f8150 2024-09-21T06:30:32.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.053+0000 7fe1a2000640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 0x7fe18c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.053+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe19809b960 con 0x7fe1a81f8150 2024-09-21T06:30:32.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.054+0000 7fe1ade00640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 0x7fe18c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.055+0000 7fe1ade00640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 0x7fe18c07b0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005e00 tx=0x7fe19c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.058+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe19801f020 con 0x7fe1a81f8150 2024-09-21T06:30:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.258+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1a80cca90 con 0x7fe1a81f8150 2024-09-21T06:30:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.259+0000 7fe1a2000640 1 -- 172.21.15.37:0/3110185180 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16967 (secure 0 0 0) 0x7fe198063b70 con 0x7fe1a81f8150 2024-09-21T06:30:32.261 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:32.262 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":53,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:30.819011+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:30:28.767088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T06:30:29.779185+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 msgr2=0x7fe18c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 0x7fe18c07b0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005e00 tx=0x7fe19c005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 msgr2=0x7fe1a81f8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe198004d60 tx=0x7fe198001710 comp rx=0 tx=0).stop 2024-09-21T06:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 shutdown_connections 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe18c078bc0 0x7fe18c07b0d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1a81f8150 0x7fe1a81f8560 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1a80acbf0 0x7fe1a80ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 --2- 172.21.15.37:0/3110185180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190005300 0x7fe1a80cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 >> 172.21.15.37:0/3110185180 conn(0x7fe1a80d4360 msgr2=0x7fe1a81e8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 shutdown_connections 2024-09-21T06:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.262+0000 7fe1af200640 1 -- 172.21.15.37:0/3110185180 wait complete. 2024-09-21T06:30:32.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:30:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 -- 172.21.15.37:0/1279441960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b806e370 msgr2=0x7f21b80b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 --2- 172.21.15.37:0/1279441960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b806e370 0x7f21b80b4f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21a8005fd0 tx=0x7f21a802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 -- 172.21.15.37:0/1279441960 shutdown_connections 2024-09-21T06:30:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 --2- 172.21.15.37:0/1279441960 >> v2:172.21.15.150:3300/0 conn(0x7f21b8156c60 0x7f21b80cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 --2- 172.21.15.37:0/1279441960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b806e370 0x7f21b80b4f90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 -- 172.21.15.37:0/1279441960 >> 172.21.15.37:0/1279441960 conn(0x7f21b806f920 msgr2=0x7f21b80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 -- 172.21.15.37:0/1279441960 shutdown_connections 2024-09-21T06:30:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.520+0000 7f21bd800640 1 -- 172.21.15.37:0/1279441960 wait complete. 2024-09-21T06:30:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.523+0000 7f21bd800640 1 Processor -- start 2024-09-21T06:30:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.523+0000 7f21bd800640 1 -- start start 2024-09-21T06:30:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21b80cc8b0 0x7f21b80a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b8156c60 0x7f21b81557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21b806d510 con 0x7f21b806e370 2024-09-21T06:30:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21b806d390 con 0x7f21b80cc8b0 2024-09-21T06:30:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21bd800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f21b806d690 con 0x7f21b8156c60 2024-09-21T06:30:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.524+0000 7f21b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44466/0 (socket says 172.21.15.37:44466) 2024-09-21T06:30:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 -- 172.21.15.37:0/175159473 learned_addr learned my addr 172.21.15.37:0/175159473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21afe00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21b80cc8b0 0x7f21b80a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21bce00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b8156c60 0x7f21b81557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 -- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b8156c60 msgr2=0x7f21b81557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b8156c60 0x7f21b81557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 -- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21b80cc8b0 msgr2=0x7f21b80a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21b80cc8b0 0x7f21b80a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 -- 172.21.15.37:0/175159473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b81f53a0 con 0x7f21b806e370 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.525+0000 7f21b7e00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006010 tx=0x7f21a80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.526+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a803e030 con 0x7f21b806e370 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.526+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21a8003ae0 con 0x7f21b806e370 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.526+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b81f90f0 con 0x7f21b806e370 2024-09-21T06:30:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.526+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8035040 con 0x7f21b806e370 2024-09-21T06:30:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.526+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b81f96b0 con 0x7f21b806e370 2024-09-21T06:30:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.527+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b806dc20 con 0x7f21b806e370 2024-09-21T06:30:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.528+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21a8041030 con 0x7f21b806e370 2024-09-21T06:30:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.529+0000 7f21b4c00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 0x7f219407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.529+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f21a80bb3e0 con 0x7f21b806e370 2024-09-21T06:30:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.530+0000 7f21afe00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 0x7f219407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.530+0000 7f21afe00640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 0x7f219407b0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21a00095f0 tx=0x7f21a0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.534+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21a8083030 con 0x7f21b806e370 2024-09-21T06:30:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:32.769+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f21b8073360 con 0x7f21b806e370 2024-09-21T06:30:33.282 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:30:33.280+0000 7f934ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:30:33.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.320+0000 7f21b4c00640 1 -- 172.21.15.37:0/175159473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f21a8087a60 con 0x7f21b806e370 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 msgr2=0x7f219407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 0x7f219407b0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21a00095f0 tx=0x7f21a0005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 msgr2=0x7f21b80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006010 tx=0x7f21a80035f0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 shutdown_connections 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2194078bc0 0x7f219407b0d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21b8156c60 0x7f21b81557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21b80cc8b0 0x7f21b80a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 --2- 172.21.15.37:0/175159473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21b806e370 0x7f21b80ca3c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 >> 172.21.15.37:0/175159473 conn(0x7f21b806f920 msgr2=0x7f21b80cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:33.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 shutdown_connections 2024-09-21T06:30:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.323+0000 7f21bd800640 1 -- 172.21.15.37:0/175159473 wait complete. 2024-09-21T06:30:33.353 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:30:33.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:30:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 -- 172.21.15.37:0/3272475702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135c005350 msgr2=0x7f135c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 --2- 172.21.15.37:0/3272475702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135c005350 0x7f135c00b9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1370005fd0 tx=0x7f135c005830 comp rx=0 tx=0).stop 2024-09-21T06:30:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 -- 172.21.15.37:0/3272475702 shutdown_connections 2024-09-21T06:30:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 --2- 172.21.15.37:0/3272475702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f135c005350 0x7f135c00b9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 -- 172.21.15.37:0/3272475702 >> 172.21.15.37:0/3272475702 conn(0x7f13780d1960 msgr2=0x7f13780d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 -- 172.21.15.37:0/3272475702 shutdown_connections 2024-09-21T06:30:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.585+0000 7f137ca00640 1 -- 172.21.15.37:0/3272475702 wait complete. 2024-09-21T06:30:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.587+0000 7f137ca00640 1 Processor -- start 2024-09-21T06:30:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.587+0000 7f137ca00640 1 -- start start 2024-09-21T06:30:33.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f135c005350 0x7f137815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13781f8a00 0x7f13781592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1378162830 con 0x7f137814d0e0 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13781626b0 con 0x7f13781f8a00 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f137ca00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f13781629b0 con 0x7f135c005350 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f136fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f136fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44494/0 (socket says 172.21.15.37:44494) 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f136fe00640 1 -- 172.21.15.37:0/1350926822 learned_addr learned my addr 172.21.15.37:0/1350926822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f1377400640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f135c005350 0x7f137815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.589+0000 7f1377e00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13781f8a00 0x7f13781592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 -- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f135c005350 msgr2=0x7f137815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f135c005350 0x7f137815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 -- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13781f8a00 msgr2=0x7f13781592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:33.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13781f8a00 0x7f13781592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:33.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 -- 172.21.15.37:0/1350926822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137814ce50 con 0x7f137814d0e0 2024-09-21T06:30:33.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136fe00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f13600066b0 tx=0x7f13600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360015070 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1360011900 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360011d70 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137815e530 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.590+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137815e250 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.591+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137815dce0 con 0x7f137814d0e0 2024-09-21T06:30:33.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.593+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1360018070 con 0x7f137814d0e0 2024-09-21T06:30:33.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.594+0000 7f136f400640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 0x7f135407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:33.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.594+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f136009bb60 con 0x7f137814d0e0 2024-09-21T06:30:33.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.595+0000 7f1377400640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 0x7f135407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:33.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.595+0000 7f1377400640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 0x7f135407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1368000c00 tx=0x7f1368006d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:33.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.599+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13600637f0 con 0x7f137814d0e0 2024-09-21T06:30:33.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:33.834+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f13780cf510 con 0x7f137814d0e0 2024-09-21T06:30:34.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.329+0000 7f136f400640 1 -- 172.21.15.37:0/1350926822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f1360068220 con 0x7f137814d0e0 2024-09-21T06:30:34.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 msgr2=0x7f135407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 0x7f135407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1368000c00 tx=0x7f1368006d10 comp rx=0 tx=0).stop 2024-09-21T06:30:34.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 msgr2=0x7f137814c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f13600066b0 tx=0x7f13600039c0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 shutdown_connections 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1354078b90 0x7f135407b0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f137814d0e0 0x7f137814c910 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13781f8a00 0x7f13781592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 --2- 172.21.15.37:0/1350926822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f135c005350 0x7f137815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 >> 172.21.15.37:0/1350926822 conn(0x7f13780d1960 msgr2=0x7f1378068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 shutdown_connections 2024-09-21T06:30:34.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.332+0000 7f137ca00640 1 -- 172.21.15.37:0/1350926822 wait complete. 2024-09-21T06:30:34.357 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:30:34.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 -- 172.21.15.37:0/708227728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80cddc0 msgr2=0x7f06a80cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 --2- 172.21.15.37:0/708227728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80cddc0 0x7f06a80cc810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0698009830 tx=0x7f069802a520 comp rx=0 tx=0).stop 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 -- 172.21.15.37:0/708227728 shutdown_connections 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 --2- 172.21.15.37:0/708227728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80cddc0 0x7f06a80cc810 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 -- 172.21.15.37:0/708227728 >> 172.21.15.37:0/708227728 conn(0x7f06a806f940 msgr2=0x7f06a80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 -- 172.21.15.37:0/708227728 shutdown_connections 2024-09-21T06:30:34.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.582+0000 7f06a7200640 1 -- 172.21.15.37:0/708227728 wait complete. 2024-09-21T06:30:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.585+0000 7f06a7200640 1 Processor -- start 2024-09-21T06:30:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.585+0000 7f06a7200640 1 -- start start 2024-09-21T06:30:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a80b50a0 0x7f06a8149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80ccd50 0x7f06a81f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:34.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80ce330 con 0x7f06a80cddc0 2024-09-21T06:30:34.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06a806e5f0 con 0x7f06a80ccd50 2024-09-21T06:30:34.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.586+0000 7f06a7200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f06a815d630 con 0x7f06a80b50a0 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80ccd50 0x7f06a81f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43790/0 (socket says 172.21.15.37:43790) 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 -- 172.21.15.37:0/2199929169 learned_addr learned my addr 172.21.15.37:0/2199929169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a5e00640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a80b50a0 0x7f06a8149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 -- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a80b50a0 msgr2=0x7f06a8149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a80b50a0 0x7f06a8149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 -- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80ccd50 msgr2=0x7f06a81f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80ccd50 0x7f06a81f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 -- 172.21.15.37:0/2199929169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81590f0 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.587+0000 7f06a6800640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f06a000cad0 tx=0x7f06a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.588+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0016020 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.588+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06a000ad20 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.588+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0012680 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.588+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81594e0 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.588+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a8159aa0 con 0x7f06a80cddc0 2024-09-21T06:30:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.589+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a8152c40 con 0x7f06a80cddc0 2024-09-21T06:30:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.590+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06a0019020 con 0x7f06a80cddc0 2024-09-21T06:30:34.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.592+0000 7f069e000640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 0x7f068807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:34.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.592+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f06a009b360 con 0x7f06a80cddc0 2024-09-21T06:30:34.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.592+0000 7f06a5e00640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 0x7f068807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.593+0000 7f06a5e00640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 0x7f068807b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0698034000 tx=0x7f0698004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.596+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06a0020020 con 0x7f06a80cddc0 2024-09-21T06:30:34.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.833+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06a80b4c70 con 0x7f06a80cddc0 2024-09-21T06:30:34.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.833+0000 7f069e000640 1 -- 172.21.15.37:0/2199929169 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5229 (secure 0 0 0) 0x7f06a80cd160 con 0x7f06a80cddc0 2024-09-21T06:30:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:34.836 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"btime":"2024-09-21T06:30:34:322066+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5506,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:30:33.835909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574]},"id":3}]} 2024-09-21T06:30:34.836 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-09-21T06:30:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 msgr2=0x7f068807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 0x7f068807b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0698034000 tx=0x7f0698004730 comp rx=0 tx=0).stop 2024-09-21T06:30:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 msgr2=0x7f06a8150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f06a000cad0 tx=0x7f06a000cf90 comp rx=0 tx=0).stop 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 shutdown_connections 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0688078b90 0x7f068807b0a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a80cddc0 0x7f06a8150050 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a80ccd50 0x7f06a81f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 --2- 172.21.15.37:0/2199929169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a80b50a0 0x7f06a8149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 >> 172.21.15.37:0/2199929169 conn(0x7f06a806f940 msgr2=0x7f06a815e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 shutdown_connections 2024-09-21T06:30:34.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:34.836+0000 7f06a7200640 1 -- 172.21.15.37:0/2199929169 wait complete. 2024-09-21T06:30:34.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 -- 172.21.15.37:0/1991967407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40cf930 msgr2=0x7f9fd40ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 --2- 172.21.15.37:0/1991967407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40cf930 0x7f9fd40ce320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005fd0 tx=0x7f9fbc02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 -- 172.21.15.37:0/1991967407 shutdown_connections 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 --2- 172.21.15.37:0/1991967407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40cf930 0x7f9fd40ce320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 --2- 172.21.15.37:0/1991967407 >> v2:172.21.15.150:3300/0 conn(0x7f9fd40ce860 0x7f9fd415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 -- 172.21.15.37:0/1991967407 >> 172.21.15.37:0/1991967407 conn(0x7f9fd40cca40 msgr2=0x7f9fd40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 -- 172.21.15.37:0/1991967407 shutdown_connections 2024-09-21T06:30:35.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.083+0000 7f9fd9200640 1 -- 172.21.15.37:0/1991967407 wait complete. 2024-09-21T06:30:35.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.086+0000 7f9fd9200640 1 Processor -- start 2024-09-21T06:30:35.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.086+0000 7f9fd9200640 1 -- start start 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40ce860 0x7f9fd41f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 0x7f9fd41f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd406df70 con 0x7f9fd40cf930 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd406ddf0 con 0x7f9fd415b9f0 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.087+0000 7f9fd9200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd406e0f0 con 0x7f9fd40ce860 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43802/0 (socket says 172.21.15.37:43802) 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 -- 172.21.15.37:0/2268991163 learned_addr learned my addr 172.21.15.37:0/2268991163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fd2800640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 0x7f9fd41f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fd1e00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40ce860 0x7f9fd41f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 -- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40ce860 msgr2=0x7f9fd41f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40ce860 0x7f9fd41f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 -- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 msgr2=0x7f9fd41f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 0x7f9fd41f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 -- 172.21.15.37:0/2268991163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd40b08b0 con 0x7f9fd40cf930 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fd2800640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 0x7f9fd41f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.088+0000 7f9fcbe00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc034040 tx=0x7f9fbc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.089+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc03e030 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.089+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fbc003ae0 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.089+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd4152a80 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.089+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc035040 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.089+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd4153040 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.090+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd406e610 con 0x7f9fd40cf930 2024-09-21T06:30:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.091+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fbc041030 con 0x7f9fd40cf930 2024-09-21T06:30:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.092+0000 7f9fca000640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 0x7f9fb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.092+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9fbc0bb4a0 con 0x7f9fd40cf930 2024-09-21T06:30:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.093+0000 7f9fd1e00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 0x7f9fb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.093+0000 7f9fd1e00640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 0x7f9fb007b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4001720 tx=0x7f9fc4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.097+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fbc082fe0 con 0x7f9fd40cf930 2024-09-21T06:30:35.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.294+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fd415ece0 con 0x7f9fd40cf930 2024-09-21T06:30:35.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.296+0000 7f9fca000640 1 -- 172.21.15.37:0/2268991163 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16954 (secure 0 0 0) 0x7f9fbc087a10 con 0x7f9fd40cf930 2024-09-21T06:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":54,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:32.301838+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:30:28.767088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T06:30:29.779185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:30:35.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 msgr2=0x7f9fb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 0x7f9fb007b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4001720 tx=0x7f9fc4009040 comp rx=0 tx=0).stop 2024-09-21T06:30:35.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 msgr2=0x7f9fd41f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc034040 tx=0x7f9fbc0035f0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 shutdown_connections 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9fb0078bc0 0x7f9fb007b0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fd40cf930 0x7f9fd41f8c70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9fd415b9f0 0x7f9fd41f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 --2- 172.21.15.37:0/2268991163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9fd40ce860 0x7f9fd41f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 >> 172.21.15.37:0/2268991163 conn(0x7f9fd40cca40 msgr2=0x7f9fd4159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 shutdown_connections 2024-09-21T06:30:35.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.298+0000 7f9fd9200640 1 -- 172.21.15.37:0/2268991163 wait complete. 2024-09-21T06:30:35.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:30:35.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:30:35.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 -- 172.21.15.37:0/3194945970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 msgr2=0x7f8ad8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3194945970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8ad8004c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8af40cc2d0 tx=0x7f8ad8005830 comp rx=0 tx=0).stop 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af3400640 1 -- 172.21.15.37:0/3194945970 reap_dead start 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 -- 172.21.15.37:0/3194945970 shutdown_connections 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 -- 172.21.15.37:0/3194945970 >> 172.21.15.37:0/3194945970 conn(0x7f8af40cca40 msgr2=0x7f8af40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 -- 172.21.15.37:0/3194945970 shutdown_connections 2024-09-21T06:30:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.552+0000 7f8af9e00640 1 -- 172.21.15.37:0/3194945970 wait complete. 2024-09-21T06:30:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.555+0000 7f8af9e00640 1 Processor -- start 2024-09-21T06:30:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.555+0000 7f8af9e00640 1 -- start start 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8af41f4160 0x7f8af4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 0x7f8af41f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8af415fae0 con 0x7f8ad8004800 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40ce6a0 con 0x7f8af41f4160 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af9e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8af415fc60 con 0x7f8af4155790 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43826/0 (socket says 172.21.15.37:43826) 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.556+0000 7f8af2a00640 1 -- 172.21.15.37:0/3797472023 learned_addr learned my addr 172.21.15.37:0/3797472023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8aebe00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8af41f4160 0x7f8af4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af3400640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 0x7f8af41f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 -- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 msgr2=0x7f8af41f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 0x7f8af41f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 -- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8af41f4160 msgr2=0x7f8af4155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8af41f4160 0x7f8af4155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 -- 172.21.15.37:0/3797472023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af4155ba0 con 0x7f8ad8004800 2024-09-21T06:30:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af3400640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 0x7f8af41f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af2a00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400e5c0 tx=0x7f8ae400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4015070 con 0x7f8ad8004800 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae400c9e0 con 0x7f8ad8004800 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41f49d0 con 0x7f8ad8004800 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.557+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400ce30 con 0x7f8ad8004800 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.558+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41f4c70 con 0x7f8ad8004800 2024-09-21T06:30:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.559+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af415f5a0 con 0x7f8ad8004800 2024-09-21T06:30:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.560+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ae4018070 con 0x7f8ad8004800 2024-09-21T06:30:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.561+0000 7f8aeb400640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 0x7f8ad007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.561+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8ae409b2c0 con 0x7f8ad8004800 2024-09-21T06:30:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.561+0000 7f8aebe00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 0x7f8ad007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:35.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.562+0000 7f8aebe00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 0x7f8ad007b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8adc035000 tx=0x7f8adc02d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.566+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8af4155ba0 con 0x7f8ad8004800 2024-09-21T06:30:35.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.761+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8af40b10b0 con 0x7f8ad0078b90 2024-09-21T06:30:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.766+0000 7f8aeb400640 1 -- 172.21.15.37:0/3797472023 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f8adc039040 con 0x7f8ad0078b90 2024-09-21T06:30:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:30:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.770+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 msgr2=0x7f8ad007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.770+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 0x7f8ad007b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8adc035000 tx=0x7f8adc02d040 comp rx=0 tx=0).stop 2024-09-21T06:30:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.770+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 msgr2=0x7f8af40b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.770+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400e5c0 tx=0x7f8ae400ea80 comp rx=0 tx=0).stop 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 shutdown_connections 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8ad0078b90 0x7f8ad007b0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8af4155790 0x7f8af41f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8af41f4160 0x7f8af4155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 --2- 172.21.15.37:0/3797472023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8004800 0x7f8af40b0b70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 >> 172.21.15.37:0/3797472023 conn(0x7f8af40cca40 msgr2=0x7f8af40733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 shutdown_connections 2024-09-21T06:30:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:35.771+0000 7f8af9e00640 1 -- 172.21.15.37:0/3797472023 wait complete. 2024-09-21T06:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-21T06:30:35.128519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234896,"kb_used_data":6160,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739824,"statfs":{"total":773094113280,"available":772853579776,"internally_reserved":0,"allocated":6307840,"data_stored":3420788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.845427"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309641+0000","last_change":"2024-09-21T06:30:31.325431+0000","last_active":"2024-09-21T06:30:32.309641+0000","last_peered":"2024-09-21T06:30:32.309641+0000","last_clean":"2024-09-21T06:30:32.309641+0000","last_became_active":"2024-09-21T06:30:31.324477+0000","last_became_peered":"2024-09-21T06:30:31.324477+0000","last_unstale":"2024-09-21T06:30:32.309641+0000","last_undegraded":"2024-09-21T06:30:32.309641+0000","last_fullsized":"2024-09-21T06:30:32.309641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:11.580508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538386+0000","last_change":"2024-09-21T06:30:31.331122+0000","last_active":"2024-09-21T06:30:32.538386+0000","last_peered":"2024-09-21T06:30:32.538386+0000","last_clean":"2024-09-21T06:30:32.538386+0000","last_became_active":"2024-09-21T06:30:31.326039+0000","last_became_peered":"2024-09-21T06:30:31.326039+0000","last_unstale":"2024-09-21T06:30:32.538386+0000","last_undegraded":"2024-09-21T06:30:32.538386+0000","last_fullsized":"2024-09-21T06:30:32.538386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:42.542980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538226+0000","last_change":"2024-09-21T06:30:31.323579+0000","last_active":"2024-09-21T06:30:32.538226+0000","last_peered":"2024-09-21T06:30:32.538226+0000","last_clean":"2024-09-21T06:30:32.538226+0000","last_became_active":"2024-09-21T06:30:31.323235+0000","last_became_peered":"2024-09-21T06:30:31.323235+0000","last_unstale":"2024-09-21T06:30:32.538226+0000","last_undegraded":"2024-09-21T06:30:32.538226+0000","last_fullsized":"2024-09-21T06:30:32.538226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:34.886039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310028+0000","last_change":"2024-09-21T06:30:31.319171+0000","last_active":"2024-09-21T06:30:32.310028+0000","last_peered":"2024-09-21T06:30:32.310028+0000","last_clean":"2024-09-21T06:30:32.310028+0000","last_became_active":"2024-09-21T06:30:31.318852+0000","last_became_peered":"2024-09-21T06:30:31.318852+0000","last_unstale":"2024-09-21T06:30:32.310028+0000","last_undegraded":"2024-09-21T06:30:32.310028+0000","last_fullsized":"2024-09-21T06:30:32.310028+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537885+0000","last_change":"2024-09-21T06:30:31.321094+0000","last_active":"2024-09-21T06:30:32.537885+0000","last_peered":"2024-09-21T06:30:32.537885+0000","last_clean":"2024-09-21T06:30:32.537885+0000","last_became_active":"2024-09-21T06:30:31.320370+0000","last_became_peered":"2024-09-21T06:30:31.320370+0000","last_unstale":"2024-09-21T06:30:32.537885+0000","last_undegraded":"2024-09-21T06:30:32.537885+0000","last_fullsized":"2024-09-21T06:30:32.537885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:58.090005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537501+0000","last_change":"2024-09-21T06:30:31.330820+0000","last_active":"2024-09-21T06:30:32.537501+0000","last_peered":"2024-09-21T06:30:32.537501+0000","last_clean":"2024-09-21T06:30:32.537501+0000","last_became_active":"2024-09-21T06:30:31.330534+0000","last_became_peered":"2024-09-21T06:30:31.330534+0000","last_unstale":"2024-09-21T06:30:32.537501+0000","last_undegraded":"2024-09-21T06:30:32.537501+0000","last_fullsized":"2024-09-21T06:30:32.537501+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:22.628559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310423+0000","last_change":"2024-09-21T06:30:31.315315+0000","last_active":"2024-09-21T06:30:32.310423+0000","last_peered":"2024-09-21T06:30:32.310423+0000","last_clean":"2024-09-21T06:30:32.310423+0000","last_became_active":"2024-09-21T06:30:31.315006+0000","last_became_peered":"2024-09-21T06:30:31.315006+0000","last_unstale":"2024-09-21T06:30:32.310423+0000","last_undegraded":"2024-09-21T06:30:32.310423+0000","last_fullsized":"2024-09-21T06:30:32.310423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.674377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537263+0000","last_change":"2024-09-21T06:30:31.325015+0000","last_active":"2024-09-21T06:30:32.537263+0000","last_peered":"2024-09-21T06:30:32.537263+0000","last_clean":"2024-09-21T06:30:32.537263+0000","last_became_active":"2024-09-21T06:30:31.324713+0000","last_became_peered":"2024-09-21T06:30:31.324713+0000","last_unstale":"2024-09-21T06:30:32.537263+0000","last_undegraded":"2024-09-21T06:30:32.537263+0000","last_fullsized":"2024-09-21T06:30:32.537263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:03.771004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538533+0000","last_change":"2024-09-21T06:30:31.327858+0000","last_active":"2024-09-21T06:30:32.538533+0000","last_peered":"2024-09-21T06:30:32.538533+0000","last_clean":"2024-09-21T06:30:32.538533+0000","last_became_active":"2024-09-21T06:30:31.326704+0000","last_became_peered":"2024-09-21T06:30:31.326704+0000","last_unstale":"2024-09-21T06:30:32.538533+0000","last_undegraded":"2024-09-21T06:30:32.538533+0000","last_fullsized":"2024-09-21T06:30:32.538533+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.222459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310851+0000","last_change":"2024-09-21T06:30:31.334860+0000","last_active":"2024-09-21T06:30:32.310851+0000","last_peered":"2024-09-21T06:30:32.310851+0000","last_clean":"2024-09-21T06:30:32.310851+0000","last_became_active":"2024-09-21T06:30:31.334387+0000","last_became_peered":"2024-09-21T06:30:31.334387+0000","last_unstale":"2024-09-21T06:30:32.310851+0000","last_undegraded":"2024-09-21T06:30:32.310851+0000","last_fullsized":"2024-09-21T06:30:32.310851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:30.557490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309518+0000","last_change":"2024-09-21T06:30:31.321566+0000","last_active":"2024-09-21T06:30:32.309518+0000","last_peered":"2024-09-21T06:30:32.309518+0000","last_clean":"2024-09-21T06:30:32.309518+0000","last_became_active":"2024-09-21T06:30:31.321066+0000","last_became_peered":"2024-09-21T06:30:31.321066+0000","last_unstale":"2024-09-21T06:30:32.309518+0000","last_undegraded":"2024-09-21T06:30:32.309518+0000","last_fullsized":"2024-09-21T06:30:32.309518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:50.623685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537639+0000","last_change":"2024-09-21T06:30:31.322990+0000","last_active":"2024-09-21T06:30:32.537639+0000","last_peered":"2024-09-21T06:30:32.537639+0000","last_clean":"2024-09-21T06:30:32.537639+0000","last_became_active":"2024-09-21T06:30:31.322551+0000","last_became_peered":"2024-09-21T06:30:31.322551+0000","last_unstale":"2024-09-21T06:30:32.537639+0000","last_undegraded":"2024-09-21T06:30:32.537639+0000","last_fullsized":"2024-09-21T06:30:32.537639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:58.090005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538261+0000","last_change":"2024-09-21T06:30:31.330123+0000","last_active":"2024-09-21T06:30:32.538261+0000","last_peered":"2024-09-21T06:30:32.538261+0000","last_clean":"2024-09-21T06:30:32.538261+0000","last_became_active":"2024-09-21T06:30:31.327189+0000","last_became_peered":"2024-09-21T06:30:31.327189+0000","last_unstale":"2024-09-21T06:30:32.538261+0000","last_undegraded":"2024-09-21T06:30:32.538261+0000","last_fullsized":"2024-09-21T06:30:32.538261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.142991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.308878+0000","last_change":"2024-09-21T06:30:31.319999+0000","last_active":"2024-09-21T06:30:32.308878+0000","last_peered":"2024-09-21T06:30:32.308878+0000","last_clean":"2024-09-21T06:30:32.308878+0000","last_became_active":"2024-09-21T06:30:31.319691+0000","last_became_peered":"2024-09-21T06:30:31.319691+0000","last_unstale":"2024-09-21T06:30:32.308878+0000","last_undegraded":"2024-09-21T06:30:32.308878+0000","last_fullsized":"2024-09-21T06:30:32.308878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:52.696536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310052+0000","last_change":"2024-09-21T06:30:31.309907+0000","last_active":"2024-09-21T06:30:32.310052+0000","last_peered":"2024-09-21T06:30:32.310052+0000","last_clean":"2024-09-21T06:30:32.310052+0000","last_became_active":"2024-09-21T06:30:31.309605+0000","last_became_peered":"2024-09-21T06:30:31.309605+0000","last_unstale":"2024-09-21T06:30:32.310052+0000","last_undegraded":"2024-09-21T06:30:32.310052+0000","last_fullsized":"2024-09-21T06:30:32.310052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:52.696536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537274+0000","last_change":"2024-09-21T06:30:31.315104+0000","last_active":"2024-09-21T06:30:32.537274+0000","last_peered":"2024-09-21T06:30:32.537274+0000","last_clean":"2024-09-21T06:30:32.537274+0000","last_became_active":"2024-09-21T06:30:31.314752+0000","last_became_peered":"2024-09-21T06:30:31.314752+0000","last_unstale":"2024-09-21T06:30:32.537274+0000","last_undegraded":"2024-09-21T06:30:32.537274+0000","last_fullsized":"2024-09-21T06:30:32.537274+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.017895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310229+0000","last_change":"2024-09-21T06:30:31.326413+0000","last_active":"2024-09-21T06:30:32.310229+0000","last_peered":"2024-09-21T06:30:32.310229+0000","last_clean":"2024-09-21T06:30:32.310229+0000","last_became_active":"2024-09-21T06:30:31.324502+0000","last_became_peered":"2024-09-21T06:30:31.324502+0000","last_unstale":"2024-09-21T06:30:32.310229+0000","last_undegraded":"2024-09-21T06:30:32.310229+0000","last_fullsized":"2024-09-21T06:30:32.310229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:56.813975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310558+0000","last_change":"2024-09-21T06:30:31.318170+0000","last_active":"2024-09-21T06:30:32.310558+0000","last_peered":"2024-09-21T06:30:32.310558+0000","last_clean":"2024-09-21T06:30:32.310558+0000","last_became_active":"2024-09-21T06:30:31.317836+0000","last_became_peered":"2024-09-21T06:30:31.317836+0000","last_unstale":"2024-09-21T06:30:32.310558+0000","last_undegraded":"2024-09-21T06:30:32.310558+0000","last_fullsized":"2024-09-21T06:30:32.310558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.869230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538216+0000","last_change":"2024-09-21T06:30:31.319226+0000","last_active":"2024-09-21T06:30:32.538216+0000","last_peered":"2024-09-21T06:30:32.538216+0000","last_clean":"2024-09-21T06:30:32.538216+0000","last_became_active":"2024-09-21T06:30:31.318861+0000","last_became_peered":"2024-09-21T06:30:31.318861+0000","last_unstale":"2024-09-21T06:30:32.538216+0000","last_undegraded":"2024-09-21T06:30:32.538216+0000","last_fullsized":"2024-09-21T06:30:32.538216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:46.128616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537969+0000","last_change":"2024-09-21T06:30:31.323026+0000","last_active":"2024-09-21T06:30:32.537969+0000","last_peered":"2024-09-21T06:30:32.537969+0000","last_clean":"2024-09-21T06:30:32.537969+0000","last_became_active":"2024-09-21T06:30:31.322653+0000","last_became_peered":"2024-09-21T06:30:31.322653+0000","last_unstale":"2024-09-21T06:30:32.537969+0000","last_undegraded":"2024-09-21T06:30:32.537969+0000","last_fullsized":"2024-09-21T06:30:32.537969+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310201+0000","last_change":"2024-09-21T06:30:31.315492+0000","last_active":"2024-09-21T06:30:32.310201+0000","last_peered":"2024-09-21T06:30:32.310201+0000","last_clean":"2024-09-21T06:30:32.310201+0000","last_became_active":"2024-09-21T06:30:31.311787+0000","last_became_peered":"2024-09-21T06:30:31.311787+0000","last_unstale":"2024-09-21T06:30:32.310201+0000","last_undegraded":"2024-09-21T06:30:32.310201+0000","last_fullsized":"2024-09-21T06:30:32.310201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:10.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537697+0000","last_change":"2024-09-21T06:30:31.312757+0000","last_active":"2024-09-21T06:30:32.537697+0000","last_peered":"2024-09-21T06:30:32.537697+0000","last_clean":"2024-09-21T06:30:32.537697+0000","last_became_active":"2024-09-21T06:30:31.312447+0000","last_became_peered":"2024-09-21T06:30:31.312447+0000","last_unstale":"2024-09-21T06:30:32.537697+0000","last_undegraded":"2024-09-21T06:30:32.537697+0000","last_fullsized":"2024-09-21T06:30:32.537697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:52.551557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538256+0000","last_change":"2024-09-21T06:30:31.327135+0000","last_active":"2024-09-21T06:30:32.538256+0000","last_peered":"2024-09-21T06:30:32.538256+0000","last_clean":"2024-09-21T06:30:32.538256+0000","last_became_active":"2024-09-21T06:30:31.326786+0000","last_became_peered":"2024-09-21T06:30:31.326786+0000","last_unstale":"2024-09-21T06:30:32.538256+0000","last_undegraded":"2024-09-21T06:30:32.538256+0000","last_fullsized":"2024-09-21T06:30:32.538256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:28.565250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537625+0000","last_change":"2024-09-21T06:30:31.315065+0000","last_active":"2024-09-21T06:30:32.537625+0000","last_peered":"2024-09-21T06:30:32.537625+0000","last_clean":"2024-09-21T06:30:32.537625+0000","last_became_active":"2024-09-21T06:30:31.314716+0000","last_became_peered":"2024-09-21T06:30:31.314716+0000","last_unstale":"2024-09-21T06:30:32.537625+0000","last_undegraded":"2024-09-21T06:30:32.537625+0000","last_fullsized":"2024-09-21T06:30:32.537625+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:12.719955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309820+0000","last_change":"2024-09-21T06:30:31.326663+0000","last_active":"2024-09-21T06:30:32.309820+0000","last_peered":"2024-09-21T06:30:32.309820+0000","last_clean":"2024-09-21T06:30:32.309820+0000","last_became_active":"2024-09-21T06:30:31.325002+0000","last_became_peered":"2024-09-21T06:30:31.325002+0000","last_unstale":"2024-09-21T06:30:32.309820+0000","last_undegraded":"2024-09-21T06:30:32.309820+0000","last_fullsized":"2024-09-21T06:30:32.309820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:51.298694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538104+0000","last_change":"2024-09-21T06:30:31.319547+0000","last_active":"2024-09-21T06:30:32.538104+0000","last_peered":"2024-09-21T06:30:32.538104+0000","last_clean":"2024-09-21T06:30:32.538104+0000","last_became_active":"2024-09-21T06:30:31.319226+0000","last_became_peered":"2024-09-21T06:30:31.319226+0000","last_unstale":"2024-09-21T06:30:32.538104+0000","last_undegraded":"2024-09-21T06:30:32.538104+0000","last_fullsized":"2024-09-21T06:30:32.538104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:38.924482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537679+0000","last_change":"2024-09-21T06:30:31.325751+0000","last_active":"2024-09-21T06:30:32.537679+0000","last_peered":"2024-09-21T06:30:32.537679+0000","last_clean":"2024-09-21T06:30:32.537679+0000","last_became_active":"2024-09-21T06:30:31.325456+0000","last_became_peered":"2024-09-21T06:30:31.325456+0000","last_unstale":"2024-09-21T06:30:32.537679+0000","last_undegraded":"2024-09-21T06:30:32.537679+0000","last_fullsized":"2024-09-21T06:30:32.537679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.933118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538520+0000","last_change":"2024-09-21T06:30:31.320518+0000","last_active":"2024-09-21T06:30:32.538520+0000","last_peered":"2024-09-21T06:30:32.538520+0000","last_clean":"2024-09-21T06:30:32.538520+0000","last_became_active":"2024-09-21T06:30:31.320174+0000","last_became_peered":"2024-09-21T06:30:31.320174+0000","last_unstale":"2024-09-21T06:30:32.538520+0000","last_undegraded":"2024-09-21T06:30:32.538520+0000","last_fullsized":"2024-09-21T06:30:32.538520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:48.270905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309146+0000","last_change":"2024-09-21T06:30:31.327891+0000","last_active":"2024-09-21T06:30:32.309146+0000","last_peered":"2024-09-21T06:30:32.309146+0000","last_clean":"2024-09-21T06:30:32.309146+0000","last_became_active":"2024-09-21T06:30:31.327350+0000","last_became_peered":"2024-09-21T06:30:31.327350+0000","last_unstale":"2024-09-21T06:30:32.309146+0000","last_undegraded":"2024-09-21T06:30:32.309146+0000","last_fullsized":"2024-09-21T06:30:32.309146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:27.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537161+0000","last_change":"2024-09-21T06:30:31.319499+0000","last_active":"2024-09-21T06:30:32.537161+0000","last_peered":"2024-09-21T06:30:32.537161+0000","last_clean":"2024-09-21T06:30:32.537161+0000","last_became_active":"2024-09-21T06:30:31.319048+0000","last_became_peered":"2024-09-21T06:30:31.319048+0000","last_unstale":"2024-09-21T06:30:32.537161+0000","last_undegraded":"2024-09-21T06:30:32.537161+0000","last_fullsized":"2024-09-21T06:30:32.537161+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:21.159737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538361+0000","last_change":"2024-09-21T06:30:31.311536+0000","last_active":"2024-09-21T06:30:32.538361+0000","last_peered":"2024-09-21T06:30:32.538361+0000","last_clean":"2024-09-21T06:30:32.538361+0000","last_became_active":"2024-09-21T06:30:31.311207+0000","last_became_peered":"2024-09-21T06:30:31.311207+0000","last_unstale":"2024-09-21T06:30:32.538361+0000","last_undegraded":"2024-09-21T06:30:32.538361+0000","last_fullsized":"2024-09-21T06:30:32.538361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:12.529593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.536605+0000","last_change":"2024-09-21T06:30:31.323803+0000","last_active":"2024-09-21T06:30:32.536605+0000","last_peered":"2024-09-21T06:30:32.536605+0000","last_clean":"2024-09-21T06:30:32.536605+0000","last_became_active":"2024-09-21T06:30:31.322361+0000","last_became_peered":"2024-09-21T06:30:31.322361+0000","last_unstale":"2024-09-21T06:30:32.536605+0000","last_undegraded":"2024-09-21T06:30:32.536605+0000","last_fullsized":"2024-09-21T06:30:32.536605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.017895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537129+0000","last_change":"2024-09-21T06:30:31.315180+0000","last_active":"2024-09-21T06:30:32.537129+0000","last_peered":"2024-09-21T06:30:32.537129+0000","last_clean":"2024-09-21T06:30:32.537129+0000","last_became_active":"2024-09-21T06:30:31.314843+0000","last_became_peered":"2024-09-21T06:30:31.314843+0000","last_unstale":"2024-09-21T06:30:32.537129+0000","last_undegraded":"2024-09-21T06:30:32.537129+0000","last_fullsized":"2024-09-21T06:30:32.537129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:40.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317602+0000","last_change":"2024-09-21T06:30:30.317061+0000","last_active":"2024-09-21T06:30:32.317602+0000","last_peered":"2024-09-21T06:30:32.317602+0000","last_clean":"2024-09-21T06:30:32.317602+0000","last_became_active":"2024-09-21T06:30:30.316746+0000","last_became_peered":"2024-09-21T06:30:30.316746+0000","last_unstale":"2024-09-21T06:30:32.317602+0000","last_undegraded":"2024-09-21T06:30:32.317602+0000","last_fullsized":"2024-09-21T06:30:32.317602+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:42.121089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537018+0000","last_change":"2024-09-21T06:30:31.314676+0000","last_active":"2024-09-21T06:30:32.537018+0000","last_peered":"2024-09-21T06:30:32.537018+0000","last_clean":"2024-09-21T06:30:32.537018+0000","last_became_active":"2024-09-21T06:30:31.312025+0000","last_became_peered":"2024-09-21T06:30:31.312025+0000","last_unstale":"2024-09-21T06:30:32.537018+0000","last_undegraded":"2024-09-21T06:30:32.537018+0000","last_fullsized":"2024-09-21T06:30:32.537018+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:45.069032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538403+0000","last_change":"2024-09-21T06:30:30.325686+0000","last_active":"2024-09-21T06:30:32.538403+0000","last_peered":"2024-09-21T06:30:32.538403+0000","last_clean":"2024-09-21T06:30:32.538403+0000","last_became_active":"2024-09-21T06:30:30.325369+0000","last_became_peered":"2024-09-21T06:30:30.325369+0000","last_unstale":"2024-09-21T06:30:32.538403+0000","last_undegraded":"2024-09-21T06:30:32.538403+0000","last_fullsized":"2024-09-21T06:30:32.538403+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:16.102746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537183+0000","last_change":"2024-09-21T06:30:31.319213+0000","last_active":"2024-09-21T06:30:32.537183+0000","last_peered":"2024-09-21T06:30:32.537183+0000","last_clean":"2024-09-21T06:30:32.537183+0000","last_became_active":"2024-09-21T06:30:31.318555+0000","last_became_peered":"2024-09-21T06:30:31.318555+0000","last_unstale":"2024-09-21T06:30:32.537183+0000","last_undegraded":"2024-09-21T06:30:32.537183+0000","last_fullsized":"2024-09-21T06:30:32.537183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:33.741956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310526+0000","last_change":"2024-09-21T06:30:30.309511+0000","last_active":"2024-09-21T06:30:32.310526+0000","last_peered":"2024-09-21T06:30:32.310526+0000","last_clean":"2024-09-21T06:30:32.310526+0000","last_became_active":"2024-09-21T06:30:30.307785+0000","last_became_peered":"2024-09-21T06:30:30.307785+0000","last_unstale":"2024-09-21T06:30:32.310526+0000","last_undegraded":"2024-09-21T06:30:32.310526+0000","last_fullsized":"2024-09-21T06:30:32.310526+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:37.385089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309631+0000","last_change":"2024-09-21T06:30:31.311405+0000","last_active":"2024-09-21T06:30:32.309631+0000","last_peered":"2024-09-21T06:30:32.309631+0000","last_clean":"2024-09-21T06:30:32.309631+0000","last_became_active":"2024-09-21T06:30:31.311013+0000","last_became_peered":"2024-09-21T06:30:31.311013+0000","last_unstale":"2024-09-21T06:30:32.309631+0000","last_undegraded":"2024-09-21T06:30:32.309631+0000","last_fullsized":"2024-09-21T06:30:32.309631+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:00.184082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538278+0000","last_change":"2024-09-21T06:30:30.325411+0000","last_active":"2024-09-21T06:30:32.538278+0000","last_peered":"2024-09-21T06:30:32.538278+0000","last_clean":"2024-09-21T06:30:32.538278+0000","last_became_active":"2024-09-21T06:30:30.325069+0000","last_became_peered":"2024-09-21T06:30:30.325069+0000","last_unstale":"2024-09-21T06:30:32.538278+0000","last_undegraded":"2024-09-21T06:30:32.538278+0000","last_fullsized":"2024-09-21T06:30:32.538278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:27.217091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317803+0000","last_change":"2024-09-21T06:30:31.332172+0000","last_active":"2024-09-21T06:30:32.317803+0000","last_peered":"2024-09-21T06:30:32.317803+0000","last_clean":"2024-09-21T06:30:32.317803+0000","last_became_active":"2024-09-21T06:30:31.331859+0000","last_became_peered":"2024-09-21T06:30:31.331859+0000","last_unstale":"2024-09-21T06:30:32.317803+0000","last_undegraded":"2024-09-21T06:30:32.317803+0000","last_fullsized":"2024-09-21T06:30:32.317803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:58.149878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538675+0000","last_change":"2024-09-21T06:30:30.314082+0000","last_active":"2024-09-21T06:30:32.538675+0000","last_peered":"2024-09-21T06:30:32.538675+0000","last_clean":"2024-09-21T06:30:32.538675+0000","last_became_active":"2024-09-21T06:30:30.311066+0000","last_became_peered":"2024-09-21T06:30:30.311066+0000","last_unstale":"2024-09-21T06:30:32.538675+0000","last_undegraded":"2024-09-21T06:30:32.538675+0000","last_fullsized":"2024-09-21T06:30:32.538675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:47.579057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537308+0000","last_change":"2024-09-21T06:30:31.325429+0000","last_active":"2024-09-21T06:30:32.537308+0000","last_peered":"2024-09-21T06:30:32.537308+0000","last_clean":"2024-09-21T06:30:32.537308+0000","last_became_active":"2024-09-21T06:30:31.325083+0000","last_became_peered":"2024-09-21T06:30:31.325083+0000","last_unstale":"2024-09-21T06:30:32.537308+0000","last_undegraded":"2024-09-21T06:30:32.537308+0000","last_fullsized":"2024-09-21T06:30:32.537308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.859081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537324+0000","last_change":"2024-09-21T06:30:30.312123+0000","last_active":"2024-09-21T06:30:32.537324+0000","last_peered":"2024-09-21T06:30:32.537324+0000","last_clean":"2024-09-21T06:30:32.537324+0000","last_became_active":"2024-09-21T06:30:30.311786+0000","last_became_peered":"2024-09-21T06:30:30.311786+0000","last_unstale":"2024-09-21T06:30:32.537324+0000","last_undegraded":"2024-09-21T06:30:32.537324+0000","last_fullsized":"2024-09-21T06:30:32.537324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:40.212309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310919+0000","last_change":"2024-09-21T06:30:31.313644+0000","last_active":"2024-09-21T06:30:32.310919+0000","last_peered":"2024-09-21T06:30:32.310919+0000","last_clean":"2024-09-21T06:30:32.310919+0000","last_became_active":"2024-09-21T06:30:31.311070+0000","last_became_peered":"2024-09-21T06:30:31.311070+0000","last_unstale":"2024-09-21T06:30:32.310919+0000","last_undegraded":"2024-09-21T06:30:32.310919+0000","last_fullsized":"2024-09-21T06:30:32.310919+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:16.594637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.536890+0000","last_change":"2024-09-21T06:30:30.317516+0000","last_active":"2024-09-21T06:30:32.536890+0000","last_peered":"2024-09-21T06:30:32.536890+0000","last_clean":"2024-09-21T06:30:32.536890+0000","last_became_active":"2024-09-21T06:30:30.317198+0000","last_became_peered":"2024-09-21T06:30:30.317198+0000","last_unstale":"2024-09-21T06:30:32.536890+0000","last_undegraded":"2024-09-21T06:30:32.536890+0000","last_fullsized":"2024-09-21T06:30:32.536890+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:02.118682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310488+0000","last_change":"2024-09-21T06:30:31.316017+0000","last_active":"2024-09-21T06:30:32.310488+0000","last_peered":"2024-09-21T06:30:32.310488+0000","last_clean":"2024-09-21T06:30:32.310488+0000","last_became_active":"2024-09-21T06:30:31.315664+0000","last_became_peered":"2024-09-21T06:30:31.315664+0000","last_unstale":"2024-09-21T06:30:32.310488+0000","last_undegraded":"2024-09-21T06:30:32.310488+0000","last_fullsized":"2024-09-21T06:30:32.310488+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:50.623685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537764+0000","last_change":"2024-09-21T06:30:30.324244+0000","last_active":"2024-09-21T06:30:32.537764+0000","last_peered":"2024-09-21T06:30:32.537764+0000","last_clean":"2024-09-21T06:30:32.537764+0000","last_became_active":"2024-09-21T06:30:30.323920+0000","last_became_peered":"2024-09-21T06:30:30.323920+0000","last_unstale":"2024-09-21T06:30:32.537764+0000","last_undegraded":"2024-09-21T06:30:32.537764+0000","last_fullsized":"2024-09-21T06:30:32.537764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:54.388458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317849+0000","last_change":"2024-09-21T06:30:31.321983+0000","last_active":"2024-09-21T06:30:32.317849+0000","last_peered":"2024-09-21T06:30:32.317849+0000","last_clean":"2024-09-21T06:30:32.317849+0000","last_became_active":"2024-09-21T06:30:31.321639+0000","last_became_peered":"2024-09-21T06:30:31.321639+0000","last_unstale":"2024-09-21T06:30:32.317849+0000","last_undegraded":"2024-09-21T06:30:32.317849+0000","last_fullsized":"2024-09-21T06:30:32.317849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.404882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310533+0000","last_change":"2024-09-21T06:30:30.307552+0000","last_active":"2024-09-21T06:30:32.310533+0000","last_peered":"2024-09-21T06:30:32.310533+0000","last_clean":"2024-09-21T06:30:32.310533+0000","last_became_active":"2024-09-21T06:30:30.307185+0000","last_became_peered":"2024-09-21T06:30:30.307185+0000","last_unstale":"2024-09-21T06:30:32.310533+0000","last_undegraded":"2024-09-21T06:30:32.310533+0000","last_fullsized":"2024-09-21T06:30:32.310533+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:11.158964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310118+0000","last_change":"2024-09-21T06:30:31.318250+0000","last_active":"2024-09-21T06:30:32.310118+0000","last_peered":"2024-09-21T06:30:32.310118+0000","last_clean":"2024-09-21T06:30:32.310118+0000","last_became_active":"2024-09-21T06:30:31.317877+0000","last_became_peered":"2024-09-21T06:30:31.317877+0000","last_unstale":"2024-09-21T06:30:32.310118+0000","last_undegraded":"2024-09-21T06:30:32.310118+0000","last_fullsized":"2024-09-21T06:30:32.310118+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:27.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310340+0000","last_change":"2024-09-21T06:30:30.317553+0000","last_active":"2024-09-21T06:30:32.310340+0000","last_peered":"2024-09-21T06:30:32.310340+0000","last_clean":"2024-09-21T06:30:32.310340+0000","last_became_active":"2024-09-21T06:30:30.317218+0000","last_became_peered":"2024-09-21T06:30:30.317218+0000","last_unstale":"2024-09-21T06:30:32.310340+0000","last_undegraded":"2024-09-21T06:30:32.310340+0000","last_fullsized":"2024-09-21T06:30:32.310340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:27.217091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309224+0000","last_change":"2024-09-21T06:30:31.329848+0000","last_active":"2024-09-21T06:30:32.309224+0000","last_peered":"2024-09-21T06:30:32.309224+0000","last_clean":"2024-09-21T06:30:32.309224+0000","last_became_active":"2024-09-21T06:30:31.329488+0000","last_became_peered":"2024-09-21T06:30:31.329488+0000","last_unstale":"2024-09-21T06:30:32.309224+0000","last_undegraded":"2024-09-21T06:30:32.309224+0000","last_fullsized":"2024-09-21T06:30:32.309224+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:10.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537330+0000","last_change":"2024-09-21T06:30:30.315505+0000","last_active":"2024-09-21T06:30:32.537330+0000","last_peered":"2024-09-21T06:30:32.537330+0000","last_clean":"2024-09-21T06:30:32.537330+0000","last_became_active":"2024-09-21T06:30:30.315212+0000","last_became_peered":"2024-09-21T06:30:30.315212+0000","last_unstale":"2024-09-21T06:30:32.537330+0000","last_undegraded":"2024-09-21T06:30:32.537330+0000","last_fullsized":"2024-09-21T06:30:32.537330+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:14.517646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537961+0000","last_change":"2024-09-21T06:30:31.317301+0000","last_active":"2024-09-21T06:30:32.537961+0000","last_peered":"2024-09-21T06:30:32.537961+0000","last_clean":"2024-09-21T06:30:32.537961+0000","last_became_active":"2024-09-21T06:30:31.316966+0000","last_became_peered":"2024-09-21T06:30:31.316966+0000","last_unstale":"2024-09-21T06:30:32.537961+0000","last_undegraded":"2024-09-21T06:30:32.537961+0000","last_fullsized":"2024-09-21T06:30:32.537961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.933118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":21,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310710+0000","last_change":"2024-09-21T06:30:30.326062+0000","last_active":"2024-09-21T06:30:32.310710+0000","last_peered":"2024-09-21T06:30:32.310710+0000","last_clean":"2024-09-21T06:30:32.310710+0000","last_became_active":"2024-09-21T06:30:30.325023+0000","last_became_peered":"2024-09-21T06:30:30.325023+0000","last_unstale":"2024-09-21T06:30:32.310710+0000","last_undegraded":"2024-09-21T06:30:32.310710+0000","last_fullsized":"2024-09-21T06:30:32.310710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:27.278349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537417+0000","last_change":"2024-09-21T06:30:31.330173+0000","last_active":"2024-09-21T06:30:32.537417+0000","last_peered":"2024-09-21T06:30:32.537417+0000","last_clean":"2024-09-21T06:30:32.537417+0000","last_became_active":"2024-09-21T06:30:31.329361+0000","last_became_peered":"2024-09-21T06:30:31.329361+0000","last_unstale":"2024-09-21T06:30:32.537417+0000","last_undegraded":"2024-09-21T06:30:32.537417+0000","last_fullsized":"2024-09-21T06:30:32.537417+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:14.993582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538460+0000","last_change":"2024-09-21T06:30:30.321481+0000","last_active":"2024-09-21T06:30:32.538460+0000","last_peered":"2024-09-21T06:30:32.538460+0000","last_clean":"2024-09-21T06:30:32.538460+0000","last_became_active":"2024-09-21T06:30:30.321104+0000","last_became_peered":"2024-09-21T06:30:30.321104+0000","last_unstale":"2024-09-21T06:30:32.538460+0000","last_undegraded":"2024-09-21T06:30:32.538460+0000","last_fullsized":"2024-09-21T06:30:32.538460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:24.050874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309559+0000","last_change":"2024-09-21T06:30:31.329600+0000","last_active":"2024-09-21T06:30:32.309559+0000","last_peered":"2024-09-21T06:30:32.309559+0000","last_clean":"2024-09-21T06:30:32.309559+0000","last_became_active":"2024-09-21T06:30:31.328920+0000","last_became_peered":"2024-09-21T06:30:31.328920+0000","last_unstale":"2024-09-21T06:30:32.309559+0000","last_undegraded":"2024-09-21T06:30:32.309559+0000","last_fullsized":"2024-09-21T06:30:32.309559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.869230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309781+0000","last_change":"2024-09-21T06:30:30.308068+0000","last_active":"2024-09-21T06:30:32.309781+0000","last_peered":"2024-09-21T06:30:32.309781+0000","last_clean":"2024-09-21T06:30:32.309781+0000","last_became_active":"2024-09-21T06:30:30.307766+0000","last_became_peered":"2024-09-21T06:30:30.307766+0000","last_unstale":"2024-09-21T06:30:32.309781+0000","last_undegraded":"2024-09-21T06:30:32.309781+0000","last_fullsized":"2024-09-21T06:30:32.309781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:29.037949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538640+0000","last_change":"2024-09-21T06:30:31.332842+0000","last_active":"2024-09-21T06:30:32.538640+0000","last_peered":"2024-09-21T06:30:32.538640+0000","last_clean":"2024-09-21T06:30:32.538640+0000","last_became_active":"2024-09-21T06:30:31.329569+0000","last_became_peered":"2024-09-21T06:30:31.329569+0000","last_unstale":"2024-09-21T06:30:32.538640+0000","last_undegraded":"2024-09-21T06:30:32.538640+0000","last_fullsized":"2024-09-21T06:30:32.538640+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:08.739252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537346+0000","last_change":"2024-09-21T06:30:30.319925+0000","last_active":"2024-09-21T06:30:32.537346+0000","last_peered":"2024-09-21T06:30:32.537346+0000","last_clean":"2024-09-21T06:30:32.537346+0000","last_became_active":"2024-09-21T06:30:30.319578+0000","last_became_peered":"2024-09-21T06:30:30.319578+0000","last_unstale":"2024-09-21T06:30:32.537346+0000","last_undegraded":"2024-09-21T06:30:32.537346+0000","last_fullsized":"2024-09-21T06:30:32.537346+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:24.050874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538661+0000","last_change":"2024-09-21T06:30:31.315221+0000","last_active":"2024-09-21T06:30:32.538661+0000","last_peered":"2024-09-21T06:30:32.538661+0000","last_clean":"2024-09-21T06:30:32.538661+0000","last_became_active":"2024-09-21T06:30:31.314845+0000","last_became_peered":"2024-09-21T06:30:31.314845+0000","last_unstale":"2024-09-21T06:30:32.538661+0000","last_undegraded":"2024-09-21T06:30:32.538661+0000","last_fullsized":"2024-09-21T06:30:32.538661+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:21.914277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537429+0000","last_change":"2024-09-21T06:30:30.327896+0000","last_active":"2024-09-21T06:30:32.537429+0000","last_peered":"2024-09-21T06:30:32.537429+0000","last_clean":"2024-09-21T06:30:32.537429+0000","last_became_active":"2024-09-21T06:30:30.327275+0000","last_became_peered":"2024-09-21T06:30:30.327275+0000","last_unstale":"2024-09-21T06:30:32.537429+0000","last_undegraded":"2024-09-21T06:30:32.537429+0000","last_fullsized":"2024-09-21T06:30:32.537429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.451060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309058+0000","last_change":"2024-09-21T06:30:31.318205+0000","last_active":"2024-09-21T06:30:32.309058+0000","last_peered":"2024-09-21T06:30:32.309058+0000","last_clean":"2024-09-21T06:30:32.309058+0000","last_became_active":"2024-09-21T06:30:31.317874+0000","last_became_peered":"2024-09-21T06:30:31.317874+0000","last_unstale":"2024-09-21T06:30:32.309058+0000","last_undegraded":"2024-09-21T06:30:32.309058+0000","last_fullsized":"2024-09-21T06:30:32.309058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.110817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.539079+0000","last_change":"2024-09-21T06:30:30.327321+0000","last_active":"2024-09-21T06:30:32.539079+0000","last_peered":"2024-09-21T06:30:32.539079+0000","last_clean":"2024-09-21T06:30:32.539079+0000","last_became_active":"2024-09-21T06:30:30.326926+0000","last_became_peered":"2024-09-21T06:30:30.326926+0000","last_unstale":"2024-09-21T06:30:32.539079+0000","last_undegraded":"2024-09-21T06:30:32.539079+0000","last_fullsized":"2024-09-21T06:30:32.539079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.308829+0000","last_change":"2024-09-21T06:30:31.322078+0000","last_active":"2024-09-21T06:30:32.308829+0000","last_peered":"2024-09-21T06:30:32.308829+0000","last_clean":"2024-09-21T06:30:32.308829+0000","last_became_active":"2024-09-21T06:30:31.321487+0000","last_became_peered":"2024-09-21T06:30:31.321487+0000","last_unstale":"2024-09-21T06:30:32.308829+0000","last_undegraded":"2024-09-21T06:30:32.308829+0000","last_fullsized":"2024-09-21T06:30:32.308829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538805+0000","last_change":"2024-09-21T06:30:30.320839+0000","last_active":"2024-09-21T06:30:32.538805+0000","last_peered":"2024-09-21T06:30:32.538805+0000","last_clean":"2024-09-21T06:30:32.538805+0000","last_became_active":"2024-09-21T06:30:30.320344+0000","last_became_peered":"2024-09-21T06:30:30.320344+0000","last_unstale":"2024-09-21T06:30:32.538805+0000","last_undegraded":"2024-09-21T06:30:32.538805+0000","last_fullsized":"2024-09-21T06:30:32.538805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:56.406494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537669+0000","last_change":"2024-09-21T06:30:31.324775+0000","last_active":"2024-09-21T06:30:32.537669+0000","last_peered":"2024-09-21T06:30:32.537669+0000","last_clean":"2024-09-21T06:30:32.537669+0000","last_became_active":"2024-09-21T06:30:31.322070+0000","last_became_peered":"2024-09-21T06:30:31.322070+0000","last_unstale":"2024-09-21T06:30:32.537669+0000","last_undegraded":"2024-09-21T06:30:32.537669+0000","last_fullsized":"2024-09-21T06:30:32.537669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:22.628559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310226+0000","last_change":"2024-09-21T06:30:30.313933+0000","last_active":"2024-09-21T06:30:32.310226+0000","last_peered":"2024-09-21T06:30:32.310226+0000","last_clean":"2024-09-21T06:30:32.310226+0000","last_became_active":"2024-09-21T06:30:30.313460+0000","last_became_peered":"2024-09-21T06:30:30.313460+0000","last_unstale":"2024-09-21T06:30:32.310226+0000","last_undegraded":"2024-09-21T06:30:32.310226+0000","last_fullsized":"2024-09-21T06:30:32.310226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.629719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537999+0000","last_change":"2024-09-21T06:30:31.333687+0000","last_active":"2024-09-21T06:30:32.537999+0000","last_peered":"2024-09-21T06:30:32.537999+0000","last_clean":"2024-09-21T06:30:32.537999+0000","last_became_active":"2024-09-21T06:30:31.326638+0000","last_became_peered":"2024-09-21T06:30:31.326638+0000","last_unstale":"2024-09-21T06:30:32.537999+0000","last_undegraded":"2024-09-21T06:30:32.537999+0000","last_fullsized":"2024-09-21T06:30:32.537999+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.923134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538350+0000","last_change":"2024-09-21T06:30:30.309923+0000","last_active":"2024-09-21T06:30:32.538350+0000","last_peered":"2024-09-21T06:30:32.538350+0000","last_clean":"2024-09-21T06:30:32.538350+0000","last_became_active":"2024-09-21T06:30:30.309628+0000","last_became_peered":"2024-09-21T06:30:30.309628+0000","last_unstale":"2024-09-21T06:30:32.538350+0000","last_undegraded":"2024-09-21T06:30:32.538350+0000","last_fullsized":"2024-09-21T06:30:32.538350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:11.158964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538187+0000","last_change":"2024-09-21T06:30:31.322754+0000","last_active":"2024-09-21T06:30:32.538187+0000","last_peered":"2024-09-21T06:30:32.538187+0000","last_clean":"2024-09-21T06:30:32.538187+0000","last_became_active":"2024-09-21T06:30:31.322414+0000","last_became_peered":"2024-09-21T06:30:31.322414+0000","last_unstale":"2024-09-21T06:30:32.538187+0000","last_undegraded":"2024-09-21T06:30:32.538187+0000","last_fullsized":"2024-09-21T06:30:32.538187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:08.739252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309005+0000","last_change":"2024-09-21T06:30:30.318255+0000","last_active":"2024-09-21T06:30:32.309005+0000","last_peered":"2024-09-21T06:30:32.309005+0000","last_clean":"2024-09-21T06:30:32.309005+0000","last_became_active":"2024-09-21T06:30:30.317860+0000","last_became_peered":"2024-09-21T06:30:30.317860+0000","last_unstale":"2024-09-21T06:30:32.309005+0000","last_undegraded":"2024-09-21T06:30:32.309005+0000","last_fullsized":"2024-09-21T06:30:32.309005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:07.459885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309981+0000","last_change":"2024-09-21T06:30:31.328769+0000","last_active":"2024-09-21T06:30:32.309981+0000","last_peered":"2024-09-21T06:30:32.309981+0000","last_clean":"2024-09-21T06:30:32.309981+0000","last_became_active":"2024-09-21T06:30:31.328406+0000","last_became_peered":"2024-09-21T06:30:31.328406+0000","last_unstale":"2024-09-21T06:30:32.309981+0000","last_undegraded":"2024-09-21T06:30:32.309981+0000","last_fullsized":"2024-09-21T06:30:32.309981+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.404882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537868+0000","last_change":"2024-09-21T06:30:30.322668+0000","last_active":"2024-09-21T06:30:32.537868+0000","last_peered":"2024-09-21T06:30:32.537868+0000","last_clean":"2024-09-21T06:30:32.537868+0000","last_became_active":"2024-09-21T06:30:30.322292+0000","last_became_peered":"2024-09-21T06:30:30.322292+0000","last_unstale":"2024-09-21T06:30:32.537868+0000","last_undegraded":"2024-09-21T06:30:32.537868+0000","last_fullsized":"2024-09-21T06:30:32.537868+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.815465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310148+0000","last_change":"2024-09-21T06:30:31.334491+0000","last_active":"2024-09-21T06:30:32.310148+0000","last_peered":"2024-09-21T06:30:32.310148+0000","last_clean":"2024-09-21T06:30:32.310148+0000","last_became_active":"2024-09-21T06:30:31.334148+0000","last_became_peered":"2024-09-21T06:30:31.334148+0000","last_unstale":"2024-09-21T06:30:32.310148+0000","last_undegraded":"2024-09-21T06:30:32.310148+0000","last_fullsized":"2024-09-21T06:30:32.310148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:24.600225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310226+0000","last_change":"2024-09-21T06:30:30.325380+0000","last_active":"2024-09-21T06:30:32.310226+0000","last_peered":"2024-09-21T06:30:32.310226+0000","last_clean":"2024-09-21T06:30:32.310226+0000","last_became_active":"2024-09-21T06:30:30.324901+0000","last_became_peered":"2024-09-21T06:30:30.324901+0000","last_unstale":"2024-09-21T06:30:32.310226+0000","last_undegraded":"2024-09-21T06:30:32.310226+0000","last_fullsized":"2024-09-21T06:30:32.310226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:14.551912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538993+0000","last_change":"2024-09-21T06:30:31.321217+0000","last_active":"2024-09-21T06:30:32.538993+0000","last_peered":"2024-09-21T06:30:32.538993+0000","last_clean":"2024-09-21T06:30:32.538993+0000","last_became_active":"2024-09-21T06:30:31.320897+0000","last_became_peered":"2024-09-21T06:30:31.320897+0000","last_unstale":"2024-09-21T06:30:32.538993+0000","last_undegraded":"2024-09-21T06:30:32.538993+0000","last_fullsized":"2024-09-21T06:30:32.538993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:48.270905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309823+0000","last_change":"2024-09-21T06:30:30.314670+0000","last_active":"2024-09-21T06:30:32.309823+0000","last_peered":"2024-09-21T06:30:32.309823+0000","last_clean":"2024-09-21T06:30:32.309823+0000","last_became_active":"2024-09-21T06:30:30.314379+0000","last_became_peered":"2024-09-21T06:30:30.314379+0000","last_unstale":"2024-09-21T06:30:32.309823+0000","last_undegraded":"2024-09-21T06:30:32.309823+0000","last_fullsized":"2024-09-21T06:30:32.309823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:18.067883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309855+0000","last_change":"2024-09-21T06:30:31.313697+0000","last_active":"2024-09-21T06:30:32.309855+0000","last_peered":"2024-09-21T06:30:32.309855+0000","last_clean":"2024-09-21T06:30:32.309855+0000","last_became_active":"2024-09-21T06:30:31.313368+0000","last_became_peered":"2024-09-21T06:30:31.313368+0000","last_unstale":"2024-09-21T06:30:32.309855+0000","last_undegraded":"2024-09-21T06:30:32.309855+0000","last_fullsized":"2024-09-21T06:30:32.309855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:58.149878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537009+0000","last_change":"2024-09-21T06:30:30.317004+0000","last_active":"2024-09-21T06:30:32.537009+0000","last_peered":"2024-09-21T06:30:32.537009+0000","last_clean":"2024-09-21T06:30:32.537009+0000","last_became_active":"2024-09-21T06:30:30.316663+0000","last_became_peered":"2024-09-21T06:30:30.316663+0000","last_unstale":"2024-09-21T06:30:32.537009+0000","last_undegraded":"2024-09-21T06:30:32.537009+0000","last_fullsized":"2024-09-21T06:30:32.537009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:51.404585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309541+0000","last_change":"2024-09-21T06:30:31.314282+0000","last_active":"2024-09-21T06:30:32.309541+0000","last_peered":"2024-09-21T06:30:32.309541+0000","last_clean":"2024-09-21T06:30:32.309541+0000","last_became_active":"2024-09-21T06:30:31.313928+0000","last_became_peered":"2024-09-21T06:30:31.313928+0000","last_unstale":"2024-09-21T06:30:32.309541+0000","last_undegraded":"2024-09-21T06:30:32.309541+0000","last_fullsized":"2024-09-21T06:30:32.309541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:01.706501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537388+0000","last_change":"2024-09-21T06:30:30.317374+0000","last_active":"2024-09-21T06:30:32.537388+0000","last_peered":"2024-09-21T06:30:32.537388+0000","last_clean":"2024-09-21T06:30:32.537388+0000","last_became_active":"2024-09-21T06:30:30.316456+0000","last_became_peered":"2024-09-21T06:30:30.316456+0000","last_unstale":"2024-09-21T06:30:32.537388+0000","last_undegraded":"2024-09-21T06:30:32.537388+0000","last_fullsized":"2024-09-21T06:30:32.537388+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:03.059109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538711+0000","last_change":"2024-09-21T06:30:31.317104+0000","last_active":"2024-09-21T06:30:32.538711+0000","last_peered":"2024-09-21T06:30:32.538711+0000","last_clean":"2024-09-21T06:30:32.538711+0000","last_became_active":"2024-09-21T06:30:31.316760+0000","last_became_peered":"2024-09-21T06:30:31.316760+0000","last_unstale":"2024-09-21T06:30:32.538711+0000","last_undegraded":"2024-09-21T06:30:32.538711+0000","last_fullsized":"2024-09-21T06:30:32.538711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:15.239306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537771+0000","last_change":"2024-09-21T06:30:30.313471+0000","last_active":"2024-09-21T06:30:32.537771+0000","last_peered":"2024-09-21T06:30:32.537771+0000","last_clean":"2024-09-21T06:30:32.537771+0000","last_became_active":"2024-09-21T06:30:30.313139+0000","last_became_peered":"2024-09-21T06:30:30.313139+0000","last_unstale":"2024-09-21T06:30:32.537771+0000","last_undegraded":"2024-09-21T06:30:32.537771+0000","last_fullsized":"2024-09-21T06:30:32.537771+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:44.906990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310634+0000","last_change":"2024-09-21T06:30:31.319796+0000","last_active":"2024-09-21T06:30:32.310634+0000","last_peered":"2024-09-21T06:30:32.310634+0000","last_clean":"2024-09-21T06:30:32.310634+0000","last_became_active":"2024-09-21T06:30:31.318515+0000","last_became_peered":"2024-09-21T06:30:31.318515+0000","last_unstale":"2024-09-21T06:30:32.310634+0000","last_undegraded":"2024-09-21T06:30:32.310634+0000","last_fullsized":"2024-09-21T06:30:32.310634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:45.378128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310100+0000","last_change":"2024-09-21T06:30:30.317597+0000","last_active":"2024-09-21T06:30:32.310100+0000","last_peered":"2024-09-21T06:30:32.310100+0000","last_clean":"2024-09-21T06:30:32.310100+0000","last_became_active":"2024-09-21T06:30:30.316722+0000","last_became_peered":"2024-09-21T06:30:30.316722+0000","last_unstale":"2024-09-21T06:30:32.310100+0000","last_undegraded":"2024-09-21T06:30:32.310100+0000","last_fullsized":"2024-09-21T06:30:32.310100+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:47.786268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538161+0000","last_change":"2024-09-21T06:30:31.312996+0000","last_active":"2024-09-21T06:30:32.538161+0000","last_peered":"2024-09-21T06:30:32.538161+0000","last_clean":"2024-09-21T06:30:32.538161+0000","last_became_active":"2024-09-21T06:30:31.312693+0000","last_became_peered":"2024-09-21T06:30:31.312693+0000","last_unstale":"2024-09-21T06:30:32.538161+0000","last_undegraded":"2024-09-21T06:30:32.538161+0000","last_fullsized":"2024-09-21T06:30:32.538161+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.142991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538102+0000","last_change":"2024-09-21T06:30:30.322601+0000","last_active":"2024-09-21T06:30:32.538102+0000","last_peered":"2024-09-21T06:30:32.538102+0000","last_clean":"2024-09-21T06:30:32.538102+0000","last_became_active":"2024-09-21T06:30:30.322261+0000","last_became_peered":"2024-09-21T06:30:30.322261+0000","last_unstale":"2024-09-21T06:30:32.538102+0000","last_undegraded":"2024-09-21T06:30:32.538102+0000","last_fullsized":"2024-09-21T06:30:32.538102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:45.567092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.311002+0000","last_change":"2024-09-21T06:30:31.313439+0000","last_active":"2024-09-21T06:30:32.311002+0000","last_peered":"2024-09-21T06:30:32.311002+0000","last_clean":"2024-09-21T06:30:32.311002+0000","last_became_active":"2024-09-21T06:30:31.312889+0000","last_became_peered":"2024-09-21T06:30:31.312889+0000","last_unstale":"2024-09-21T06:30:32.311002+0000","last_undegraded":"2024-09-21T06:30:32.311002+0000","last_fullsized":"2024-09-21T06:30:32.311002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:24.498593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537491+0000","last_change":"2024-09-21T06:30:30.313209+0000","last_active":"2024-09-21T06:30:32.537491+0000","last_peered":"2024-09-21T06:30:32.537491+0000","last_clean":"2024-09-21T06:30:32.537491+0000","last_became_active":"2024-09-21T06:30:30.312858+0000","last_became_peered":"2024-09-21T06:30:30.312858+0000","last_unstale":"2024-09-21T06:30:32.537491+0000","last_undegraded":"2024-09-21T06:30:32.537491+0000","last_fullsized":"2024-09-21T06:30:32.537491+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538944+0000","last_change":"2024-09-21T06:30:31.332947+0000","last_active":"2024-09-21T06:30:32.538944+0000","last_peered":"2024-09-21T06:30:32.538944+0000","last_clean":"2024-09-21T06:30:32.538944+0000","last_became_active":"2024-09-21T06:30:31.328041+0000","last_became_peered":"2024-09-21T06:30:31.328041+0000","last_unstale":"2024-09-21T06:30:32.538944+0000","last_undegraded":"2024-09-21T06:30:32.538944+0000","last_fullsized":"2024-09-21T06:30:32.538944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:42.542980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310076+0000","last_change":"2024-09-21T06:30:30.317313+0000","last_active":"2024-09-21T06:30:32.310076+0000","last_peered":"2024-09-21T06:30:32.310076+0000","last_clean":"2024-09-21T06:30:32.310076+0000","last_became_active":"2024-09-21T06:30:30.316807+0000","last_became_peered":"2024-09-21T06:30:30.316807+0000","last_unstale":"2024-09-21T06:30:32.310076+0000","last_undegraded":"2024-09-21T06:30:32.310076+0000","last_fullsized":"2024-09-21T06:30:32.310076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317568+0000","last_change":"2024-09-21T06:30:31.330489+0000","last_active":"2024-09-21T06:30:32.317568+0000","last_peered":"2024-09-21T06:30:32.317568+0000","last_clean":"2024-09-21T06:30:32.317568+0000","last_became_active":"2024-09-21T06:30:31.330195+0000","last_became_peered":"2024-09-21T06:30:31.330195+0000","last_unstale":"2024-09-21T06:30:32.317568+0000","last_undegraded":"2024-09-21T06:30:32.317568+0000","last_fullsized":"2024-09-21T06:30:32.317568+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:50.599219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.311011+0000","last_change":"2024-09-21T06:30:30.313879+0000","last_active":"2024-09-21T06:30:32.311011+0000","last_peered":"2024-09-21T06:30:32.311011+0000","last_clean":"2024-09-21T06:30:32.311011+0000","last_became_active":"2024-09-21T06:30:30.310301+0000","last_became_peered":"2024-09-21T06:30:30.310301+0000","last_unstale":"2024-09-21T06:30:32.311011+0000","last_undegraded":"2024-09-21T06:30:32.311011+0000","last_fullsized":"2024-09-21T06:30:32.311011+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.990441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309936+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:30:32.309936+0000","last_peered":"2024-09-21T06:30:32.309936+0000","last_clean":"2024-09-21T06:30:32.309936+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:30:32.309936+0000","last_undegraded":"2024-09-21T06:30:32.309936+0000","last_fullsized":"2024-09-21T06:30:32.309936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":643072,"data_stored":281133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":643072,"data_stored":276089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":281111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1220608,"data_stored":864823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":873147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:30:35.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:30:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 -- 172.21.15.37:0/2757188755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac406d9c0 msgr2=0x7f8ac4154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 --2- 172.21.15.37:0/2757188755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac406d9c0 0x7f8ac4154ea0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8abc005fd0 tx=0x7f8abc02aa90 comp rx=0 tx=0).stop 2024-09-21T06:30:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 -- 172.21.15.37:0/2757188755 shutdown_connections 2024-09-21T06:30:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 --2- 172.21.15.37:0/2757188755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac406d9c0 0x7f8ac4154ea0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 -- 172.21.15.37:0/2757188755 >> 172.21.15.37:0/2757188755 conn(0x7f8ac40d1930 msgr2=0x7f8ac40d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 -- 172.21.15.37:0/2757188755 shutdown_connections 2024-09-21T06:30:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.061+0000 7f8ac3200640 1 -- 172.21.15.37:0/2757188755 wait complete. 2024-09-21T06:30:36.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.063+0000 7f8ac3200640 1 Processor -- start 2024-09-21T06:30:36.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.063+0000 7f8ac3200640 1 -- start start 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.064+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ac406d9c0 0x7f8ac414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac414dbc0 0x7f8ac41f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4155560 con 0x7f8ac40b9e40 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41553e0 con 0x7f8ac414dbc0 2024-09-21T06:30:36.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac3200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac41556e0 con 0x7f8ac406d9c0 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43854/0 (socket says 172.21.15.37:43854) 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 -- 172.21.15.37:0/135478071 learned_addr learned my addr 172.21.15.37:0/135478071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac1e00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ac406d9c0 0x7f8ac414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8ac2800640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac414dbc0 0x7f8ac41f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 -- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ac406d9c0 msgr2=0x7f8ac414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ac406d9c0 0x7f8ac414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 -- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac414dbc0 msgr2=0x7f8ac41f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac414dbc0 0x7f8ac41f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.065+0000 7f8abbe00640 1 -- 172.21.15.37:0/135478071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac41f9c50 con 0x7f8ac40b9e40 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8abbe00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b3f0 tx=0x7f8aac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac015020 con 0x7f8ac40b9e40 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aac011920 con 0x7f8ac40b9e40 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac414ef00 con 0x7f8ac40b9e40 2024-09-21T06:30:36.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac011d70 con 0x7f8ac40b9e40 2024-09-21T06:30:36.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.066+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41f8f10 con 0x7f8ac40b9e40 2024-09-21T06:30:36.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.067+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac40b7b30 con 0x7f8ac40b9e40 2024-09-21T06:30:36.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.068+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8aac018020 con 0x7f8ac40b9e40 2024-09-21T06:30:36.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.069+0000 7f8aba000640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 0x7f8aa007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.069+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8aac09a9c0 con 0x7f8ac40b9e40 2024-09-21T06:30:36.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.070+0000 7f8ac1e00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 0x7f8aa007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.070+0000 7f8ac1e00640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 0x7f8aa007b0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4001720 tx=0x7f8ab4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:36.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.074+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ac40b7b30 con 0x7f8ac40b9e40 2024-09-21T06:30:36.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.266+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ac40d03f0 con 0x7f8aa0078b90 2024-09-21T06:30:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.271+0000 7f8aba000640 1 -- 172.21.15.37:0/135478071 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f8ac40d03f0 con 0x7f8aa0078b90 2024-09-21T06:30:36.276 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:36.277 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:30:36.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 msgr2=0x7f8aa007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 0x7f8aa007b0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4001720 tx=0x7f8ab4009040 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 msgr2=0x7f8ac414d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b3f0 tx=0x7f8aac00b8b0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 shutdown_connections 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8aa0078b90 0x7f8aa007b0a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ac40b9e40 0x7f8ac414d550 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac414dbc0 0x7f8ac41f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 --2- 172.21.15.37:0/135478071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ac406d9c0 0x7f8ac414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 >> 172.21.15.37:0/135478071 conn(0x7f8ac40d1930 msgr2=0x7f8ac41606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 shutdown_connections 2024-09-21T06:30:36.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.275+0000 7f8ac3200640 1 -- 172.21.15.37:0/135478071 wait complete. 2024-09-21T06:30:36.299 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-21T06:30:35.128519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234896,"kb_used_data":6160,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739824,"statfs":{"total":773094113280,"available":772853579776,"internally_reserved":0,"allocated":6307840,"data_stored":3420788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.845427"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309641+0000","last_change":"2024-09-21T06:30:31.325431+0000","last_active":"2024-09-21T06:30:32.309641+0000","last_peered":"2024-09-21T06:30:32.309641+0000","last_clean":"2024-09-21T06:30:32.309641+0000","last_became_active":"2024-09-21T06:30:31.324477+0000","last_became_peered":"2024-09-21T06:30:31.324477+0000","last_unstale":"2024-09-21T06:30:32.309641+0000","last_undegraded":"2024-09-21T06:30:32.309641+0000","last_fullsized":"2024-09-21T06:30:32.309641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:11.580508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538386+0000","last_change":"2024-09-21T06:30:31.331122+0000","last_active":"2024-09-21T06:30:32.538386+0000","last_peered":"2024-09-21T06:30:32.538386+0000","last_clean":"2024-09-21T06:30:32.538386+0000","last_became_active":"2024-09-21T06:30:31.326039+0000","last_became_peered":"2024-09-21T06:30:31.326039+0000","last_unstale":"2024-09-21T06:30:32.538386+0000","last_undegraded":"2024-09-21T06:30:32.538386+0000","last_fullsized":"2024-09-21T06:30:32.538386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:42.542980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538226+0000","last_change":"2024-09-21T06:30:31.323579+0000","last_active":"2024-09-21T06:30:32.538226+0000","last_peered":"2024-09-21T06:30:32.538226+0000","last_clean":"2024-09-21T06:30:32.538226+0000","last_became_active":"2024-09-21T06:30:31.323235+0000","last_became_peered":"2024-09-21T06:30:31.323235+0000","last_unstale":"2024-09-21T06:30:32.538226+0000","last_undegraded":"2024-09-21T06:30:32.538226+0000","last_fullsized":"2024-09-21T06:30:32.538226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:34.886039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310028+0000","last_change":"2024-09-21T06:30:31.319171+0000","last_active":"2024-09-21T06:30:32.310028+0000","last_peered":"2024-09-21T06:30:32.310028+0000","last_clean":"2024-09-21T06:30:32.310028+0000","last_became_active":"2024-09-21T06:30:31.318852+0000","last_became_peered":"2024-09-21T06:30:31.318852+0000","last_unstale":"2024-09-21T06:30:32.310028+0000","last_undegraded":"2024-09-21T06:30:32.310028+0000","last_fullsized":"2024-09-21T06:30:32.310028+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537885+0000","last_change":"2024-09-21T06:30:31.321094+0000","last_active":"2024-09-21T06:30:32.537885+0000","last_peered":"2024-09-21T06:30:32.537885+0000","last_clean":"2024-09-21T06:30:32.537885+0000","last_became_active":"2024-09-21T06:30:31.320370+0000","last_became_peered":"2024-09-21T06:30:31.320370+0000","last_unstale":"2024-09-21T06:30:32.537885+0000","last_undegraded":"2024-09-21T06:30:32.537885+0000","last_fullsized":"2024-09-21T06:30:32.537885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:58.090005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537501+0000","last_change":"2024-09-21T06:30:31.330820+0000","last_active":"2024-09-21T06:30:32.537501+0000","last_peered":"2024-09-21T06:30:32.537501+0000","last_clean":"2024-09-21T06:30:32.537501+0000","last_became_active":"2024-09-21T06:30:31.330534+0000","last_became_peered":"2024-09-21T06:30:31.330534+0000","last_unstale":"2024-09-21T06:30:32.537501+0000","last_undegraded":"2024-09-21T06:30:32.537501+0000","last_fullsized":"2024-09-21T06:30:32.537501+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:22.628559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310423+0000","last_change":"2024-09-21T06:30:31.315315+0000","last_active":"2024-09-21T06:30:32.310423+0000","last_peered":"2024-09-21T06:30:32.310423+0000","last_clean":"2024-09-21T06:30:32.310423+0000","last_became_active":"2024-09-21T06:30:31.315006+0000","last_became_peered":"2024-09-21T06:30:31.315006+0000","last_unstale":"2024-09-21T06:30:32.310423+0000","last_undegraded":"2024-09-21T06:30:32.310423+0000","last_fullsized":"2024-09-21T06:30:32.310423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.674377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537263+0000","last_change":"2024-09-21T06:30:31.325015+0000","last_active":"2024-09-21T06:30:32.537263+0000","last_peered":"2024-09-21T06:30:32.537263+0000","last_clean":"2024-09-21T06:30:32.537263+0000","last_became_active":"2024-09-21T06:30:31.324713+0000","last_became_peered":"2024-09-21T06:30:31.324713+0000","last_unstale":"2024-09-21T06:30:32.537263+0000","last_undegraded":"2024-09-21T06:30:32.537263+0000","last_fullsized":"2024-09-21T06:30:32.537263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:03.771004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538533+0000","last_change":"2024-09-21T06:30:31.327858+0000","last_active":"2024-09-21T06:30:32.538533+0000","last_peered":"2024-09-21T06:30:32.538533+0000","last_clean":"2024-09-21T06:30:32.538533+0000","last_became_active":"2024-09-21T06:30:31.326704+0000","last_became_peered":"2024-09-21T06:30:31.326704+0000","last_unstale":"2024-09-21T06:30:32.538533+0000","last_undegraded":"2024-09-21T06:30:32.538533+0000","last_fullsized":"2024-09-21T06:30:32.538533+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.222459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310851+0000","last_change":"2024-09-21T06:30:31.334860+0000","last_active":"2024-09-21T06:30:32.310851+0000","last_peered":"2024-09-21T06:30:32.310851+0000","last_clean":"2024-09-21T06:30:32.310851+0000","last_became_active":"2024-09-21T06:30:31.334387+0000","last_became_peered":"2024-09-21T06:30:31.334387+0000","last_unstale":"2024-09-21T06:30:32.310851+0000","last_undegraded":"2024-09-21T06:30:32.310851+0000","last_fullsized":"2024-09-21T06:30:32.310851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:30.557490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309518+0000","last_change":"2024-09-21T06:30:31.321566+0000","last_active":"2024-09-21T06:30:32.309518+0000","last_peered":"2024-09-21T06:30:32.309518+0000","last_clean":"2024-09-21T06:30:32.309518+0000","last_became_active":"2024-09-21T06:30:31.321066+0000","last_became_peered":"2024-09-21T06:30:31.321066+0000","last_unstale":"2024-09-21T06:30:32.309518+0000","last_undegraded":"2024-09-21T06:30:32.309518+0000","last_fullsized":"2024-09-21T06:30:32.309518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:50.623685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537639+0000","last_change":"2024-09-21T06:30:31.322990+0000","last_active":"2024-09-21T06:30:32.537639+0000","last_peered":"2024-09-21T06:30:32.537639+0000","last_clean":"2024-09-21T06:30:32.537639+0000","last_became_active":"2024-09-21T06:30:31.322551+0000","last_became_peered":"2024-09-21T06:30:31.322551+0000","last_unstale":"2024-09-21T06:30:32.537639+0000","last_undegraded":"2024-09-21T06:30:32.537639+0000","last_fullsized":"2024-09-21T06:30:32.537639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:58.090005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538261+0000","last_change":"2024-09-21T06:30:31.330123+0000","last_active":"2024-09-21T06:30:32.538261+0000","last_peered":"2024-09-21T06:30:32.538261+0000","last_clean":"2024-09-21T06:30:32.538261+0000","last_became_active":"2024-09-21T06:30:31.327189+0000","last_became_peered":"2024-09-21T06:30:31.327189+0000","last_unstale":"2024-09-21T06:30:32.538261+0000","last_undegraded":"2024-09-21T06:30:32.538261+0000","last_fullsized":"2024-09-21T06:30:32.538261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.142991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.308878+0000","last_change":"2024-09-21T06:30:31.319999+0000","last_active":"2024-09-21T06:30:32.308878+0000","last_peered":"2024-09-21T06:30:32.308878+0000","last_clean":"2024-09-21T06:30:32.308878+0000","last_became_active":"2024-09-21T06:30:31.319691+0000","last_became_peered":"2024-09-21T06:30:31.319691+0000","last_unstale":"2024-09-21T06:30:32.308878+0000","last_undegraded":"2024-09-21T06:30:32.308878+0000","last_fullsized":"2024-09-21T06:30:32.308878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:52.696536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310052+0000","last_change":"2024-09-21T06:30:31.309907+0000","last_active":"2024-09-21T06:30:32.310052+0000","last_peered":"2024-09-21T06:30:32.310052+0000","last_clean":"2024-09-21T06:30:32.310052+0000","last_became_active":"2024-09-21T06:30:31.309605+0000","last_became_peered":"2024-09-21T06:30:31.309605+0000","last_unstale":"2024-09-21T06:30:32.310052+0000","last_undegraded":"2024-09-21T06:30:32.310052+0000","last_fullsized":"2024-09-21T06:30:32.310052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:52.696536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537274+0000","last_change":"2024-09-21T06:30:31.315104+0000","last_active":"2024-09-21T06:30:32.537274+0000","last_peered":"2024-09-21T06:30:32.537274+0000","last_clean":"2024-09-21T06:30:32.537274+0000","last_became_active":"2024-09-21T06:30:31.314752+0000","last_became_peered":"2024-09-21T06:30:31.314752+0000","last_unstale":"2024-09-21T06:30:32.537274+0000","last_undegraded":"2024-09-21T06:30:32.537274+0000","last_fullsized":"2024-09-21T06:30:32.537274+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.017895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310229+0000","last_change":"2024-09-21T06:30:31.326413+0000","last_active":"2024-09-21T06:30:32.310229+0000","last_peered":"2024-09-21T06:30:32.310229+0000","last_clean":"2024-09-21T06:30:32.310229+0000","last_became_active":"2024-09-21T06:30:31.324502+0000","last_became_peered":"2024-09-21T06:30:31.324502+0000","last_unstale":"2024-09-21T06:30:32.310229+0000","last_undegraded":"2024-09-21T06:30:32.310229+0000","last_fullsized":"2024-09-21T06:30:32.310229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:56.813975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310558+0000","last_change":"2024-09-21T06:30:31.318170+0000","last_active":"2024-09-21T06:30:32.310558+0000","last_peered":"2024-09-21T06:30:32.310558+0000","last_clean":"2024-09-21T06:30:32.310558+0000","last_became_active":"2024-09-21T06:30:31.317836+0000","last_became_peered":"2024-09-21T06:30:31.317836+0000","last_unstale":"2024-09-21T06:30:32.310558+0000","last_undegraded":"2024-09-21T06:30:32.310558+0000","last_fullsized":"2024-09-21T06:30:32.310558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.869230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538216+0000","last_change":"2024-09-21T06:30:31.319226+0000","last_active":"2024-09-21T06:30:32.538216+0000","last_peered":"2024-09-21T06:30:32.538216+0000","last_clean":"2024-09-21T06:30:32.538216+0000","last_became_active":"2024-09-21T06:30:31.318861+0000","last_became_peered":"2024-09-21T06:30:31.318861+0000","last_unstale":"2024-09-21T06:30:32.538216+0000","last_undegraded":"2024-09-21T06:30:32.538216+0000","last_fullsized":"2024-09-21T06:30:32.538216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:46.128616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537969+0000","last_change":"2024-09-21T06:30:31.323026+0000","last_active":"2024-09-21T06:30:32.537969+0000","last_peered":"2024-09-21T06:30:32.537969+0000","last_clean":"2024-09-21T06:30:32.537969+0000","last_became_active":"2024-09-21T06:30:31.322653+0000","last_became_peered":"2024-09-21T06:30:31.322653+0000","last_unstale":"2024-09-21T06:30:32.537969+0000","last_undegraded":"2024-09-21T06:30:32.537969+0000","last_fullsized":"2024-09-21T06:30:32.537969+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310201+0000","last_change":"2024-09-21T06:30:31.315492+0000","last_active":"2024-09-21T06:30:32.310201+0000","last_peered":"2024-09-21T06:30:32.310201+0000","last_clean":"2024-09-21T06:30:32.310201+0000","last_became_active":"2024-09-21T06:30:31.311787+0000","last_became_peered":"2024-09-21T06:30:31.311787+0000","last_unstale":"2024-09-21T06:30:32.310201+0000","last_undegraded":"2024-09-21T06:30:32.310201+0000","last_fullsized":"2024-09-21T06:30:32.310201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:10.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537697+0000","last_change":"2024-09-21T06:30:31.312757+0000","last_active":"2024-09-21T06:30:32.537697+0000","last_peered":"2024-09-21T06:30:32.537697+0000","last_clean":"2024-09-21T06:30:32.537697+0000","last_became_active":"2024-09-21T06:30:31.312447+0000","last_became_peered":"2024-09-21T06:30:31.312447+0000","last_unstale":"2024-09-21T06:30:32.537697+0000","last_undegraded":"2024-09-21T06:30:32.537697+0000","last_fullsized":"2024-09-21T06:30:32.537697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:52.551557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538256+0000","last_change":"2024-09-21T06:30:31.327135+0000","last_active":"2024-09-21T06:30:32.538256+0000","last_peered":"2024-09-21T06:30:32.538256+0000","last_clean":"2024-09-21T06:30:32.538256+0000","last_became_active":"2024-09-21T06:30:31.326786+0000","last_became_peered":"2024-09-21T06:30:31.326786+0000","last_unstale":"2024-09-21T06:30:32.538256+0000","last_undegraded":"2024-09-21T06:30:32.538256+0000","last_fullsized":"2024-09-21T06:30:32.538256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:28.565250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537625+0000","last_change":"2024-09-21T06:30:31.315065+0000","last_active":"2024-09-21T06:30:32.537625+0000","last_peered":"2024-09-21T06:30:32.537625+0000","last_clean":"2024-09-21T06:30:32.537625+0000","last_became_active":"2024-09-21T06:30:31.314716+0000","last_became_peered":"2024-09-21T06:30:31.314716+0000","last_unstale":"2024-09-21T06:30:32.537625+0000","last_undegraded":"2024-09-21T06:30:32.537625+0000","last_fullsized":"2024-09-21T06:30:32.537625+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:12.719955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309820+0000","last_change":"2024-09-21T06:30:31.326663+0000","last_active":"2024-09-21T06:30:32.309820+0000","last_peered":"2024-09-21T06:30:32.309820+0000","last_clean":"2024-09-21T06:30:32.309820+0000","last_became_active":"2024-09-21T06:30:31.325002+0000","last_became_peered":"2024-09-21T06:30:31.325002+0000","last_unstale":"2024-09-21T06:30:32.309820+0000","last_undegraded":"2024-09-21T06:30:32.309820+0000","last_fullsized":"2024-09-21T06:30:32.309820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:51.298694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538104+0000","last_change":"2024-09-21T06:30:31.319547+0000","last_active":"2024-09-21T06:30:32.538104+0000","last_peered":"2024-09-21T06:30:32.538104+0000","last_clean":"2024-09-21T06:30:32.538104+0000","last_became_active":"2024-09-21T06:30:31.319226+0000","last_became_peered":"2024-09-21T06:30:31.319226+0000","last_unstale":"2024-09-21T06:30:32.538104+0000","last_undegraded":"2024-09-21T06:30:32.538104+0000","last_fullsized":"2024-09-21T06:30:32.538104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:38.924482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537679+0000","last_change":"2024-09-21T06:30:31.325751+0000","last_active":"2024-09-21T06:30:32.537679+0000","last_peered":"2024-09-21T06:30:32.537679+0000","last_clean":"2024-09-21T06:30:32.537679+0000","last_became_active":"2024-09-21T06:30:31.325456+0000","last_became_peered":"2024-09-21T06:30:31.325456+0000","last_unstale":"2024-09-21T06:30:32.537679+0000","last_undegraded":"2024-09-21T06:30:32.537679+0000","last_fullsized":"2024-09-21T06:30:32.537679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.933118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538520+0000","last_change":"2024-09-21T06:30:31.320518+0000","last_active":"2024-09-21T06:30:32.538520+0000","last_peered":"2024-09-21T06:30:32.538520+0000","last_clean":"2024-09-21T06:30:32.538520+0000","last_became_active":"2024-09-21T06:30:31.320174+0000","last_became_peered":"2024-09-21T06:30:31.320174+0000","last_unstale":"2024-09-21T06:30:32.538520+0000","last_undegraded":"2024-09-21T06:30:32.538520+0000","last_fullsized":"2024-09-21T06:30:32.538520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:48.270905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309146+0000","last_change":"2024-09-21T06:30:31.327891+0000","last_active":"2024-09-21T06:30:32.309146+0000","last_peered":"2024-09-21T06:30:32.309146+0000","last_clean":"2024-09-21T06:30:32.309146+0000","last_became_active":"2024-09-21T06:30:31.327350+0000","last_became_peered":"2024-09-21T06:30:31.327350+0000","last_unstale":"2024-09-21T06:30:32.309146+0000","last_undegraded":"2024-09-21T06:30:32.309146+0000","last_fullsized":"2024-09-21T06:30:32.309146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:27.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537161+0000","last_change":"2024-09-21T06:30:31.319499+0000","last_active":"2024-09-21T06:30:32.537161+0000","last_peered":"2024-09-21T06:30:32.537161+0000","last_clean":"2024-09-21T06:30:32.537161+0000","last_became_active":"2024-09-21T06:30:31.319048+0000","last_became_peered":"2024-09-21T06:30:31.319048+0000","last_unstale":"2024-09-21T06:30:32.537161+0000","last_undegraded":"2024-09-21T06:30:32.537161+0000","last_fullsized":"2024-09-21T06:30:32.537161+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:21.159737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538361+0000","last_change":"2024-09-21T06:30:31.311536+0000","last_active":"2024-09-21T06:30:32.538361+0000","last_peered":"2024-09-21T06:30:32.538361+0000","last_clean":"2024-09-21T06:30:32.538361+0000","last_became_active":"2024-09-21T06:30:31.311207+0000","last_became_peered":"2024-09-21T06:30:31.311207+0000","last_unstale":"2024-09-21T06:30:32.538361+0000","last_undegraded":"2024-09-21T06:30:32.538361+0000","last_fullsized":"2024-09-21T06:30:32.538361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:12.529593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.536605+0000","last_change":"2024-09-21T06:30:31.323803+0000","last_active":"2024-09-21T06:30:32.536605+0000","last_peered":"2024-09-21T06:30:32.536605+0000","last_clean":"2024-09-21T06:30:32.536605+0000","last_became_active":"2024-09-21T06:30:31.322361+0000","last_became_peered":"2024-09-21T06:30:31.322361+0000","last_unstale":"2024-09-21T06:30:32.536605+0000","last_undegraded":"2024-09-21T06:30:32.536605+0000","last_fullsized":"2024-09-21T06:30:32.536605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.017895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537129+0000","last_change":"2024-09-21T06:30:31.315180+0000","last_active":"2024-09-21T06:30:32.537129+0000","last_peered":"2024-09-21T06:30:32.537129+0000","last_clean":"2024-09-21T06:30:32.537129+0000","last_became_active":"2024-09-21T06:30:31.314843+0000","last_became_peered":"2024-09-21T06:30:31.314843+0000","last_unstale":"2024-09-21T06:30:32.537129+0000","last_undegraded":"2024-09-21T06:30:32.537129+0000","last_fullsized":"2024-09-21T06:30:32.537129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:40.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317602+0000","last_change":"2024-09-21T06:30:30.317061+0000","last_active":"2024-09-21T06:30:32.317602+0000","last_peered":"2024-09-21T06:30:32.317602+0000","last_clean":"2024-09-21T06:30:32.317602+0000","last_became_active":"2024-09-21T06:30:30.316746+0000","last_became_peered":"2024-09-21T06:30:30.316746+0000","last_unstale":"2024-09-21T06:30:32.317602+0000","last_undegraded":"2024-09-21T06:30:32.317602+0000","last_fullsized":"2024-09-21T06:30:32.317602+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:42.121089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537018+0000","last_change":"2024-09-21T06:30:31.314676+0000","last_active":"2024-09-21T06:30:32.537018+0000","last_peered":"2024-09-21T06:30:32.537018+0000","last_clean":"2024-09-21T06:30:32.537018+0000","last_became_active":"2024-09-21T06:30:31.312025+0000","last_became_peered":"2024-09-21T06:30:31.312025+0000","last_unstale":"2024-09-21T06:30:32.537018+0000","last_undegraded":"2024-09-21T06:30:32.537018+0000","last_fullsized":"2024-09-21T06:30:32.537018+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:45.069032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538403+0000","last_change":"2024-09-21T06:30:30.325686+0000","last_active":"2024-09-21T06:30:32.538403+0000","last_peered":"2024-09-21T06:30:32.538403+0000","last_clean":"2024-09-21T06:30:32.538403+0000","last_became_active":"2024-09-21T06:30:30.325369+0000","last_became_peered":"2024-09-21T06:30:30.325369+0000","last_unstale":"2024-09-21T06:30:32.538403+0000","last_undegraded":"2024-09-21T06:30:32.538403+0000","last_fullsized":"2024-09-21T06:30:32.538403+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:16.102746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537183+0000","last_change":"2024-09-21T06:30:31.319213+0000","last_active":"2024-09-21T06:30:32.537183+0000","last_peered":"2024-09-21T06:30:32.537183+0000","last_clean":"2024-09-21T06:30:32.537183+0000","last_became_active":"2024-09-21T06:30:31.318555+0000","last_became_peered":"2024-09-21T06:30:31.318555+0000","last_unstale":"2024-09-21T06:30:32.537183+0000","last_undegraded":"2024-09-21T06:30:32.537183+0000","last_fullsized":"2024-09-21T06:30:32.537183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:33.741956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310526+0000","last_change":"2024-09-21T06:30:30.309511+0000","last_active":"2024-09-21T06:30:32.310526+0000","last_peered":"2024-09-21T06:30:32.310526+0000","last_clean":"2024-09-21T06:30:32.310526+0000","last_became_active":"2024-09-21T06:30:30.307785+0000","last_became_peered":"2024-09-21T06:30:30.307785+0000","last_unstale":"2024-09-21T06:30:32.310526+0000","last_undegraded":"2024-09-21T06:30:32.310526+0000","last_fullsized":"2024-09-21T06:30:32.310526+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:37.385089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309631+0000","last_change":"2024-09-21T06:30:31.311405+0000","last_active":"2024-09-21T06:30:32.309631+0000","last_peered":"2024-09-21T06:30:32.309631+0000","last_clean":"2024-09-21T06:30:32.309631+0000","last_became_active":"2024-09-21T06:30:31.311013+0000","last_became_peered":"2024-09-21T06:30:31.311013+0000","last_unstale":"2024-09-21T06:30:32.309631+0000","last_undegraded":"2024-09-21T06:30:32.309631+0000","last_fullsized":"2024-09-21T06:30:32.309631+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:00.184082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538278+0000","last_change":"2024-09-21T06:30:30.325411+0000","last_active":"2024-09-21T06:30:32.538278+0000","last_peered":"2024-09-21T06:30:32.538278+0000","last_clean":"2024-09-21T06:30:32.538278+0000","last_became_active":"2024-09-21T06:30:30.325069+0000","last_became_peered":"2024-09-21T06:30:30.325069+0000","last_unstale":"2024-09-21T06:30:32.538278+0000","last_undegraded":"2024-09-21T06:30:32.538278+0000","last_fullsized":"2024-09-21T06:30:32.538278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:27.217091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317803+0000","last_change":"2024-09-21T06:30:31.332172+0000","last_active":"2024-09-21T06:30:32.317803+0000","last_peered":"2024-09-21T06:30:32.317803+0000","last_clean":"2024-09-21T06:30:32.317803+0000","last_became_active":"2024-09-21T06:30:31.331859+0000","last_became_peered":"2024-09-21T06:30:31.331859+0000","last_unstale":"2024-09-21T06:30:32.317803+0000","last_undegraded":"2024-09-21T06:30:32.317803+0000","last_fullsized":"2024-09-21T06:30:32.317803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:58.149878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538675+0000","last_change":"2024-09-21T06:30:30.314082+0000","last_active":"2024-09-21T06:30:32.538675+0000","last_peered":"2024-09-21T06:30:32.538675+0000","last_clean":"2024-09-21T06:30:32.538675+0000","last_became_active":"2024-09-21T06:30:30.311066+0000","last_became_peered":"2024-09-21T06:30:30.311066+0000","last_unstale":"2024-09-21T06:30:32.538675+0000","last_undegraded":"2024-09-21T06:30:32.538675+0000","last_fullsized":"2024-09-21T06:30:32.538675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:47.579057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537308+0000","last_change":"2024-09-21T06:30:31.325429+0000","last_active":"2024-09-21T06:30:32.537308+0000","last_peered":"2024-09-21T06:30:32.537308+0000","last_clean":"2024-09-21T06:30:32.537308+0000","last_became_active":"2024-09-21T06:30:31.325083+0000","last_became_peered":"2024-09-21T06:30:31.325083+0000","last_unstale":"2024-09-21T06:30:32.537308+0000","last_undegraded":"2024-09-21T06:30:32.537308+0000","last_fullsized":"2024-09-21T06:30:32.537308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.859081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537324+0000","last_change":"2024-09-21T06:30:30.312123+0000","last_active":"2024-09-21T06:30:32.537324+0000","last_peered":"2024-09-21T06:30:32.537324+0000","last_clean":"2024-09-21T06:30:32.537324+0000","last_became_active":"2024-09-21T06:30:30.311786+0000","last_became_peered":"2024-09-21T06:30:30.311786+0000","last_unstale":"2024-09-21T06:30:32.537324+0000","last_undegraded":"2024-09-21T06:30:32.537324+0000","last_fullsized":"2024-09-21T06:30:32.537324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:40.212309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310919+0000","last_change":"2024-09-21T06:30:31.313644+0000","last_active":"2024-09-21T06:30:32.310919+0000","last_peered":"2024-09-21T06:30:32.310919+0000","last_clean":"2024-09-21T06:30:32.310919+0000","last_became_active":"2024-09-21T06:30:31.311070+0000","last_became_peered":"2024-09-21T06:30:31.311070+0000","last_unstale":"2024-09-21T06:30:32.310919+0000","last_undegraded":"2024-09-21T06:30:32.310919+0000","last_fullsized":"2024-09-21T06:30:32.310919+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:16.594637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.536890+0000","last_change":"2024-09-21T06:30:30.317516+0000","last_active":"2024-09-21T06:30:32.536890+0000","last_peered":"2024-09-21T06:30:32.536890+0000","last_clean":"2024-09-21T06:30:32.536890+0000","last_became_active":"2024-09-21T06:30:30.317198+0000","last_became_peered":"2024-09-21T06:30:30.317198+0000","last_unstale":"2024-09-21T06:30:32.536890+0000","last_undegraded":"2024-09-21T06:30:32.536890+0000","last_fullsized":"2024-09-21T06:30:32.536890+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:02.118682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310488+0000","last_change":"2024-09-21T06:30:31.316017+0000","last_active":"2024-09-21T06:30:32.310488+0000","last_peered":"2024-09-21T06:30:32.310488+0000","last_clean":"2024-09-21T06:30:32.310488+0000","last_became_active":"2024-09-21T06:30:31.315664+0000","last_became_peered":"2024-09-21T06:30:31.315664+0000","last_unstale":"2024-09-21T06:30:32.310488+0000","last_undegraded":"2024-09-21T06:30:32.310488+0000","last_fullsized":"2024-09-21T06:30:32.310488+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:50.623685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537764+0000","last_change":"2024-09-21T06:30:30.324244+0000","last_active":"2024-09-21T06:30:32.537764+0000","last_peered":"2024-09-21T06:30:32.537764+0000","last_clean":"2024-09-21T06:30:32.537764+0000","last_became_active":"2024-09-21T06:30:30.323920+0000","last_became_peered":"2024-09-21T06:30:30.323920+0000","last_unstale":"2024-09-21T06:30:32.537764+0000","last_undegraded":"2024-09-21T06:30:32.537764+0000","last_fullsized":"2024-09-21T06:30:32.537764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:54.388458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317849+0000","last_change":"2024-09-21T06:30:31.321983+0000","last_active":"2024-09-21T06:30:32.317849+0000","last_peered":"2024-09-21T06:30:32.317849+0000","last_clean":"2024-09-21T06:30:32.317849+0000","last_became_active":"2024-09-21T06:30:31.321639+0000","last_became_peered":"2024-09-21T06:30:31.321639+0000","last_unstale":"2024-09-21T06:30:32.317849+0000","last_undegraded":"2024-09-21T06:30:32.317849+0000","last_fullsized":"2024-09-21T06:30:32.317849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.404882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310533+0000","last_change":"2024-09-21T06:30:30.307552+0000","last_active":"2024-09-21T06:30:32.310533+0000","last_peered":"2024-09-21T06:30:32.310533+0000","last_clean":"2024-09-21T06:30:32.310533+0000","last_became_active":"2024-09-21T06:30:30.307185+0000","last_became_peered":"2024-09-21T06:30:30.307185+0000","last_unstale":"2024-09-21T06:30:32.310533+0000","last_undegraded":"2024-09-21T06:30:32.310533+0000","last_fullsized":"2024-09-21T06:30:32.310533+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:11.158964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310118+0000","last_change":"2024-09-21T06:30:31.318250+0000","last_active":"2024-09-21T06:30:32.310118+0000","last_peered":"2024-09-21T06:30:32.310118+0000","last_clean":"2024-09-21T06:30:32.310118+0000","last_became_active":"2024-09-21T06:30:31.317877+0000","last_became_peered":"2024-09-21T06:30:31.317877+0000","last_unstale":"2024-09-21T06:30:32.310118+0000","last_undegraded":"2024-09-21T06:30:32.310118+0000","last_fullsized":"2024-09-21T06:30:32.310118+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:27.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310340+0000","last_change":"2024-09-21T06:30:30.317553+0000","last_active":"2024-09-21T06:30:32.310340+0000","last_peered":"2024-09-21T06:30:32.310340+0000","last_clean":"2024-09-21T06:30:32.310340+0000","last_became_active":"2024-09-21T06:30:30.317218+0000","last_became_peered":"2024-09-21T06:30:30.317218+0000","last_unstale":"2024-09-21T06:30:32.310340+0000","last_undegraded":"2024-09-21T06:30:32.310340+0000","last_fullsized":"2024-09-21T06:30:32.310340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:27.217091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309224+0000","last_change":"2024-09-21T06:30:31.329848+0000","last_active":"2024-09-21T06:30:32.309224+0000","last_peered":"2024-09-21T06:30:32.309224+0000","last_clean":"2024-09-21T06:30:32.309224+0000","last_became_active":"2024-09-21T06:30:31.329488+0000","last_became_peered":"2024-09-21T06:30:31.329488+0000","last_unstale":"2024-09-21T06:30:32.309224+0000","last_undegraded":"2024-09-21T06:30:32.309224+0000","last_fullsized":"2024-09-21T06:30:32.309224+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:10.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537330+0000","last_change":"2024-09-21T06:30:30.315505+0000","last_active":"2024-09-21T06:30:32.537330+0000","last_peered":"2024-09-21T06:30:32.537330+0000","last_clean":"2024-09-21T06:30:32.537330+0000","last_became_active":"2024-09-21T06:30:30.315212+0000","last_became_peered":"2024-09-21T06:30:30.315212+0000","last_unstale":"2024-09-21T06:30:32.537330+0000","last_undegraded":"2024-09-21T06:30:32.537330+0000","last_fullsized":"2024-09-21T06:30:32.537330+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:14.517646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537961+0000","last_change":"2024-09-21T06:30:31.317301+0000","last_active":"2024-09-21T06:30:32.537961+0000","last_peered":"2024-09-21T06:30:32.537961+0000","last_clean":"2024-09-21T06:30:32.537961+0000","last_became_active":"2024-09-21T06:30:31.316966+0000","last_became_peered":"2024-09-21T06:30:31.316966+0000","last_unstale":"2024-09-21T06:30:32.537961+0000","last_undegraded":"2024-09-21T06:30:32.537961+0000","last_fullsized":"2024-09-21T06:30:32.537961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.933118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":21,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310710+0000","last_change":"2024-09-21T06:30:30.326062+0000","last_active":"2024-09-21T06:30:32.310710+0000","last_peered":"2024-09-21T06:30:32.310710+0000","last_clean":"2024-09-21T06:30:32.310710+0000","last_became_active":"2024-09-21T06:30:30.325023+0000","last_became_peered":"2024-09-21T06:30:30.325023+0000","last_unstale":"2024-09-21T06:30:32.310710+0000","last_undegraded":"2024-09-21T06:30:32.310710+0000","last_fullsized":"2024-09-21T06:30:32.310710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:27.278349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537417+0000","last_change":"2024-09-21T06:30:31.330173+0000","last_active":"2024-09-21T06:30:32.537417+0000","last_peered":"2024-09-21T06:30:32.537417+0000","last_clean":"2024-09-21T06:30:32.537417+0000","last_became_active":"2024-09-21T06:30:31.329361+0000","last_became_peered":"2024-09-21T06:30:31.329361+0000","last_unstale":"2024-09-21T06:30:32.537417+0000","last_undegraded":"2024-09-21T06:30:32.537417+0000","last_fullsized":"2024-09-21T06:30:32.537417+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:14.993582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538460+0000","last_change":"2024-09-21T06:30:30.321481+0000","last_active":"2024-09-21T06:30:32.538460+0000","last_peered":"2024-09-21T06:30:32.538460+0000","last_clean":"2024-09-21T06:30:32.538460+0000","last_became_active":"2024-09-21T06:30:30.321104+0000","last_became_peered":"2024-09-21T06:30:30.321104+0000","last_unstale":"2024-09-21T06:30:32.538460+0000","last_undegraded":"2024-09-21T06:30:32.538460+0000","last_fullsized":"2024-09-21T06:30:32.538460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:24.050874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309559+0000","last_change":"2024-09-21T06:30:31.329600+0000","last_active":"2024-09-21T06:30:32.309559+0000","last_peered":"2024-09-21T06:30:32.309559+0000","last_clean":"2024-09-21T06:30:32.309559+0000","last_became_active":"2024-09-21T06:30:31.328920+0000","last_became_peered":"2024-09-21T06:30:31.328920+0000","last_unstale":"2024-09-21T06:30:32.309559+0000","last_undegraded":"2024-09-21T06:30:32.309559+0000","last_fullsized":"2024-09-21T06:30:32.309559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.869230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309781+0000","last_change":"2024-09-21T06:30:30.308068+0000","last_active":"2024-09-21T06:30:32.309781+0000","last_peered":"2024-09-21T06:30:32.309781+0000","last_clean":"2024-09-21T06:30:32.309781+0000","last_became_active":"2024-09-21T06:30:30.307766+0000","last_became_peered":"2024-09-21T06:30:30.307766+0000","last_unstale":"2024-09-21T06:30:32.309781+0000","last_undegraded":"2024-09-21T06:30:32.309781+0000","last_fullsized":"2024-09-21T06:30:32.309781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:29.037949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538640+0000","last_change":"2024-09-21T06:30:31.332842+0000","last_active":"2024-09-21T06:30:32.538640+0000","last_peered":"2024-09-21T06:30:32.538640+0000","last_clean":"2024-09-21T06:30:32.538640+0000","last_became_active":"2024-09-21T06:30:31.329569+0000","last_became_peered":"2024-09-21T06:30:31.329569+0000","last_unstale":"2024-09-21T06:30:32.538640+0000","last_undegraded":"2024-09-21T06:30:32.538640+0000","last_fullsized":"2024-09-21T06:30:32.538640+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:08.739252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537346+0000","last_change":"2024-09-21T06:30:30.319925+0000","last_active":"2024-09-21T06:30:32.537346+0000","last_peered":"2024-09-21T06:30:32.537346+0000","last_clean":"2024-09-21T06:30:32.537346+0000","last_became_active":"2024-09-21T06:30:30.319578+0000","last_became_peered":"2024-09-21T06:30:30.319578+0000","last_unstale":"2024-09-21T06:30:32.537346+0000","last_undegraded":"2024-09-21T06:30:32.537346+0000","last_fullsized":"2024-09-21T06:30:32.537346+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:24.050874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538661+0000","last_change":"2024-09-21T06:30:31.315221+0000","last_active":"2024-09-21T06:30:32.538661+0000","last_peered":"2024-09-21T06:30:32.538661+0000","last_clean":"2024-09-21T06:30:32.538661+0000","last_became_active":"2024-09-21T06:30:31.314845+0000","last_became_peered":"2024-09-21T06:30:31.314845+0000","last_unstale":"2024-09-21T06:30:32.538661+0000","last_undegraded":"2024-09-21T06:30:32.538661+0000","last_fullsized":"2024-09-21T06:30:32.538661+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:21.914277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537429+0000","last_change":"2024-09-21T06:30:30.327896+0000","last_active":"2024-09-21T06:30:32.537429+0000","last_peered":"2024-09-21T06:30:32.537429+0000","last_clean":"2024-09-21T06:30:32.537429+0000","last_became_active":"2024-09-21T06:30:30.327275+0000","last_became_peered":"2024-09-21T06:30:30.327275+0000","last_unstale":"2024-09-21T06:30:32.537429+0000","last_undegraded":"2024-09-21T06:30:32.537429+0000","last_fullsized":"2024-09-21T06:30:32.537429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.451060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309058+0000","last_change":"2024-09-21T06:30:31.318205+0000","last_active":"2024-09-21T06:30:32.309058+0000","last_peered":"2024-09-21T06:30:32.309058+0000","last_clean":"2024-09-21T06:30:32.309058+0000","last_became_active":"2024-09-21T06:30:31.317874+0000","last_became_peered":"2024-09-21T06:30:31.317874+0000","last_unstale":"2024-09-21T06:30:32.309058+0000","last_undegraded":"2024-09-21T06:30:32.309058+0000","last_fullsized":"2024-09-21T06:30:32.309058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.110817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.539079+0000","last_change":"2024-09-21T06:30:30.327321+0000","last_active":"2024-09-21T06:30:32.539079+0000","last_peered":"2024-09-21T06:30:32.539079+0000","last_clean":"2024-09-21T06:30:32.539079+0000","last_became_active":"2024-09-21T06:30:30.326926+0000","last_became_peered":"2024-09-21T06:30:30.326926+0000","last_unstale":"2024-09-21T06:30:32.539079+0000","last_undegraded":"2024-09-21T06:30:32.539079+0000","last_fullsized":"2024-09-21T06:30:32.539079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.308829+0000","last_change":"2024-09-21T06:30:31.322078+0000","last_active":"2024-09-21T06:30:32.308829+0000","last_peered":"2024-09-21T06:30:32.308829+0000","last_clean":"2024-09-21T06:30:32.308829+0000","last_became_active":"2024-09-21T06:30:31.321487+0000","last_became_peered":"2024-09-21T06:30:31.321487+0000","last_unstale":"2024-09-21T06:30:32.308829+0000","last_undegraded":"2024-09-21T06:30:32.308829+0000","last_fullsized":"2024-09-21T06:30:32.308829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:51.274160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538805+0000","last_change":"2024-09-21T06:30:30.320839+0000","last_active":"2024-09-21T06:30:32.538805+0000","last_peered":"2024-09-21T06:30:32.538805+0000","last_clean":"2024-09-21T06:30:32.538805+0000","last_became_active":"2024-09-21T06:30:30.320344+0000","last_became_peered":"2024-09-21T06:30:30.320344+0000","last_unstale":"2024-09-21T06:30:32.538805+0000","last_undegraded":"2024-09-21T06:30:32.538805+0000","last_fullsized":"2024-09-21T06:30:32.538805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:56.406494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537669+0000","last_change":"2024-09-21T06:30:31.324775+0000","last_active":"2024-09-21T06:30:32.537669+0000","last_peered":"2024-09-21T06:30:32.537669+0000","last_clean":"2024-09-21T06:30:32.537669+0000","last_became_active":"2024-09-21T06:30:31.322070+0000","last_became_peered":"2024-09-21T06:30:31.322070+0000","last_unstale":"2024-09-21T06:30:32.537669+0000","last_undegraded":"2024-09-21T06:30:32.537669+0000","last_fullsized":"2024-09-21T06:30:32.537669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:22.628559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310226+0000","last_change":"2024-09-21T06:30:30.313933+0000","last_active":"2024-09-21T06:30:32.310226+0000","last_peered":"2024-09-21T06:30:32.310226+0000","last_clean":"2024-09-21T06:30:32.310226+0000","last_became_active":"2024-09-21T06:30:30.313460+0000","last_became_peered":"2024-09-21T06:30:30.313460+0000","last_unstale":"2024-09-21T06:30:32.310226+0000","last_undegraded":"2024-09-21T06:30:32.310226+0000","last_fullsized":"2024-09-21T06:30:32.310226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.629719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537999+0000","last_change":"2024-09-21T06:30:31.333687+0000","last_active":"2024-09-21T06:30:32.537999+0000","last_peered":"2024-09-21T06:30:32.537999+0000","last_clean":"2024-09-21T06:30:32.537999+0000","last_became_active":"2024-09-21T06:30:31.326638+0000","last_became_peered":"2024-09-21T06:30:31.326638+0000","last_unstale":"2024-09-21T06:30:32.537999+0000","last_undegraded":"2024-09-21T06:30:32.537999+0000","last_fullsized":"2024-09-21T06:30:32.537999+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.923134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538350+0000","last_change":"2024-09-21T06:30:30.309923+0000","last_active":"2024-09-21T06:30:32.538350+0000","last_peered":"2024-09-21T06:30:32.538350+0000","last_clean":"2024-09-21T06:30:32.538350+0000","last_became_active":"2024-09-21T06:30:30.309628+0000","last_became_peered":"2024-09-21T06:30:30.309628+0000","last_unstale":"2024-09-21T06:30:32.538350+0000","last_undegraded":"2024-09-21T06:30:32.538350+0000","last_fullsized":"2024-09-21T06:30:32.538350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:11.158964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538187+0000","last_change":"2024-09-21T06:30:31.322754+0000","last_active":"2024-09-21T06:30:32.538187+0000","last_peered":"2024-09-21T06:30:32.538187+0000","last_clean":"2024-09-21T06:30:32.538187+0000","last_became_active":"2024-09-21T06:30:31.322414+0000","last_became_peered":"2024-09-21T06:30:31.322414+0000","last_unstale":"2024-09-21T06:30:32.538187+0000","last_undegraded":"2024-09-21T06:30:32.538187+0000","last_fullsized":"2024-09-21T06:30:32.538187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:08.739252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309005+0000","last_change":"2024-09-21T06:30:30.318255+0000","last_active":"2024-09-21T06:30:32.309005+0000","last_peered":"2024-09-21T06:30:32.309005+0000","last_clean":"2024-09-21T06:30:32.309005+0000","last_became_active":"2024-09-21T06:30:30.317860+0000","last_became_peered":"2024-09-21T06:30:30.317860+0000","last_unstale":"2024-09-21T06:30:32.309005+0000","last_undegraded":"2024-09-21T06:30:32.309005+0000","last_fullsized":"2024-09-21T06:30:32.309005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:07.459885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309981+0000","last_change":"2024-09-21T06:30:31.328769+0000","last_active":"2024-09-21T06:30:32.309981+0000","last_peered":"2024-09-21T06:30:32.309981+0000","last_clean":"2024-09-21T06:30:32.309981+0000","last_became_active":"2024-09-21T06:30:31.328406+0000","last_became_peered":"2024-09-21T06:30:31.328406+0000","last_unstale":"2024-09-21T06:30:32.309981+0000","last_undegraded":"2024-09-21T06:30:32.309981+0000","last_fullsized":"2024-09-21T06:30:32.309981+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.404882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537868+0000","last_change":"2024-09-21T06:30:30.322668+0000","last_active":"2024-09-21T06:30:32.537868+0000","last_peered":"2024-09-21T06:30:32.537868+0000","last_clean":"2024-09-21T06:30:32.537868+0000","last_became_active":"2024-09-21T06:30:30.322292+0000","last_became_peered":"2024-09-21T06:30:30.322292+0000","last_unstale":"2024-09-21T06:30:32.537868+0000","last_undegraded":"2024-09-21T06:30:32.537868+0000","last_fullsized":"2024-09-21T06:30:32.537868+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.815465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310148+0000","last_change":"2024-09-21T06:30:31.334491+0000","last_active":"2024-09-21T06:30:32.310148+0000","last_peered":"2024-09-21T06:30:32.310148+0000","last_clean":"2024-09-21T06:30:32.310148+0000","last_became_active":"2024-09-21T06:30:31.334148+0000","last_became_peered":"2024-09-21T06:30:31.334148+0000","last_unstale":"2024-09-21T06:30:32.310148+0000","last_undegraded":"2024-09-21T06:30:32.310148+0000","last_fullsized":"2024-09-21T06:30:32.310148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:24.600225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310226+0000","last_change":"2024-09-21T06:30:30.325380+0000","last_active":"2024-09-21T06:30:32.310226+0000","last_peered":"2024-09-21T06:30:32.310226+0000","last_clean":"2024-09-21T06:30:32.310226+0000","last_became_active":"2024-09-21T06:30:30.324901+0000","last_became_peered":"2024-09-21T06:30:30.324901+0000","last_unstale":"2024-09-21T06:30:32.310226+0000","last_undegraded":"2024-09-21T06:30:32.310226+0000","last_fullsized":"2024-09-21T06:30:32.310226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:14.551912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538993+0000","last_change":"2024-09-21T06:30:31.321217+0000","last_active":"2024-09-21T06:30:32.538993+0000","last_peered":"2024-09-21T06:30:32.538993+0000","last_clean":"2024-09-21T06:30:32.538993+0000","last_became_active":"2024-09-21T06:30:31.320897+0000","last_became_peered":"2024-09-21T06:30:31.320897+0000","last_unstale":"2024-09-21T06:30:32.538993+0000","last_undegraded":"2024-09-21T06:30:32.538993+0000","last_fullsized":"2024-09-21T06:30:32.538993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:48.270905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309823+0000","last_change":"2024-09-21T06:30:30.314670+0000","last_active":"2024-09-21T06:30:32.309823+0000","last_peered":"2024-09-21T06:30:32.309823+0000","last_clean":"2024-09-21T06:30:32.309823+0000","last_became_active":"2024-09-21T06:30:30.314379+0000","last_became_peered":"2024-09-21T06:30:30.314379+0000","last_unstale":"2024-09-21T06:30:32.309823+0000","last_undegraded":"2024-09-21T06:30:32.309823+0000","last_fullsized":"2024-09-21T06:30:32.309823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:18.067883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309855+0000","last_change":"2024-09-21T06:30:31.313697+0000","last_active":"2024-09-21T06:30:32.309855+0000","last_peered":"2024-09-21T06:30:32.309855+0000","last_clean":"2024-09-21T06:30:32.309855+0000","last_became_active":"2024-09-21T06:30:31.313368+0000","last_became_peered":"2024-09-21T06:30:31.313368+0000","last_unstale":"2024-09-21T06:30:32.309855+0000","last_undegraded":"2024-09-21T06:30:32.309855+0000","last_fullsized":"2024-09-21T06:30:32.309855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:58.149878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537009+0000","last_change":"2024-09-21T06:30:30.317004+0000","last_active":"2024-09-21T06:30:32.537009+0000","last_peered":"2024-09-21T06:30:32.537009+0000","last_clean":"2024-09-21T06:30:32.537009+0000","last_became_active":"2024-09-21T06:30:30.316663+0000","last_became_peered":"2024-09-21T06:30:30.316663+0000","last_unstale":"2024-09-21T06:30:32.537009+0000","last_undegraded":"2024-09-21T06:30:32.537009+0000","last_fullsized":"2024-09-21T06:30:32.537009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:51.404585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309541+0000","last_change":"2024-09-21T06:30:31.314282+0000","last_active":"2024-09-21T06:30:32.309541+0000","last_peered":"2024-09-21T06:30:32.309541+0000","last_clean":"2024-09-21T06:30:32.309541+0000","last_became_active":"2024-09-21T06:30:31.313928+0000","last_became_peered":"2024-09-21T06:30:31.313928+0000","last_unstale":"2024-09-21T06:30:32.309541+0000","last_undegraded":"2024-09-21T06:30:32.309541+0000","last_fullsized":"2024-09-21T06:30:32.309541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:01.706501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537388+0000","last_change":"2024-09-21T06:30:30.317374+0000","last_active":"2024-09-21T06:30:32.537388+0000","last_peered":"2024-09-21T06:30:32.537388+0000","last_clean":"2024-09-21T06:30:32.537388+0000","last_became_active":"2024-09-21T06:30:30.316456+0000","last_became_peered":"2024-09-21T06:30:30.316456+0000","last_unstale":"2024-09-21T06:30:32.537388+0000","last_undegraded":"2024-09-21T06:30:32.537388+0000","last_fullsized":"2024-09-21T06:30:32.537388+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:03.059109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538711+0000","last_change":"2024-09-21T06:30:31.317104+0000","last_active":"2024-09-21T06:30:32.538711+0000","last_peered":"2024-09-21T06:30:32.538711+0000","last_clean":"2024-09-21T06:30:32.538711+0000","last_became_active":"2024-09-21T06:30:31.316760+0000","last_became_peered":"2024-09-21T06:30:31.316760+0000","last_unstale":"2024-09-21T06:30:32.538711+0000","last_undegraded":"2024-09-21T06:30:32.538711+0000","last_fullsized":"2024-09-21T06:30:32.538711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:15.239306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537771+0000","last_change":"2024-09-21T06:30:30.313471+0000","last_active":"2024-09-21T06:30:32.537771+0000","last_peered":"2024-09-21T06:30:32.537771+0000","last_clean":"2024-09-21T06:30:32.537771+0000","last_became_active":"2024-09-21T06:30:30.313139+0000","last_became_peered":"2024-09-21T06:30:30.313139+0000","last_unstale":"2024-09-21T06:30:32.537771+0000","last_undegraded":"2024-09-21T06:30:32.537771+0000","last_fullsized":"2024-09-21T06:30:32.537771+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:44.906990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310634+0000","last_change":"2024-09-21T06:30:31.319796+0000","last_active":"2024-09-21T06:30:32.310634+0000","last_peered":"2024-09-21T06:30:32.310634+0000","last_clean":"2024-09-21T06:30:32.310634+0000","last_became_active":"2024-09-21T06:30:31.318515+0000","last_became_peered":"2024-09-21T06:30:31.318515+0000","last_unstale":"2024-09-21T06:30:32.310634+0000","last_undegraded":"2024-09-21T06:30:32.310634+0000","last_fullsized":"2024-09-21T06:30:32.310634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:45.378128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310100+0000","last_change":"2024-09-21T06:30:30.317597+0000","last_active":"2024-09-21T06:30:32.310100+0000","last_peered":"2024-09-21T06:30:32.310100+0000","last_clean":"2024-09-21T06:30:32.310100+0000","last_became_active":"2024-09-21T06:30:30.316722+0000","last_became_peered":"2024-09-21T06:30:30.316722+0000","last_unstale":"2024-09-21T06:30:32.310100+0000","last_undegraded":"2024-09-21T06:30:32.310100+0000","last_fullsized":"2024-09-21T06:30:32.310100+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:47.786268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538161+0000","last_change":"2024-09-21T06:30:31.312996+0000","last_active":"2024-09-21T06:30:32.538161+0000","last_peered":"2024-09-21T06:30:32.538161+0000","last_clean":"2024-09-21T06:30:32.538161+0000","last_became_active":"2024-09-21T06:30:31.312693+0000","last_became_peered":"2024-09-21T06:30:31.312693+0000","last_unstale":"2024-09-21T06:30:32.538161+0000","last_undegraded":"2024-09-21T06:30:32.538161+0000","last_fullsized":"2024-09-21T06:30:32.538161+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.142991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538102+0000","last_change":"2024-09-21T06:30:30.322601+0000","last_active":"2024-09-21T06:30:32.538102+0000","last_peered":"2024-09-21T06:30:32.538102+0000","last_clean":"2024-09-21T06:30:32.538102+0000","last_became_active":"2024-09-21T06:30:30.322261+0000","last_became_peered":"2024-09-21T06:30:30.322261+0000","last_unstale":"2024-09-21T06:30:32.538102+0000","last_undegraded":"2024-09-21T06:30:32.538102+0000","last_fullsized":"2024-09-21T06:30:32.538102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:45.567092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.311002+0000","last_change":"2024-09-21T06:30:31.313439+0000","last_active":"2024-09-21T06:30:32.311002+0000","last_peered":"2024-09-21T06:30:32.311002+0000","last_clean":"2024-09-21T06:30:32.311002+0000","last_became_active":"2024-09-21T06:30:31.312889+0000","last_became_peered":"2024-09-21T06:30:31.312889+0000","last_unstale":"2024-09-21T06:30:32.311002+0000","last_undegraded":"2024-09-21T06:30:32.311002+0000","last_fullsized":"2024-09-21T06:30:32.311002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:24.498593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.537491+0000","last_change":"2024-09-21T06:30:30.313209+0000","last_active":"2024-09-21T06:30:32.537491+0000","last_peered":"2024-09-21T06:30:32.537491+0000","last_clean":"2024-09-21T06:30:32.537491+0000","last_became_active":"2024-09-21T06:30:30.312858+0000","last_became_peered":"2024-09-21T06:30:30.312858+0000","last_unstale":"2024-09-21T06:30:32.537491+0000","last_undegraded":"2024-09-21T06:30:32.537491+0000","last_fullsized":"2024-09-21T06:30:32.537491+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.538944+0000","last_change":"2024-09-21T06:30:31.332947+0000","last_active":"2024-09-21T06:30:32.538944+0000","last_peered":"2024-09-21T06:30:32.538944+0000","last_clean":"2024-09-21T06:30:32.538944+0000","last_became_active":"2024-09-21T06:30:31.328041+0000","last_became_peered":"2024-09-21T06:30:31.328041+0000","last_unstale":"2024-09-21T06:30:32.538944+0000","last_undegraded":"2024-09-21T06:30:32.538944+0000","last_fullsized":"2024-09-21T06:30:32.538944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:42.542980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.310076+0000","last_change":"2024-09-21T06:30:30.317313+0000","last_active":"2024-09-21T06:30:32.310076+0000","last_peered":"2024-09-21T06:30:32.310076+0000","last_clean":"2024-09-21T06:30:32.310076+0000","last_became_active":"2024-09-21T06:30:30.316807+0000","last_became_peered":"2024-09-21T06:30:30.316807+0000","last_unstale":"2024-09-21T06:30:32.310076+0000","last_undegraded":"2024-09-21T06:30:32.310076+0000","last_fullsized":"2024-09-21T06:30:32.310076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:22.458706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.317568+0000","last_change":"2024-09-21T06:30:31.330489+0000","last_active":"2024-09-21T06:30:32.317568+0000","last_peered":"2024-09-21T06:30:32.317568+0000","last_clean":"2024-09-21T06:30:32.317568+0000","last_became_active":"2024-09-21T06:30:31.330195+0000","last_became_peered":"2024-09-21T06:30:31.330195+0000","last_unstale":"2024-09-21T06:30:32.317568+0000","last_undegraded":"2024-09-21T06:30:32.317568+0000","last_fullsized":"2024-09-21T06:30:32.317568+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:30.282222+0000","last_clean_scrub_stamp":"2024-09-21T06:30:30.282222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:50.599219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.311011+0000","last_change":"2024-09-21T06:30:30.313879+0000","last_active":"2024-09-21T06:30:32.311011+0000","last_peered":"2024-09-21T06:30:32.311011+0000","last_clean":"2024-09-21T06:30:32.311011+0000","last_became_active":"2024-09-21T06:30:30.310301+0000","last_became_peered":"2024-09-21T06:30:30.310301+0000","last_unstale":"2024-09-21T06:30:32.311011+0000","last_undegraded":"2024-09-21T06:30:32.311011+0000","last_fullsized":"2024-09-21T06:30:32.311011+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:30:29.274531+0000","last_clean_scrub_stamp":"2024-09-21T06:30:29.274531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.990441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T06:30:32.309936+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:30:32.309936+0000","last_peered":"2024-09-21T06:30:32.309936+0000","last_clean":"2024-09-21T06:30:32.309936+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:30:32.309936+0000","last_undegraded":"2024-09-21T06:30:32.309936+0000","last_fullsized":"2024-09-21T06:30:32.309936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":643072,"data_stored":281133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":643072,"data_stored":276089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":281111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1220608,"data_stored":864823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":281659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":873147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:30:36.303 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:30:36.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 -- 172.21.15.37:0/4183586405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00cddc0 msgr2=0x7faaa00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 --2- 172.21.15.37:0/4183586405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00cddc0 0x7faaa00cc810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7faaa01611d0 tx=0x7faa9002a480 comp rx=0 tx=0).stop 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 -- 172.21.15.37:0/4183586405 shutdown_connections 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 --2- 172.21.15.37:0/4183586405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00ccd50 0x7faaa00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 --2- 172.21.15.37:0/4183586405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00cddc0 0x7faaa00cc810 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 -- 172.21.15.37:0/4183586405 >> 172.21.15.37:0/4183586405 conn(0x7faaa006f940 msgr2=0x7faaa00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 -- 172.21.15.37:0/4183586405 shutdown_connections 2024-09-21T06:30:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.573+0000 7faaa5a00640 1 -- 172.21.15.37:0/4183586405 wait complete. 2024-09-21T06:30:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.576+0000 7faaa5a00640 1 Processor -- start 2024-09-21T06:30:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.576+0000 7faaa5a00640 1 -- start start 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 0x7faaa00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00cddc0 0x7faaa00b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faaa006e560 con 0x7faaa00b4ed0 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faaa006e3e0 con 0x7faaa00ccd50 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faaa5a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faaa015d630 con 0x7faaa00cddc0 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faa9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faa9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43866/0 (socket says 172.21.15.37:43866) 2024-09-21T06:30:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.577+0000 7faa9fe00640 1 -- 172.21.15.37:0/1865426327 learned_addr learned my addr 172.21.15.37:0/1865426327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa97e00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 0x7faaa00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faaa5000640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00cddc0 0x7faaa00b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 -- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00cddc0 msgr2=0x7faaa00b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00cddc0 0x7faaa00b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 -- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 msgr2=0x7faaa00b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 0x7faaa00b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 -- 172.21.15.37:0/1865426327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa0158de0 con 0x7faaa00b4ed0 2024-09-21T06:30:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa97e00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 0x7faaa00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9fe00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faa90009780 tx=0x7faa90033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.578+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90031d60 con 0x7faaa00b4ed0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.579+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa90033e10 con 0x7faaa00b4ed0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.579+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90005260 con 0x7faaa00b4ed0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.579+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa0159250 con 0x7faaa00b4ed0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.579+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa0159810 con 0x7faaa00b4ed0 2024-09-21T06:30:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.580+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa00b4990 con 0x7faaa00b4ed0 2024-09-21T06:30:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.581+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa90005400 con 0x7faaa00b4ed0 2024-09-21T06:30:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.582+0000 7faa9cc00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 0x7faa7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.582+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7faa900bc1c0 con 0x7faaa00b4ed0 2024-09-21T06:30:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.582+0000 7faa97e00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 0x7faa7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.583+0000 7faa97e00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 0x7faa7c07b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faaa01562f0 tx=0x7faa88005f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.586+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faa9003f030 con 0x7faaa00b4ed0 2024-09-21T06:30:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.859+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faaa00696c0 con 0x7faaa00b4ed0 2024-09-21T06:30:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.862+0000 7faa9cc00640 1 -- 172.21.15.37:0/1865426327 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7faa900843d0 con 0x7faaa00b4ed0 2024-09-21T06:30:36.864 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:30:36.864 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:30:36.865 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:30:36.865 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:30:36.865 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:30:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 msgr2=0x7faa7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 0x7faa7c07b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faaa01562f0 tx=0x7faa88005f10 comp rx=0 tx=0).stop 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 msgr2=0x7faaa0156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faa90009780 tx=0x7faa90033c20 comp rx=0 tx=0).stop 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 shutdown_connections 2024-09-21T06:30:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa7c078b90 0x7faa7c07b0a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaa00cddc0 0x7faaa00b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaa00ccd50 0x7faaa00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 --2- 172.21.15.37:0/1865426327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa00b4ed0 0x7faaa0156420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 >> 172.21.15.37:0/1865426327 conn(0x7faaa006f940 msgr2=0x7faaa0073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 shutdown_connections 2024-09-21T06:30:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:36.865+0000 7faaa5a00640 1 -- 172.21.15.37:0/1865426327 wait complete. 2024-09-21T06:30:36.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 -- 172.21.15.37:0/870371916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 msgr2=0x7f41a8073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 --2- 172.21.15.37:0/870371916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a8073dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f41a809e110 tx=0x7f419c02a560 comp rx=0 tx=0).stop 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 -- 172.21.15.37:0/870371916 shutdown_connections 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 --2- 172.21.15.37:0/870371916 >> v2:172.21.15.150:3300/0 conn(0x7f41a80cbc10 0x7f41a80cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 --2- 172.21.15.37:0/870371916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a8073dd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.115+0000 7f41afa00640 1 -- 172.21.15.37:0/870371916 >> 172.21.15.37:0/870371916 conn(0x7f41a80cca40 msgr2=0x7f41a80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:37.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.116+0000 7f41afa00640 1 -- 172.21.15.37:0/870371916 shutdown_connections 2024-09-21T06:30:37.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.116+0000 7f41afa00640 1 -- 172.21.15.37:0/870371916 wait complete. 2024-09-21T06:30:37.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.118+0000 7f41afa00640 1 Processor -- start 2024-09-21T06:30:37.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.118+0000 7f41afa00640 1 -- start start 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a806ddf0 0x7f41a8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41a80cbc10 0x7f41a81f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41a806f360 con 0x7f41a8074cb0 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41a806f1e0 con 0x7f41a806ddf0 2024-09-21T06:30:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.119+0000 7f41afa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f41a806f4e0 con 0x7f41a80cbc10 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43878/0 (socket says 172.21.15.37:43878) 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 -- 172.21.15.37:0/3092835770 learned_addr learned my addr 172.21.15.37:0/3092835770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41ada00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41a80cbc10 0x7f41a81f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41ad000640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a806ddf0 0x7f41a8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 -- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41a80cbc10 msgr2=0x7f41a81f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41a80cbc10 0x7f41a81f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 -- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a806ddf0 msgr2=0x7f41a8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a806ddf0 0x7f41a8157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.120+0000 7f41a7e00640 1 -- 172.21.15.37:0/3092835770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a8157d80 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41a7e00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f419400d6d0 tx=0x7f419400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194017070 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4194009b10 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a8158060 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194010460 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.121+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a81f6570 con 0x7f41a8074cb0 2024-09-21T06:30:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.122+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a80c9ef0 con 0x7f41a8074cb0 2024-09-21T06:30:37.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.123+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f419401a070 con 0x7f41a8074cb0 2024-09-21T06:30:37.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.124+0000 7f41a4c00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 0x7f418807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:37.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.124+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f419409af80 con 0x7f41a8074cb0 2024-09-21T06:30:37.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.125+0000 7f41ad000640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 0x7f418807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:37.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.125+0000 7f41ad000640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 0x7f418807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f419c033000 tx=0x7f419c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:37.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.129+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4194062b40 con 0x7f41a8074cb0 2024-09-21T06:30:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.364+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41a806f9d0 con 0x7f41a8074cb0 2024-09-21T06:30:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.364+0000 7f41a4c00640 1 -- 172.21.15.37:0/3092835770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5229 (secure 0 0 0) 0x7f4194067570 con 0x7f41a8074cb0 2024-09-21T06:30:37.366 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"btime":"2024-09-21T06:30:34:322066+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5506,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:30:33.835909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574]},"id":3}]} 2024-09-21T06:30:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.366+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 msgr2=0x7f418807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.366+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 0x7f418807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f419c033000 tx=0x7f419c004730 comp rx=0 tx=0).stop 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 msgr2=0x7f41a81f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f419400d6d0 tx=0x7f419400db90 comp rx=0 tx=0).stop 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 shutdown_connections 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4188078bc0 0x7f418807b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8074cb0 0x7f41a81f54f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41a80cbc10 0x7f41a81f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 --2- 172.21.15.37:0/3092835770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a806ddf0 0x7f41a8157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:37.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 >> 172.21.15.37:0/3092835770 conn(0x7f41a80cca40 msgr2=0x7f41a815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:37.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 shutdown_connections 2024-09-21T06:30:37.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:37.367+0000 7f41afa00640 1 -- 172.21.15.37:0/3092835770 wait complete. 2024-09-21T06:30:37.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:30:30.818966+0000', 'modified': '2024-09-21T06:30:33.835909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5574': {'gid': 5574, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/3035654558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3035654558}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 3035654558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5574, 'qdb_cluster': [5574]} 2024-09-21T06:30:37.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:30:37.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:30:38.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 -- 172.21.15.37:0/2538861618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc15c005250 msgr2=0x7fc15c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 --2- 172.21.15.37:0/2538861618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc15c005250 0x7fc15c00b9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc17000c770 tx=0x7fc15c005830 comp rx=0 tx=0).stop 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 -- 172.21.15.37:0/2538861618 shutdown_connections 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 --2- 172.21.15.37:0/2538861618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc15c005250 0x7fc15c00b9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 -- 172.21.15.37:0/2538861618 >> 172.21.15.37:0/2538861618 conn(0x7fc1780b1b90 msgr2=0x7fc1780b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 -- 172.21.15.37:0/2538861618 shutdown_connections 2024-09-21T06:30:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.026+0000 7fc17da00640 1 -- 172.21.15.37:0/2538861618 wait complete. 2024-09-21T06:30:38.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.029+0000 7fc17da00640 1 Processor -- start 2024-09-21T06:30:38.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.029+0000 7fc17da00640 1 -- start start 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c005250 0x7fc1781f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780af250 0x7fc1781e9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc178070d20 con 0x7fc1781e98b0 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc178070ba0 con 0x7fc1780af250 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.030+0000 7fc17da00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc17806ddf0 con 0x7fc15c005250 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43884/0 (socket says 172.21.15.37:43884) 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 -- 172.21.15.37:0/1739954613 learned_addr learned my addr 172.21.15.37:0/1739954613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc176a00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c005250 0x7fc1781f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc176000640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780af250 0x7fc1781e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 -- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c005250 msgr2=0x7fc1781f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c005250 0x7fc1781f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 -- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780af250 msgr2=0x7fc1781e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780af250 0x7fc1781e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 -- 172.21.15.37:0/1739954613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1781e9cc0 con 0x7fc1781e98b0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.031+0000 7fc177400640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc17002b620 tx=0x7fc170001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.032+0000 7fc16ea00640 1 -- 172.21.15.37:0/1739954613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170039070 con 0x7fc1781e98b0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.032+0000 7fc16ea00640 1 -- 172.21.15.37:0/1739954613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc170034470 con 0x7fc1781e98b0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.032+0000 7fc16ea00640 1 -- 172.21.15.37:0/1739954613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700348a0 con 0x7fc1781e98b0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.032+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1781f9350 con 0x7fc1781e98b0 2024-09-21T06:30:38.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.032+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781f9b20 con 0x7fc1781e98b0 2024-09-21T06:30:38.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.035+0000 7fc16ea00640 1 -- 172.21.15.37:0/1739954613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc170034a40 con 0x7fc1781e98b0 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.036+0000 7fc16ea00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.036+0000 7fc16ea00640 1 -- 172.21.15.37:0/1739954613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc1700bae50 con 0x7fc1781e98b0 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.037+0000 7fc184a83740 1 Processor -- start 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.037+0000 7fc176a00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.037+0000 7fc184a83740 1 -- start start 2024-09-21T06:30:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc176a00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc168009bd0 tx=0x7fc168005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db41590 0x55983df02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55983de8c430 con 0x55983db62270 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55983de7c5d0 con 0x55983db393b0 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.038+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55983de6ba00 con 0x55983db41590 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43888/0 (socket says 172.21.15.37:43888) 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 -- 172.21.15.37:0/3567045814 learned_addr learned my addr 172.21.15.37:0/3567045814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc177400640 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db41590 0x55983df02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176000640 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 -- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db41590 msgr2=0x55983df02f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db41590 0x55983df02f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 -- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 msgr2=0x55983df32eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df32eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 -- 172.21.15.37:0/3567045814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983de34520 con 0x55983db62270 2024-09-21T06:30:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.039+0000 7fc176a00640 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1680096d0 tx=0x7fc168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2bd4569c54a832 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.040+0000 7fc163400640 1 -- 172.21.15.37:0/3567045814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168003b70 con 0x55983db62270 2024-09-21T06:30:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.040+0000 7fc163400640 1 -- 172.21.15.37:0/3567045814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc168003d10 con 0x55983db62270 2024-09-21T06:30:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.040+0000 7fc163400640 1 -- 172.21.15.37:0/3567045814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168005280 con 0x55983db62270 2024-09-21T06:30:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.040+0000 7fc184a83740 1 -- 172.21.15.37:0/3567045814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 msgr2=0x55983df32970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.040+0000 7fc184a83740 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1680096d0 tx=0x7fc168017000 comp rx=0 tx=0).stop 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 -- 172.21.15.37:0/3567045814 shutdown_connections 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db41590 0x55983df02f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df32eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 --2- 172.21.15.37:0/3567045814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df32970 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 -- 172.21.15.37:0/3567045814 >> 172.21.15.37:0/3567045814 conn(0x55983db563c0 msgr2=0x55983df08970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 -- 172.21.15.37:0/3567045814 shutdown_connections 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.041+0000 7fc184a83740 1 -- 172.21.15.37:0/3567045814 wait complete. 2024-09-21T06:30:38.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.042+0000 7fc184a83740 1 -- start start 2024-09-21T06:30:38.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.042+0000 7fc184a83740 1 Processor -- start 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df5dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55983de12330 con 0x55983db62270 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55983de6f810 con 0x55983db41590 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.043+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55983dbdc620 con 0x55983db393b0 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc184a83740 10 client.0 _ll_register_callbacks cb 0x55983de17420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43902/0 (socket says 172.21.15.37:43902) 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 -- 172.21.15.37:0/3183063429 learned_addr learned my addr 172.21.15.37:0/3183063429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc176a00640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 -- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 msgr2=0x55983df485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc176000640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df5dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 -- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 msgr2=0x55983df5dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df5dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 -- 172.21.15.37:0/3183063429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983dbcd850 con 0x55983db62270 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc176a00640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc176000640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df5dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc177400640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b9150 tx=0x7fc1700824a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce66ddff4d594e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.044+0000 7fc157e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170082a20 con 0x55983db62270 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc168003d10 con 0x55983db62270 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc17008bfb0 con 0x55983db62270 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc170087500 con 0x55983db62270 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 10 client.5697 fetch_fsmap learned FSMap version 33 2024-09-21T06:30:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55983dc446b0 con 0x55983db62270 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc168003d10 con 0x55983db62270 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7fc170081c80 con 0x55983db62270 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 10 client.5697 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 10 client.5697 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 20 client.5697 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983db20e40 0x55983df6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 4 client.5697 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.045+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55983de7e520 con 0x55983db20e40 2024-09-21T06:30:38.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.046+0000 7fc176a00640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983db20e40 0x55983df6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.046+0000 7fc176a00640 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983db20e40 0x55983df6a340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f2c51bf30b9c71 server_cookie=f51c30bfc71c250f in_seq=0 out_seq=0 2024-09-21T06:30:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.046+0000 7fc157e00640 10 client.5697 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.047+0000 7fc157e00640 1 -- 172.21.15.37:0/3183063429 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55983de7e520 con 0x55983db20e40 2024-09-21T06:30:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.047+0000 7fc157e00640 10 client.5697 handle_command_reply: tid=0 2024-09-21T06:30:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.047+0000 7fc184a83740 1 client.5697 shutdown 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.047+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 msgr2=0x55983df5e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.047+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b9150 tx=0x7fc1700824a0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 shutdown_connections 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df5e110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983db20e40 0x55983df6a340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df5dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 --2- 172.21.15.37:0/3183063429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc157e00640 0 client.5697 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 >> 172.21.15.37:0/3183063429 conn(0x55983db563c0 msgr2=0x55983df4f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 shutdown_connections 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 1 -- 172.21.15.37:0/3183063429 wait complete. 2024-09-21T06:30:38.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.048+0000 7fc184a83740 20 client.5697 trim_cache size 0 max 16384 2024-09-21T06:30:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.067+0000 7fc184a83740 1 Processor -- start 2024-09-21T06:30:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.067+0000 7fc184a83740 1 -- start start 2024-09-21T06:30:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df5dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db62270 0x55983df2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc178201bf0 con 0x55983db41590 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc17815e900 con 0x55983db393b0 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1782015a0 con 0x55983db62270 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43914/0 (socket says 172.21.15.37:43914) 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 -- 172.21.15.37:0/1127673151 learned_addr learned my addr 172.21.15.37:0/1127673151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176a00640 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df5dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc177400640 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db62270 0x55983df2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 -- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db62270 msgr2=0x55983df2bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db62270 0x55983df2bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 -- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 msgr2=0x55983df5dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df5dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.069+0000 7fc176000640 1 -- 172.21.15.37:0/1127673151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983df2f8b0 con 0x55983db41590 2024-09-21T06:30:38.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc176000640 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc1640046b0 tx=0x7fc164004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e42e86bfca054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc162a00640 1 -- 172.21.15.37:0/1127673151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164013930 con 0x55983db41590 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc162a00640 1 -- 172.21.15.37:0/1127673151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc164013ad0 con 0x55983db41590 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc162a00640 1 -- 172.21.15.37:0/1127673151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16401d420 con 0x55983db41590 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc184a83740 1 -- 172.21.15.37:0/1127673151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 msgr2=0x55983df2b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.070+0000 7fc184a83740 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc1640046b0 tx=0x7fc164004b70 comp rx=0 tx=0).stop 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 -- 172.21.15.37:0/1127673151 shutdown_connections 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db41590 0x55983df2b830 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db62270 0x55983df2bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 --2- 172.21.15.37:0/1127673151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db393b0 0x55983df5dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 -- 172.21.15.37:0/1127673151 >> 172.21.15.37:0/1127673151 conn(0x55983db563c0 msgr2=0x55983df02e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 -- 172.21.15.37:0/1127673151 shutdown_connections 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.071+0000 7fc184a83740 1 -- 172.21.15.37:0/1127673151 wait complete. 2024-09-21T06:30:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.072+0000 7fc184a83740 1 -- start start 2024-09-21T06:30:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.072+0000 7fc184a83740 1 Processor -- start 2024-09-21T06:30:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55983df5ec50 con 0x55983db62270 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55983de12330 con 0x55983db41590 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.073+0000 7fc184a83740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55983df5cd70 con 0x55983db393b0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc184a83740 10 client.0 _ll_register_callbacks cb 0x7fc170082860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43922/0 (socket says 172.21.15.37:43922) 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 -- 172.21.15.37:0/77288237 learned_addr learned my addr 172.21.15.37:0/77288237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc176a00640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc176000640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 -- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 msgr2=0x55983df2af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df2af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 -- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 msgr2=0x55983df4e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df4e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 -- 172.21.15.37:0/77288237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983df04740 con 0x55983db62270 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc177400640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc17007fc90 tx=0x7fc170081eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1036242a645d59b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.074+0000 7fc157e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700b8060 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc17008c1c0 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc164013ad0 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc170092050 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 10 client.5703 fetch_fsmap learned FSMap version 33 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55983df2b480 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc164013ad0 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7fc170095020 con 0x55983db62270 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 10 client.5703 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 10 client.5703 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 20 client.5703 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983dfcd370 0x55983df01c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 4 client.5703 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:30:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.075+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "status"}) -- 0x55983de79fd0 con 0x55983dfcd370 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc176a00640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983dfcd370 0x55983df01c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc176a00640 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983dfcd370 0x55983df01c60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78a65781ff12a44 server_cookie=9c0d674577bad634 in_seq=0 out_seq=0 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc157e00640 10 client.5703 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc157e00640 1 -- 172.21.15.37:0/77288237 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55983de79fd0 con 0x55983dfcd370 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc157e00640 10 client.5703 handle_command_reply: tid=0 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc184a83740 1 client.5703 shutdown 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 msgr2=0x55983df4ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.076+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc17007fc90 tx=0x7fc170081eb0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 shutdown_connections 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55983db62270 0x55983df4ec80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55983dfcd370 0x55983df01c60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55983db41590 0x55983df4e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 --2- 172.21.15.37:0/77288237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55983db393b0 0x55983df2af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc157e00640 0 client.5703 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 >> 172.21.15.37:0/77288237 conn(0x55983db563c0 msgr2=0x55983df08940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 shutdown_connections 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 1 -- 172.21.15.37:0/77288237 wait complete. 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.077+0000 7fc184a83740 20 client.5703 trim_cache size 0 max 16384 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5574, 2024-09-21T06:30:38.082 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 6.7722029910000003, 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 33, 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 54, 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 54, 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.760525253000001 2024-09-21T06:30:38.083 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 msgr2=0x7fc15807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc168009bd0 tx=0x7fc168005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 msgr2=0x7fc1780b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc17002b620 tx=0x7fc170001740 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 shutdown_connections 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781e98b0 0x7fc1780b2ed0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780af250 0x7fc1781e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 --2- 172.21.15.37:0/1739954613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c005250 0x7fc1781f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 >> 172.21.15.37:0/1739954613 conn(0x7fc1780b1b90 msgr2=0x7fc1780cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 shutdown_connections 2024-09-21T06:30:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.078+0000 7fc17da00640 1 -- 172.21.15.37:0/1739954613 wait complete. 2024-09-21T06:30:38.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:30:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 -- 172.21.15.37:0/2180494938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50006e500 msgr2=0x7fc5000ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 --2- 172.21.15.37:0/2180494938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50006e500 0x7fc5000ce960 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8005fd0 tx=0x7fc4f802aa90 comp rx=0 tx=0).stop 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 -- 172.21.15.37:0/2180494938 shutdown_connections 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 --2- 172.21.15.37:0/2180494938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50006e500 0x7fc5000ce960 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 --2- 172.21.15.37:0/2180494938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc500073f20 0x7fc50015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 -- 172.21.15.37:0/2180494938 >> 172.21.15.37:0/2180494938 conn(0x7fc500073400 msgr2=0x7fc500073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 -- 172.21.15.37:0/2180494938 shutdown_connections 2024-09-21T06:30:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.331+0000 7fc505000640 1 -- 172.21.15.37:0/2180494938 wait complete. 2024-09-21T06:30:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.333+0000 7fc505000640 1 Processor -- start 2024-09-21T06:30:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.334+0000 7fc505000640 1 -- start start 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc500073f20 0x7fc5001f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5000afb80 0x7fc500157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000cf3f0 con 0x7fc5001e8460 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000cf270 con 0x7fc500073f20 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc505000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000cf570 con 0x7fc5000afb80 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43952/0 (socket says 172.21.15.37:43952) 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc4ffe00640 1 -- 172.21.15.37:0/3518645662 learned_addr learned my addr 172.21.15.37:0/3518645662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc4ff400640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc500073f20 0x7fc5001f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.335+0000 7fc4f7e00640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5000afb80 0x7fc500157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 -- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5000afb80 msgr2=0x7fc500157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5000afb80 0x7fc500157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 -- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc500073f20 msgr2=0x7fc5001f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc500073f20 0x7fc5001f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 -- 172.21.15.37:0/3518645662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5001f6cd0 con 0x7fc5001e8460 2024-09-21T06:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4ffe00640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006010 tx=0x7fc4f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8044070 con 0x7fc5001e8460 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f802e7f0 con 0x7fc5001e8460 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80044b0 con 0x7fc5001e8460 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.336+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5001e73c0 con 0x7fc5001e8460 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.337+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5001e7980 con 0x7fc5001e8460 2024-09-21T06:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.338+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5001e7130 con 0x7fc5001e8460 2024-09-21T06:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.339+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4f8047070 con 0x7fc5001e8460 2024-09-21T06:30:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.340+0000 7fc4f7400640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 0x7fc4dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.340+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc4f80bbf00 con 0x7fc5001e8460 2024-09-21T06:30:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.340+0000 7fc4ff400640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 0x7fc4dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:38.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.341+0000 7fc4ff400640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 0x7fc4dc07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000bb50 tx=0x7fc4f000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.344+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5001f6cd0 con 0x7fc5001e8460 2024-09-21T06:30:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.579+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc500159580 con 0x7fc5001e8460 2024-09-21T06:30:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.579+0000 7fc4f7400640 1 -- 172.21.15.37:0/3518645662 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7fc500159580 con 0x7fc5001e8460 2024-09-21T06:30:38.582 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.581+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 msgr2=0x7fc4dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.581+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 0x7fc4dc07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000bb50 tx=0x7fc4f000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.581+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 msgr2=0x7fc5000b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.581+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006010 tx=0x7fc4f8003f10 comp rx=0 tx=0).stop 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 shutdown_connections 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4dc078b90 0x7fc4dc07b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5001e8460 0x7fc5000b56f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5000afb80 0x7fc500157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 --2- 172.21.15.37:0/3518645662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc500073f20 0x7fc5001f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 >> 172.21.15.37:0/3518645662 conn(0x7fc500073400 msgr2=0x7fc500159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 shutdown_connections 2024-09-21T06:30:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38.582+0000 7fc505000640 1 -- 172.21.15.37:0/3518645662 wait complete. 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:30:38.606 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:30:38.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:38.606 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:30:38.643 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:30:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:30:38.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:38.645 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:30:38.736 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:30:38.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:38.739 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:30:38.775 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:30:38.776 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:30:38.776 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:30:38.776 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:30:38.776 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:30:38.777 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:30:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:30:38.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:38.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:38.975 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:30:39.013 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:30:39.013 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:30:39.013 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:30:39.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:39.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:30:39.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:30:39.110 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:30:39.145 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:30:39.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:30:39.146 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:30:39.221 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:30:39.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:39.221 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:30:39.442 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:30:39.443 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:30:39.443 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:30:39.443 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:30:39.443 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:30:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:30:39.475 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:30:39.476 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:30:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:30:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:30:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:30:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:30:39.479 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:30:39.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:39.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:30:39.535 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:30:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:30:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:client5327 2024-09-21T06:30:39.633 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:30:39.633 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:30:39.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 -- 172.21.15.37:0/1639636286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 msgr2=0x7f4e5406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 --2- 172.21.15.37:0/1639636286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e5406ddf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009830 tx=0x7f4e4002a530 comp rx=0 tx=0).stop 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 -- 172.21.15.37:0/1639636286 shutdown_connections 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 --2- 172.21.15.37:0/1639636286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e5406ddf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 --2- 172.21.15.37:0/1639636286 >> v2:172.21.15.150:3300/0 conn(0x7f4e54156a50 0x7f4e54074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 -- 172.21.15.37:0/1639636286 >> 172.21.15.37:0/1639636286 conn(0x7f4e540b1b90 msgr2=0x7f4e540b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 -- 172.21.15.37:0/1639636286 shutdown_connections 2024-09-21T06:30:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.938+0000 7f4e5c400640 1 -- 172.21.15.37:0/1639636286 wait complete. 2024-09-21T06:30:39.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.941+0000 7f4e5c400640 1 Processor -- start 2024-09-21T06:30:39.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.941+0000 7f4e5c400640 1 -- start start 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e540c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e54156a50 0x7f4e540cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540ce500 con 0x7f4e5406e330 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540ce380 con 0x7f4e54156a50 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.942+0000 7f4e5c400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540ce680 con 0x7f4e54074b60 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43964/0 (socket says 172.21.15.37:43964) 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 -- 172.21.15.37:0/3028150214 learned_addr learned my addr 172.21.15.37:0/3028150214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59000640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e540c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e5a400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e54156a50 0x7f4e540cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 -- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 msgr2=0x7f4e540c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e540c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 -- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e54156a50 msgr2=0x7f4e540cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e54156a50 0x7f4e540cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 -- 172.21.15.37:0/3028150214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540c7610 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.943+0000 7f4e59a00640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.944+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800cd20 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.944+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e48004020 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.944+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48004450 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.944+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e540ca9d0 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.944+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e540caf90 con 0x7f4e5406e330 2024-09-21T06:30:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.945+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5406ddf0 con 0x7f4e5406e330 2024-09-21T06:30:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.946+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e480035f0 con 0x7f4e5406e330 2024-09-21T06:30:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.947+0000 7f4e51600640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.947+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f4e4809b700 con 0x7f4e5406e330 2024-09-21T06:30:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.948+0000 7f4e59000640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.949+0000 7f4e59000640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 0x7f4e3407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e40034000 tx=0x7f4e40004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:39.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:39.951+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e48063330 con 0x7f4e5406e330 2024-09-21T06:30:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.195+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f4e540cde10 con 0x7f4e5406e330 2024-09-21T06:30:40.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.875+0000 7f4e51600640 1 -- 172.21.15.37:0/3028150214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v34) ==== 105+0+0 (secure 0 0 0) 0x7f4e48067d60 con 0x7f4e5406e330 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 msgr2=0x7f4e3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 0x7f4e3407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e40034000 tx=0x7f4e40004730 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 msgr2=0x7f4e540b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 shutdown_connections 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e54156a50 0x7f4e540cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e54074b60 0x7f4e540c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 --2- 172.21.15.37:0/3028150214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5406e330 0x7f4e540b5870 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.878+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 >> 172.21.15.37:0/3028150214 conn(0x7f4e540b1b90 msgr2=0x7f4e54156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.879+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 shutdown_connections 2024-09-21T06:30:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:40.879+0000 7f4e5c400640 1 -- 172.21.15.37:0/3028150214 wait complete. 2024-09-21T06:30:40.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 -- 172.21.15.37:0/4156887776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c84003a00 msgr2=0x7f2c84003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/4156887776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c84003a00 0x7f2c84003e70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009b20 tx=0x7f2c84005830 comp rx=0 tx=0).stop 2024-09-21T06:30:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca1c00640 1 -- 172.21.15.37:0/4156887776 reap_dead start 2024-09-21T06:30:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 -- 172.21.15.37:0/4156887776 shutdown_connections 2024-09-21T06:30:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 -- 172.21.15.37:0/4156887776 >> 172.21.15.37:0/4156887776 conn(0x7f2c9c0cca40 msgr2=0x7f2c9c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 -- 172.21.15.37:0/4156887776 shutdown_connections 2024-09-21T06:30:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.143+0000 7f2ca3c00640 1 -- 172.21.15.37:0/4156887776 wait complete. 2024-09-21T06:30:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.145+0000 7f2ca3c00640 1 Processor -- start 2024-09-21T06:30:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.145+0000 7f2ca3c00640 1 -- start start 2024-09-21T06:30:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 0x7f2c9c0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0b5700 0x7f2c9c1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c06ede0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c074200 con 0x7f2c84003a00 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9c06ef60 con 0x7f2c9c0b5700 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 0x7f2c9c0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43996/0 (socket says 172.21.15.37:43996) 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2c9be00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0b5700 0x7f2c9c1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.147+0000 7f2ca1c00640 1 -- 172.21.15.37:0/2150525706 learned_addr learned my addr 172.21.15.37:0/2150525706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 -- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0b5700 msgr2=0x7f2c9c1f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0b5700 0x7f2c9c1f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 -- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 msgr2=0x7f2c9c0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 0x7f2c9c0b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 -- 172.21.15.37:0/2150525706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c1f70f0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1200640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 0x7f2c9c0b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca1c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c15dc90 tx=0x7f2c9400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003970 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c94015070 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c1f3000 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003d90 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.148+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c0c47a0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.150+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c1f6e60 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.151+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2c94004030 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.152+0000 7f2c98c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.152+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f2c9409baa0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.153+0000 7f2ca1200640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.153+0000 7f2ca1200640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009b20 tx=0x7f2c9002d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.157+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c940636e0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.392+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c9c1f2df0 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.393+0000 7f2c98c00640 1 -- 172.21.15.37:0/2150525706 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5245 (secure 0 0 0) 0x7f2c94068110 con 0x7f2c9c1f3ae0 2024-09-21T06:30:41.395 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:41.395 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"btime":"2024-09-21T06:30:40:879882+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:30:40.879853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5574,"mds_1":5506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:creating","state_seq":1,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574]},"id":3}]} 2024-09-21T06:30:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 msgr2=0x7f2c7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009b20 tx=0x7f2c9002d040 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 msgr2=0x7f2c9c1f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c15dc90 tx=0x7f2c9400e720 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 shutdown_connections 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c9c1f3ae0 0x7f2c9c1f68d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0b5700 0x7f2c9c1f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 --2- 172.21.15.37:0/2150525706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c84003a00 0x7f2c9c0b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 >> 172.21.15.37:0/2150525706 conn(0x7f2c9c0cca40 msgr2=0x7f2c9c15e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 shutdown_connections 2024-09-21T06:30:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:41.395+0000 7f2ca3c00640 1 -- 172.21.15.37:0/2150525706 wait complete. 2024-09-21T06:30:41.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:30:30.818966+0000', 'modified': '2024-09-21T06:30:40.879853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5574, 'mds_1': 5506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5506': {'gid': 5506, 'name': 'a', 'rank': 1, 'incarnation': 35, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.37:6835/1998484049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1998484049}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1998484049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5574': {'gid': 5574, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/3035654558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3035654558}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 3035654558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5574, 'qdb_cluster': [5574]} 2024-09-21T06:30:41.823 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5506:up:creating 2024-09-21T06:30:42.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 -- 172.21.15.37:0/3054004769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 msgr2=0x7f6fb8073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 --2- 172.21.15.37:0/3054004769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb8073dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009830 tx=0x7f6fac02a510 comp rx=0 tx=0).stop 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 -- 172.21.15.37:0/3054004769 shutdown_connections 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 --2- 172.21.15.37:0/3054004769 >> v2:172.21.15.150:3300/0 conn(0x7f6fb806e500 0x7f6fb80cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 --2- 172.21.15.37:0/3054004769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb8073dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 -- 172.21.15.37:0/3054004769 >> 172.21.15.37:0/3054004769 conn(0x7f6fb80cca40 msgr2=0x7f6fb80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 -- 172.21.15.37:0/3054004769 shutdown_connections 2024-09-21T06:30:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.065+0000 7f6fc0400640 1 -- 172.21.15.37:0/3054004769 wait complete. 2024-09-21T06:30:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.067+0000 7f6fc0400640 1 Processor -- start 2024-09-21T06:30:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.067+0000 7f6fc0400640 1 -- start start 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb81f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 0x7f6fb81f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb806f360 con 0x7f6fb80cbc10 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb806f1e0 con 0x7f6fb80a0db0 2024-09-21T06:30:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fc0400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb806f4e0 con 0x7f6fb8074cb0 2024-09-21T06:30:43.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44006/0 (socket says 172.21.15.37:44006) 2024-09-21T06:30:43.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbd000640 1 -- 172.21.15.37:0/855018808 learned_addr learned my addr 172.21.15.37:0/855018808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbda00640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb81f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbe400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 0x7f6fb81f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbd000640 1 -- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 msgr2=0x7f6fb81f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.069+0000 7f6fbd000640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb81f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fbd000640 1 -- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 msgr2=0x7f6fb81f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fbd000640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 0x7f6fb81f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fbd000640 1 -- 172.21.15.37:0/855018808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb80b5ec0 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fbe400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 0x7f6fb81f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fbd000640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c460 tx=0x7f6fa400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4016020 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fa4012940 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4012d90 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb81f8170 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.070+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb809d9d0 con 0x7f6fb80cbc10 2024-09-21T06:30:43.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.071+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb806fa00 con 0x7f6fb80cbc10 2024-09-21T06:30:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.072+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fa4019020 con 0x7f6fb80cbc10 2024-09-21T06:30:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.073+0000 7f6fb5600640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 0x7f6f9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.073+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f6fa409ab90 con 0x7f6fb80cbc10 2024-09-21T06:30:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.074+0000 7f6fbda00640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 0x7f6f9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.075+0000 7f6fbda00640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 0x7f6f9807b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fac034000 tx=0x7f6fac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.078+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fa4020030 con 0x7f6fb80cbc10 2024-09-21T06:30:43.282 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:30:43.280+0000 7f934ea00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:30:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.311+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fb80b2f80 con 0x7f6fb80cbc10 2024-09-21T06:30:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.312+0000 7f6fb5600640 1 -- 172.21.15.37:0/855018808 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5267 (secure 0 0 0) 0x7f6fb80b5ec0 con 0x7f6fb80cbc10 2024-09-21T06:30:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:43.315 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":38,"btime":"2024-09-21T06:30:42:894731+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:30:42.894719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5500,5506,5574]},"id":3}]} 2024-09-21T06:30:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 38 2024-09-21T06:30:43.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 msgr2=0x7f6f9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 0x7f6f9807b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fac034000 tx=0x7f6fac004730 comp rx=0 tx=0).stop 2024-09-21T06:30:43.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 msgr2=0x7f6fb80cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c460 tx=0x7f6fa400c920 comp rx=0 tx=0).stop 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 shutdown_connections 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f98078b90 0x7f6f9807b0a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb80cbc10 0x7f6fb80cb110 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fb80a0db0 0x7f6fb81f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 --2- 172.21.15.37:0/855018808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fb8074cb0 0x7f6fb81f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 >> 172.21.15.37:0/855018808 conn(0x7f6fb80cca40 msgr2=0x7f6fb81509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 shutdown_connections 2024-09-21T06:30:43.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.315+0000 7f6fc0400640 1 -- 172.21.15.37:0/855018808 wait complete. 2024-09-21T06:30:43.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:30:30.818966+0000', 'modified': '2024-09-21T06:30:42.894719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5574, 'mds_1': 5506, 'mds_2': 5500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5500': {'gid': 5500, 'name': 'd', 'rank': 2, 'incarnation': 37, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.150:6835/1705702768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 1705702768}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 1705702768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5506': {'gid': 5506, 'name': 'a', 'rank': 1, 'incarnation': 35, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6835/1998484049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1998484049}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1998484049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5574': {'gid': 5574, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/3035654558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3035654558}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 3035654558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5574, 'qdb_cluster': [5500, 5506, 5574]} 2024-09-21T06:30:43.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:30:43.742 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:30:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.971+0000 7f7eb5400640 1 -- 172.21.15.37:0/845260433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb01605f0 msgr2=0x7f7eb00702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 --2- 172.21.15.37:0/845260433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb01605f0 0x7f7eb00702a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005fd0 tx=0x7f7e9c02a540 comp rx=0 tx=0).stop 2024-09-21T06:30:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 -- 172.21.15.37:0/845260433 shutdown_connections 2024-09-21T06:30:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 --2- 172.21.15.37:0/845260433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb01605f0 0x7f7eb00702a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 -- 172.21.15.37:0/845260433 >> 172.21.15.37:0/845260433 conn(0x7f7eb00d05c0 msgr2=0x7f7eb006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 -- 172.21.15.37:0/845260433 shutdown_connections 2024-09-21T06:30:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.972+0000 7f7eb5400640 1 -- 172.21.15.37:0/845260433 wait complete. 2024-09-21T06:30:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.974+0000 7f7eb5400640 1 Processor -- start 2024-09-21T06:30:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.974+0000 7f7eb5400640 1 -- start start 2024-09-21T06:30:43.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 0x7f7eb0143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb00cf6d0 0x7f7eb01e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b3e50 con 0x7f7eb0141680 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b3cd0 con 0x7f7eb00cc0e0 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.975+0000 7f7eb5400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb00b3fd0 con 0x7f7eb00cf6d0 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41798/0 (socket says 172.21.15.37:41798) 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 -- 172.21.15.37:0/2132041325 learned_addr learned my addr 172.21.15.37:0/2132041325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eafe00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 0x7f7eb0143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eaf400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb00cf6d0 0x7f7eb01e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 -- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb00cf6d0 msgr2=0x7f7eb01e93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb00cf6d0 0x7f7eb01e93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 -- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 msgr2=0x7f7eb0143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 0x7f7eb0143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eb4a00640 1 -- 172.21.15.37:0/2132041325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01f89d0 con 0x7f7eb0141680 2024-09-21T06:30:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.976+0000 7f7eafe00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 0x7f7eb0143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800b9a0 tx=0x7f7ea800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7ea7e00640 1 -- 172.21.15.37:0/2132041325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8019070 con 0x7f7eb0141680 2024-09-21T06:30:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7ea7e00640 1 -- 172.21.15.37:0/2132041325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea800cc90 con 0x7f7eb0141680 2024-09-21T06:30:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7ea7e00640 1 -- 172.21.15.37:0/2132041325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80146a0 con 0x7f7eb0141680 2024-09-21T06:30:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01f8cb0 con 0x7f7eb0141680 2024-09-21T06:30:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.977+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01f9270 con 0x7f7eb0141680 2024-09-21T06:30:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.979+0000 7f7ea7e00640 1 -- 172.21.15.37:0/2132041325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ea801c070 con 0x7f7eb0141680 2024-09-21T06:30:43.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.980+0000 7f7ea7e00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 0x7f7e9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.980+0000 7f7ea7e00640 1 -- 172.21.15.37:0/2132041325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f7ea809ac50 con 0x7f7eb0141680 2024-09-21T06:30:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.981+0000 7f7ebdc00740 1 Processor -- start 2024-09-21T06:30:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.981+0000 7f7eafe00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 0x7f7e9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.981+0000 7f7ebdc00740 1 -- start start 2024-09-21T06:30:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.982+0000 7f7eafe00640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 0x7f7e9007b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000f60 tx=0x7f7ea0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf904b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf50b3b0 0x560fbf9050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560fbf85e430 con 0x560fbf513590 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560fbf84e5d0 con 0x560fbf534270 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560fbf83da00 con 0x560fbf50b3b0 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41808/0 (socket says 172.21.15.37:41808) 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 -- 172.21.15.37:0/1589486594 learned_addr learned my addr 172.21.15.37:0/1589486594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eaf400640 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf50b3b0 0x560fbf9050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eafe00640 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf904b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 -- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf50b3b0 msgr2=0x560fbf9050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf50b3b0 0x560fbf9050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 -- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 msgr2=0x560fbf904b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf904b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.983+0000 7f7eb4a00640 1 -- 172.21.15.37:0/1589486594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbf806520 con 0x560fbf513590 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.984+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8098e70 tx=0x7f7ea8062640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68052a14360818b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.984+0000 7f7e9be00640 1 -- 172.21.15.37:0/1589486594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8062e40 con 0x560fbf513590 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.984+0000 7f7e9be00640 1 -- 172.21.15.37:0/1589486594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea8062740 con 0x560fbf513590 2024-09-21T06:30:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.984+0000 7f7e9be00640 1 -- 172.21.15.37:0/1589486594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea806bf30 con 0x560fbf513590 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 -- 172.21.15.37:0/1589486594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 msgr2=0x560fbf8d5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8098e70 tx=0x7f7ea8062640 comp rx=0 tx=0).stop 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 -- 172.21.15.37:0/1589486594 shutdown_connections 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf513590 0x560fbf8d5150 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf50b3b0 0x560fbf9050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/1589486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf904b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 -- 172.21.15.37:0/1589486594 >> 172.21.15.37:0/1589486594 conn(0x560fbf5283c0 msgr2=0x560fbf8dab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 -- 172.21.15.37:0/1589486594 shutdown_connections 2024-09-21T06:30:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.985+0000 7f7ebdc00740 1 -- 172.21.15.37:0/1589486594 wait complete. 2024-09-21T06:30:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.986+0000 7f7ebdc00740 1 -- start start 2024-09-21T06:30:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.986+0000 7f7ebdc00740 1 Processor -- start 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf932740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf932c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560fbf7e4330 con 0x560fbf50b3b0 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560fbf841810 con 0x560fbf513590 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560fbf5ae620 con 0x560fbf534270 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7ebdc00740 10 client.0 _ll_register_callbacks cb 0x560fbf7e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41818/0 (socket says 172.21.15.37:41818) 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eafe00640 1 -- 172.21.15.37:0/967278255 learned_addr learned my addr 172.21.15.37:0/967278255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eaf400640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf932740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf932c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.988+0000 7f7eafe00640 1 -- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 msgr2=0x560fbf932c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7eafe00640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf932c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7eafe00640 1 -- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 msgr2=0x560fbf932740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7eafe00640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf932740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7eafe00640 1 -- 172.21.15.37:0/967278255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbf59f850 con 0x560fbf50b3b0 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7eafe00640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ea60 tx=0x7f7ea0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c585183833f91e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0027060 con 0x560fbf50b3b0 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f7ea0010040 con 0x560fbf50b3b0 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea8062740 con 0x560fbf50b3b0 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7ea0005480 con 0x560fbf50b3b0 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7ebdc00740 10 client.5739 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.989+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x560fbf6166b0 con 0x560fbf50b3b0 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea8062740 con 0x560fbf50b3b0 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7f7ea0013420 con 0x560fbf50b3b0 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 10 client.5739 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 10 client.5739 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 20 client.5739 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf4f2e40 0x560fbf9005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 4 client.5739 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbf850520 con 0x560fbf4f2e40 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.990+0000 7f7eaf400640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf4f2e40 0x560fbf9005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.991+0000 7f7eaf400640 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf4f2e40 0x560fbf9005b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c2a46c1066efe65 server_cookie=47dede7d52479773 in_seq=0 out_seq=0 2024-09-21T06:30:43.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.991+0000 7f7e98c00640 10 client.5739 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:43.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7e98c00640 1 -- 172.21.15.37:0/967278255 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560fbf850520 con 0x560fbf4f2e40 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7e98c00640 10 client.5739 handle_command_reply: tid=0 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 client.5739 shutdown 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 msgr2=0x560fbf8fa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ea60 tx=0x7f7ea0000ed0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 shutdown_connections 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf4f2e40 0x560fbf9005b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf932c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf932740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/967278255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf8fa920 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7e98c00640 0 client.5739 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.992+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 >> 172.21.15.37:0/967278255 conn(0x560fbf5283c0 msgr2=0x560fbf904b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.993+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 shutdown_connections 2024-09-21T06:30:43.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.993+0000 7f7ebdc00740 1 -- 172.21.15.37:0/967278255 wait complete. 2024-09-21T06:30:43.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:43.993+0000 7f7ebdc00740 20 client.5739 trim_cache size 0 max 16384 2024-09-21T06:30:44.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.012+0000 7f7ebdc00740 1 Processor -- start 2024-09-21T06:30:44.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.012+0000 7f7ebdc00740 1 -- start start 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf942c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf943140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0201720 con 0x560fbf50b3b0 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0201df0 con 0x560fbf513590 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb0200980 con 0x560fbf534270 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41822/0 (socket says 172.21.15.37:41822) 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 -- 172.21.15.37:0/4268966808 learned_addr learned my addr 172.21.15.37:0/4268966808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf943140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eaf400640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf942c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 -- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 msgr2=0x560fbf943140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf943140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 -- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 msgr2=0x560fbf942c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf942c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eafe00640 1 -- 172.21.15.37:0/4268966808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbf9030b0 con 0x560fbf50b3b0 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.014+0000 7f7eaf400640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf942c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.015+0000 7f7eafe00640 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e5e0 tx=0x7f7ea000e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4348ebd6c3c43b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.015+0000 7f7e9b400640 1 -- 172.21.15.37:0/4268966808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0003d80 con 0x560fbf50b3b0 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.015+0000 7f7e9b400640 1 -- 172.21.15.37:0/4268966808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea0013420 con 0x560fbf50b3b0 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.015+0000 7f7e9b400640 1 -- 172.21.15.37:0/4268966808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00139c0 con 0x560fbf50b3b0 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 -- 172.21.15.37:0/4268966808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 msgr2=0x560fbf932720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e5e0 tx=0x7f7ea000e610 comp rx=0 tx=0).stop 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 -- 172.21.15.37:0/4268966808 shutdown_connections 2024-09-21T06:30:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf534270 0x560fbf943140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf513590 0x560fbf942c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/4268966808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf932720 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 -- 172.21.15.37:0/4268966808 >> 172.21.15.37:0/4268966808 conn(0x560fbf5283c0 msgr2=0x560fbf9200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 -- 172.21.15.37:0/4268966808 shutdown_connections 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.016+0000 7f7ebdc00740 1 -- 172.21.15.37:0/4268966808 wait complete. 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.017+0000 7f7ebdc00740 1 -- start start 2024-09-21T06:30:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.017+0000 7f7ebdc00740 1 Processor -- start 2024-09-21T06:30:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.018+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.018+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf513590 0x560fbf996460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7ebdc00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf941bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560fbf8fe030 con 0x560fbf50b3b0 2024-09-21T06:30:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560fbf9331c0 con 0x560fbf534270 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7ebdc00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560fbf8fe5f0 con 0x560fbf513590 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7ebdc00740 10 client.0 _ll_register_callbacks cb 0x7f7ea00053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41824/0 (socket says 172.21.15.37:41824) 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 -- 172.21.15.37:0/759994404 learned_addr learned my addr 172.21.15.37:0/759994404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf941bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eaf400640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf513590 0x560fbf996460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 -- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf513590 msgr2=0x560fbf996460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf513590 0x560fbf996460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 -- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 msgr2=0x560fbf941bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf941bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eafe00640 1 -- 172.21.15.37:0/759994404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbf99fd20 con 0x560fbf50b3b0 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.019+0000 7f7eb4a00640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf941bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7eafe00640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00039c0 tx=0x7f7ea0007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d2c249d0728bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0013b80 con 0x560fbf50b3b0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f7ea002f8a0 con 0x560fbf50b3b0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea0013420 con 0x560fbf50b3b0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7ea002faa0 con 0x560fbf50b3b0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7ebdc00740 10 client.5745 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x560fbf9969a0 con 0x560fbf50b3b0 2024-09-21T06:30:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea0013420 con 0x560fbf50b3b0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7f7ea003e030 con 0x560fbf50b3b0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7ebdc00740 10 client.5745 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.020+0000 7f7ebdc00740 10 client.5745 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.021+0000 7f7ebdc00740 20 client.5745 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.021+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf900dd0 0x560fbf9a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.021+0000 7f7ebdc00740 4 client.5745 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.021+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "status"}) -- 0x560fbf84bfd0 con 0x560fbf900dd0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.021+0000 7f7eaf400640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf900dd0 0x560fbf9a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7eaf400640 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf900dd0 0x560fbf9a3200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eaf3b6e0210b3fe3 server_cookie=8c7f3631b5835a77 in_seq=0 out_seq=0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7e98c00640 10 client.5745 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7e98c00640 1 -- 172.21.15.37:0/759994404 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560fbf84bfd0 con 0x560fbf900dd0 2024-09-21T06:30:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7e98c00640 10 client.5745 handle_command_reply: tid=0 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 client.5745 shutdown 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 msgr2=0x560fbf935710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00039c0 tx=0x7f7ea0007940 comp rx=0 tx=0).stop 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 shutdown_connections 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x560fbf900dd0 0x560fbf9a3200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbf534270 0x560fbf941bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fbf513590 0x560fbf996460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 --2- 172.21.15.37:0/759994404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560fbf50b3b0 0x560fbf935710 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7e98c00640 0 client.5745 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 >> 172.21.15.37:0/759994404 conn(0x560fbf5283c0 msgr2=0x560fbf8d6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 shutdown_connections 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.022+0000 7f7ebdc00740 1 -- 172.21.15.37:0/759994404 wait complete. 2024-09-21T06:30:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7ebdc00740 20 client.5745 trim_cache size 0 max 16384 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5500, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1271138409999999, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 38, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 56, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 56, 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.708447959000001 2024-09-21T06:30:44.028 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 msgr2=0x7f7e9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 0x7f7e9007b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000f60 tx=0x7f7ea0009040 comp rx=0 tx=0).stop 2024-09-21T06:30:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 msgr2=0x7f7eb01451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800b9a0 tx=0x7f7ea800be60 comp rx=0 tx=0).stop 2024-09-21T06:30:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 shutdown_connections 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e90078b90 0x7f7e9007b0a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb0141680 0x7f7eb01451a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb00cf6d0 0x7f7eb01e93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 --2- 172.21.15.37:0/2132041325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb00cc0e0 0x7f7eb0143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 >> 172.21.15.37:0/2132041325 conn(0x7f7eb00d05c0 msgr2=0x7f7eb015f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 shutdown_connections 2024-09-21T06:30:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.023+0000 7f7eb5400640 1 -- 172.21.15.37:0/2132041325 wait complete. 2024-09-21T06:30:44.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.272+0000 7fe1b6c00640 1 -- 172.21.15.37:0/540510252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe198004800 msgr2=0x7fe198004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.272+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/540510252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe198004800 0x7fe198004c10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00d2480 tx=0x7fe198005830 comp rx=0 tx=0).stop 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.273+0000 7fe1b6c00640 1 -- 172.21.15.37:0/540510252 shutdown_connections 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.273+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/540510252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe198004800 0x7fe198004c10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.273+0000 7fe1b6c00640 1 -- 172.21.15.37:0/540510252 >> 172.21.15.37:0/540510252 conn(0x7fe1b00d42e0 msgr2=0x7fe1b00d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.273+0000 7fe1b6c00640 1 -- 172.21.15.37:0/540510252 shutdown_connections 2024-09-21T06:30:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.273+0000 7fe1b6c00640 1 -- 172.21.15.37:0/540510252 wait complete. 2024-09-21T06:30:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.275+0000 7fe1b6c00640 1 Processor -- start 2024-09-21T06:30:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.275+0000 7fe1b6c00640 1 -- start start 2024-09-21T06:30:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.276+0000 7fe1b6c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 0x7fe1b0158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.276+0000 7fe1b6c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.276+0000 7fe1b6c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 0x7fe1b0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b6c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b00ba270 con 0x7fe1b01525e0 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b6c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b00ba0f0 con 0x7fe1b0160410 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b6c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b0162230 con 0x7fe198004800 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41838/0 (socket says 172.21.15.37:41838) 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 -- 172.21.15.37:0/4170270730 learned_addr learned my addr 172.21.15.37:0/4170270730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 0x7fe1b0158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b6200640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 0x7fe1b0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 -- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 msgr2=0x7fe1b0158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 0x7fe1b0158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 -- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 msgr2=0x7fe1b0160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 0x7fe1b0160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b4e00640 1 -- 172.21.15.37:0/4170270730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b0160d60 con 0x7fe1b01525e0 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b6200640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 0x7fe1b0160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.277+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 0x7fe1b0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe19c02a8d0 tx=0x7fe19c0063c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1ad600640 1 -- 172.21.15.37:0/4170270730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c003a00 con 0x7fe1b01525e0 2024-09-21T06:30:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1ad600640 1 -- 172.21.15.37:0/4170270730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19c039070 con 0x7fe1b01525e0 2024-09-21T06:30:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1ad600640 1 -- 172.21.15.37:0/4170270730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c033420 con 0x7fe1b01525e0 2024-09-21T06:30:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b0154eb0 con 0x7fe1b01525e0 2024-09-21T06:30:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.278+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b0155470 con 0x7fe1b01525e0 2024-09-21T06:30:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.281+0000 7fe1ad600640 1 -- 172.21.15.37:0/4170270730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe19c044020 con 0x7fe1b01525e0 2024-09-21T06:30:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.282+0000 7fe1ad600640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 0x7fe19007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.282+0000 7fe1ad600640 1 -- 172.21.15.37:0/4170270730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fe19c0ba9c0 con 0x7fe1b01525e0 2024-09-21T06:30:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.282+0000 7fe1bf2be740 1 Processor -- start 2024-09-21T06:30:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.282+0000 7fe1bf2be740 1 -- start start 2024-09-21T06:30:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.282+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 0x7fe19007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.283+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 0x7fe19007b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00521d0 tx=0x7fe1a4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf2828f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf28218430 con 0x55cf27eee270 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cf282085d0 con 0x55cf27ec53b0 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cf281f7a00 con 0x55cf27ecd590 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41852/0 (socket says 172.21.15.37:41852) 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 -- 172.21.15.37:0/4198405563 learned_addr learned my addr 172.21.15.37:0/4198405563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b6200640 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf2828f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 -- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 msgr2=0x55cf2828f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf2828f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 -- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 msgr2=0x55cf282bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.284+0000 7fe1b5800640 1 -- 172.21.15.37:0/4198405563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf281c0520 con 0x55cf27eee270 2024-09-21T06:30:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.285+0000 7fe1b5800640 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40093e0 tx=0x7fe1a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81448376ed15f3e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.285+0000 7fe1a2000640 1 -- 172.21.15.37:0/4198405563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a401c660 con 0x55cf27eee270 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.285+0000 7fe1a2000640 1 -- 172.21.15.37:0/4198405563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1a401c800 con 0x55cf27eee270 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.285+0000 7fe1a2000640 1 -- 172.21.15.37:0/4198405563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a401cbd0 con 0x55cf27eee270 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 -- 172.21.15.37:0/4198405563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 msgr2=0x55cf282beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40093e0 tx=0x7fe1a4016000 comp rx=0 tx=0).stop 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 -- 172.21.15.37:0/4198405563 shutdown_connections 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf2828f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/4198405563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282beb60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 -- 172.21.15.37:0/4198405563 >> 172.21.15.37:0/4198405563 conn(0x55cf27ee23c0 msgr2=0x55cf28294b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 -- 172.21.15.37:0/4198405563 shutdown_connections 2024-09-21T06:30:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.286+0000 7fe1bf2be740 1 -- 172.21.15.37:0/4198405563 wait complete. 2024-09-21T06:30:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.287+0000 7fe1bf2be740 1 -- start start 2024-09-21T06:30:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.287+0000 7fe1bf2be740 1 Processor -- start 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ecd590 0x55cf282daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf2819e330 con 0x55cf27eee270 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cf281fb810 con 0x55cf27ecd590 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cf27f68620 con 0x55cf27ec53b0 2024-09-21T06:30:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1bf2be740 10 client.0 _ll_register_callbacks cb 0x55cf281a3420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41858/0 (socket says 172.21.15.37:41858) 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b6200640 1 -- 172.21.15.37:0/2343317615 learned_addr learned my addr 172.21.15.37:0/2343317615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b5800640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ecd590 0x55cf282daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b6200640 1 -- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 msgr2=0x55cf282a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.289+0000 7fe1b6200640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1b6200640 1 -- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ecd590 msgr2=0x55cf282daa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1b6200640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ecd590 0x55cf282daa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1b6200640 1 -- 172.21.15.37:0/2343317615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf27f59850 con 0x55cf27eee270 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1b6200640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8002230 tx=0x7fe1a8012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff605db321d1c87a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe196a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8003d80 con 0x55cf27eee270 2024-09-21T06:30:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a401c800 con 0x55cf27eee270 2024-09-21T06:30:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fe1a8005110 con 0x55cf27eee270 2024-09-21T06:30:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe1a8022050 con 0x55cf27eee270 2024-09-21T06:30:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1bf2be740 10 client.5757 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.290+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55cf27fd06b0 con 0x55cf27eee270 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1a401c800 con 0x55cf27eee270 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7fe1a8025020 con 0x55cf27eee270 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 10 client.5757 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 10 client.5757 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 20 client.5757 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf27eace40 0x55cf282b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 4 client.5757 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2820a520 con 0x55cf27eace40 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.291+0000 7fe1b5800640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf27eace40 0x55cf282b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.292+0000 7fe1b5800640 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf27eace40 0x55cf282b7fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a7d140c296a8df0 server_cookie=7f8807de795aece0 in_seq=0 out_seq=0 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.292+0000 7fe196a00640 10 client.5757 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe196a00640 1 -- 172.21.15.37:0/2343317615 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf2820a520 con 0x55cf27eace40 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe196a00640 10 client.5757 handle_command_reply: tid=0 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 client.5757 shutdown 2024-09-21T06:30:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 msgr2=0x55cf282daf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8002230 tx=0x7fe1a8012870 comp rx=0 tx=0).stop 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 shutdown_connections 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf27eace40 0x55cf282b7fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282daf50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ecd590 0x55cf282daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2343317615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe196a00640 0 client.5757 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 >> 172.21.15.37:0/2343317615 conn(0x55cf27ee23c0 msgr2=0x55cf282b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 shutdown_connections 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2343317615 wait complete. 2024-09-21T06:30:44.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.293+0000 7fe1bf2be740 20 client.5757 trim_cache size 0 max 16384 2024-09-21T06:30:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.313+0000 7fe1bf2be740 1 Processor -- start 2024-09-21T06:30:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.313+0000 7fe1bf2be740 1 -- start start 2024-09-21T06:30:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 0x55cf282da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b0201b60 con 0x55cf27ecd590 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b0202230 con 0x55cf27eee270 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.314+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b01f8ff0 con 0x55cf27ec53b0 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41872/0 (socket says 172.21.15.37:41872) 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 -- 172.21.15.37:0/2572974406 learned_addr learned my addr 172.21.15.37:0/2572974406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b5800640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b6200640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 0x55cf282da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 -- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 msgr2=0x55cf282dc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282dc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 -- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 msgr2=0x55cf282da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 0x55cf282da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 -- 172.21.15.37:0/2572974406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf28295840 con 0x55cf27ecd590 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b6200640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 0x55cf282da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe19c082480 tx=0x7fe19c082110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cb9a2a90c1bda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1a1600640 1 -- 172.21.15.37:0/2572974406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c087020 con 0x55cf27ecd590 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1a1600640 1 -- 172.21.15.37:0/2572974406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19c09c020 con 0x55cf27ecd590 2024-09-21T06:30:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.315+0000 7fe1a1600640 1 -- 172.21.15.37:0/2572974406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c08bca0 con 0x55cf27ecd590 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2572974406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 msgr2=0x55cf282b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe19c082480 tx=0x7fe19c082110 comp rx=0 tx=0).stop 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2572974406 shutdown_connections 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27ecd590 0x55cf282b4a20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27eee270 0x55cf282da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/2572974406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ec53b0 0x55cf282dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2572974406 >> 172.21.15.37:0/2572974406 conn(0x55cf27ee23c0 msgr2=0x55cf282fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2572974406 shutdown_connections 2024-09-21T06:30:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.316+0000 7fe1bf2be740 1 -- 172.21.15.37:0/2572974406 wait complete. 2024-09-21T06:30:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.317+0000 7fe1bf2be740 1 -- start start 2024-09-21T06:30:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.318+0000 7fe1bf2be740 1 Processor -- start 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf282dac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf282ebf30 con 0x55cf27eee270 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cf282d4620 con 0x55cf27ec53b0 2024-09-21T06:30:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cf282ec4c0 con 0x55cf27ecd590 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1bf2be740 10 client.0 _ll_register_callbacks cb 0x7fe1a8005be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41884/0 (socket says 172.21.15.37:41884) 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1b6200640 1 -- 172.21.15.37:0/3770910379 learned_addr learned my addr 172.21.15.37:0/3770910379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1b4e00640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf282dac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.319+0000 7fe1b5800640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 -- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 msgr2=0x55cf282dac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf282dac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 -- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 msgr2=0x55cf282ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282ea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 -- 172.21.15.37:0/3770910379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2835c820 con 0x55cf27eee270 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe1b6200640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8005cf0 tx=0x7fe1a8012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9598404e2728a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe196a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8013510 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.320+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fe1a80136b0 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19c09c020 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe1a8036050 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 10 client.5763 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55cf282db160 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19c09c020 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7fe1a8039020 con 0x55cf27eee270 2024-09-21T06:30:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 10 client.5763 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 10 client.5763 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 20 client.5763 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf282d54d0 0x55cf282be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 4 client.5763 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "status"}) -- 0x55cf28205fd0 con 0x55cf282d54d0 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.321+0000 7fe1b5800640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf282d54d0 0x55cf282be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe1b5800640 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf282d54d0 0x55cf282be570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd6ac92f85d4d96d server_cookie=61cde0519f4e38a0 in_seq=0 out_seq=0 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe196a00640 10 client.5763 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe196a00640 1 -- 172.21.15.37:0/3770910379 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55cf28205fd0 con 0x55cf282d54d0 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe196a00640 10 client.5763 handle_command_reply: tid=0 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe1bf2be740 1 client.5763 shutdown 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 msgr2=0x55cf282a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.322+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8005cf0 tx=0x7fe1a8012f50 comp rx=0 tx=0).stop 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 shutdown_connections 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55cf282d54d0 0x55cf282be570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf27eee270 0x55cf282a4180 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cf27ecd590 0x55cf282dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 --2- 172.21.15.37:0/3770910379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf27ec53b0 0x55cf282ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe196a00640 0 client.5763 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 >> 172.21.15.37:0/3770910379 conn(0x55cf27ee23c0 msgr2=0x55cf282fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 shutdown_connections 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 1 -- 172.21.15.37:0/3770910379 wait complete. 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.323+0000 7fe1bf2be740 20 client.5763 trim_cache size 0 max 16384 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5506, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.44086253, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 38, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 56, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 56, 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 18.006319836999999 2024-09-21T06:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 msgr2=0x7fe19007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 0x7fe19007b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00521d0 tx=0x7fe1a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 msgr2=0x7fe1b015fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe19c02a8d0 tx=0x7fe19c0063c0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 shutdown_connections 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe190078b90 0x7fe19007b0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1b01525e0 0x7fe1b015fed0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b0160410 0x7fe1b0160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 --2- 172.21.15.37:0/4170270730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe198004800 0x7fe1b0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 >> 172.21.15.37:0/4170270730 conn(0x7fe1b00d42e0 msgr2=0x7fe1b00b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 shutdown_connections 2024-09-21T06:30:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.324+0000 7fe1b6c00640 1 -- 172.21.15.37:0/4170270730 wait complete. 2024-09-21T06:30:44.350 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 -- 172.21.15.37:0/3344863410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cc0e0 msgr2=0x7ff5d40702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 --2- 172.21.15.37:0/3344863410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40702a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005fd0 tx=0x7ff5c802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 -- 172.21.15.37:0/3344863410 shutdown_connections 2024-09-21T06:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 --2- 172.21.15.37:0/3344863410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d41605f0 0x7ff5d40b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 --2- 172.21.15.37:0/3344863410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40702a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 -- 172.21.15.37:0/3344863410 >> 172.21.15.37:0/3344863410 conn(0x7ff5d40d05c0 msgr2=0x7ff5d406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 -- 172.21.15.37:0/3344863410 shutdown_connections 2024-09-21T06:30:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.575+0000 7ff5da200640 1 -- 172.21.15.37:0/3344863410 wait complete. 2024-09-21T06:30:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.577+0000 7ff5da200640 1 Processor -- start 2024-09-21T06:30:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.577+0000 7ff5da200640 1 -- start start 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d415b390 0x7ff5d4159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d40b3c80 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d40b3b00 con 0x7ff5d40cc0e0 2024-09-21T06:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5da200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d40b3e00 con 0x7ff5d415b390 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41898/0 (socket says 172.21.15.37:41898) 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 -- 172.21.15.37:0/595599640 learned_addr learned my addr 172.21.15.37:0/595599640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d9800640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d415b390 0x7ff5d4159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 -- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d415b390 msgr2=0x7ff5d4159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d415b390 0x7ff5d4159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 -- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d40cc0e0 msgr2=0x7ff5d40c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.579+0000 7ff5d3e00640 1 -- 172.21.15.37:0/595599640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d4143aa0 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000c390 tx=0x7ff5c000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5d0c00640 1 -- 172.21.15.37:0/595599640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5d0c00640 1 -- 172.21.15.37:0/595599640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c00035f0 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5d0c00640 1 -- 172.21.15.37:0/595599640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0003a20 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d41f2c90 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.580+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41f3250 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.582+0000 7ff5d0c00640 1 -- 172.21.15.37:0/595599640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5c001a070 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.583+0000 7ff5d0c00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.583+0000 7ff5d0c00640 1 -- 172.21.15.37:0/595599640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff5c009a200 con 0x7ff5d40cf6d0 2024-09-21T06:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.584+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.584+0000 7ff5e2522740 1 Processor -- start 2024-09-21T06:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.584+0000 7ff5e2522740 1 -- start start 2024-09-21T06:30:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.585+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8033000 tx=0x7ff5c8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.585+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.585+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c27560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5e2522740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b12bac430 con 0x560b128593b0 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560b12b9c5d0 con 0x560b12882270 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560b12b8ba00 con 0x560b12861590 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41912/0 (socket says 172.21.15.37:41912) 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 -- 172.21.15.37:0/3018015372 learned_addr learned my addr 172.21.15.37:0/3018015372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 -- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 msgr2=0x560b12c27560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d9800640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c27560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c27560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 -- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 msgr2=0x560b12c56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 -- 172.21.15.37:0/3018015372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b12b54520 con 0x560b128593b0 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d9800640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c27560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.586+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0001250 tx=0x7ff5c005dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc431918c13fda87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.587+0000 7ff5c5600640 1 -- 172.21.15.37:0/3018015372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0062490 con 0x560b128593b0 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.587+0000 7ff5c5600640 1 -- 172.21.15.37:0/3018015372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c0061d90 con 0x560b128593b0 2024-09-21T06:30:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.587+0000 7ff5c5600640 1 -- 172.21.15.37:0/3018015372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0066870 con 0x560b128593b0 2024-09-21T06:30:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.587+0000 7ff5e2522740 1 -- 172.21.15.37:0/3018015372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 msgr2=0x560b12c574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.587+0000 7ff5e2522740 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0001250 tx=0x7ff5c005dd60 comp rx=0 tx=0).stop 2024-09-21T06:30:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 -- 172.21.15.37:0/3018015372 shutdown_connections 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c574b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c27560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 --2- 172.21.15.37:0/3018015372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 -- 172.21.15.37:0/3018015372 >> 172.21.15.37:0/3018015372 conn(0x560b128763c0 msgr2=0x560b12c2cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 -- 172.21.15.37:0/3018015372 shutdown_connections 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.588+0000 7ff5e2522740 1 -- 172.21.15.37:0/3018015372 wait complete. 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.589+0000 7ff5e2522740 1 -- start start 2024-09-21T06:30:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.589+0000 7ff5e2522740 1 Processor -- start 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b128593b0 0x560b12c6a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12c81ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b12b32330 con 0x560b12882270 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560b12b8f810 con 0x560b12861590 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560b128fc620 con 0x560b128593b0 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.590+0000 7ff5e2522740 10 client.0 _ll_register_callbacks cb 0x560b12b37420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41924/0 (socket says 172.21.15.37:41924) 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 -- 172.21.15.37:0/34340700 learned_addr learned my addr 172.21.15.37:0/34340700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b128593b0 0x560b12c6a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12c81ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 -- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b128593b0 msgr2=0x560b12c6a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b128593b0 0x560b12c6a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 -- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 msgr2=0x560b12c81ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12c81ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.591+0000 7ff5d9800640 1 -- 172.21.15.37:0/34340700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b128ed850 con 0x560b12882270 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5d9800640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc010710 tx=0x7ff5cc010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65c69dbe0e4a3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc004d20 con 0x560b12882270 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c0061d90 con 0x560b12882270 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff5cc00c8b0 con 0x560b12882270 2024-09-21T06:30:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff5cc023050 con 0x560b12882270 2024-09-21T06:30:44.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 10 client.5778 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:44.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x560b129646b0 con 0x560b12882270 2024-09-21T06:30:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c0061d90 con 0x560b12882270 2024-09-21T06:30:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7ff5cc026020 con 0x560b12882270 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 10 client.5778 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 10 client.5778 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.592+0000 7ff5e2522740 20 client.5778 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12840e40 0x560b12c84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5e2522740 4 client.5778 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b12b9e520 con 0x560b12840e40 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12840e40 0x560b12c84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5ba000640 10 client.5778 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.593+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12840e40 0x560b12c84c10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642dd6a86a19b390 server_cookie=c38a0e362b3116ac in_seq=0 out_seq=0 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5ba000640 1 -- 172.21.15.37:0/34340700 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b12b9e520 con 0x560b12840e40 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5ba000640 10 client.5778 handle_command_reply: tid=0 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 client.5778 shutdown 2024-09-21T06:30:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 msgr2=0x560b12c529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc010710 tx=0x7ff5cc010bd0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 shutdown_connections 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b12882270 0x560b12c529c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12840e40 0x560b12c84c10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12c81ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 --2- 172.21.15.37:0/34340700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b128593b0 0x560b12c6a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 >> 172.21.15.37:0/34340700 conn(0x560b128763c0 msgr2=0x560b12c68470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.594+0000 7ff5ba000640 0 client.5778 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.595+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 shutdown_connections 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.595+0000 7ff5e2522740 1 -- 172.21.15.37:0/34340700 wait complete. 2024-09-21T06:30:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.595+0000 7ff5e2522740 20 client.5778 trim_cache size 0 max 16384 2024-09-21T06:30:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.614+0000 7ff5e2522740 1 Processor -- start 2024-09-21T06:30:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.614+0000 7ff5e2522740 1 -- start start 2024-09-21T06:30:44.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c8b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41fd320 con 0x560b128593b0 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41fd9d0 con 0x560b12882270 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d4074010 con 0x560b12861590 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41930/0 (socket says 172.21.15.37:41930) 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4005129570 learned_addr learned my addr 172.21.15.37:0/4005129570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 msgr2=0x560b12c8b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c8b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d9800640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c8b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 msgr2=0x560b12c56c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4005129570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b12c52ce0 con 0x560b128593b0 2024-09-21T06:30:44.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c8b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.616+0000 7ff5d9800640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:44.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38d89b3946411e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5c4c00640 1 -- 172.21.15.37:0/4005129570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8045040 con 0x560b128593b0 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5c4c00640 1 -- 172.21.15.37:0/4005129570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c8004f60 con 0x560b128593b0 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5c4c00640 1 -- 172.21.15.37:0/4005129570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8035040 con 0x560b128593b0 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5e2522740 1 -- 172.21.15.37:0/4005129570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 msgr2=0x560b12c6a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.617+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c803c000 comp rx=0 tx=0).stop 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 -- 172.21.15.37:0/4005129570 shutdown_connections 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12882270 0x560b12c56c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12861590 0x560b12c8b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4005129570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c6a4a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 -- 172.21.15.37:0/4005129570 >> 172.21.15.37:0/4005129570 conn(0x560b128763c0 msgr2=0x560b12c22970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 -- 172.21.15.37:0/4005129570 shutdown_connections 2024-09-21T06:30:44.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.618+0000 7ff5e2522740 1 -- 172.21.15.37:0/4005129570 wait complete. 2024-09-21T06:30:44.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.619+0000 7ff5e2522740 1 -- start start 2024-09-21T06:30:44.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.619+0000 7ff5e2522740 1 Processor -- start 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12cf0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12882270 0x560b12c53690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b12c4f110 con 0x560b128593b0 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560b12c817f0 con 0x560b12861590 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560b12c4f700 con 0x560b12882270 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5e2522740 10 client.0 _ll_register_callbacks cb 0x7ff5cc009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:44.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5d8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5d8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41940/0 (socket says 172.21.15.37:41940) 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.620+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4044715939 learned_addr learned my addr 172.21.15.37:0/4044715939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12882270 msgr2=0x560b12c53690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12882270 0x560b12c53690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 msgr2=0x560b12cf0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12cf0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 -- 172.21.15.37:0/4044715939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b12c50aa0 con 0x560b128593b0 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5d8e00640 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80051f0 tx=0x7ff5c8004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cee45d14a9bc8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5ba000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:44.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c805d060 con 0x560b128593b0 2024-09-21T06:30:44.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff5c80037f0 con 0x560b128593b0 2024-09-21T06:30:44.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.621+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c8004f60 con 0x560b128593b0 2024-09-21T06:30:44.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff5c8045040 con 0x560b128593b0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 10 client.5784 fetch_fsmap learned FSMap version 38 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x560b12c52a20 con 0x560b128593b0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c8004f60 con 0x560b128593b0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 3542+0+0 (secure 0 0 0) 0x7ff5c803a040 con 0x560b128593b0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 10 client.5784 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 10 client.5784 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 20 client.5784 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12c6a330 0x560b12c7ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 4 client.5784 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "status"}) -- 0x560b12b99fd0 con 0x560b12c6a330 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.622+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12c6a330 0x560b12c7ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5d3e00640 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12c6a330 0x560b12c7ff70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b33e362ea5bc12c server_cookie=2182d61ca3631f69 in_seq=0 out_seq=0 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5ba000640 10 client.5784 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:44.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5ba000640 1 -- 172.21.15.37:0/4044715939 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560b12b99fd0 con 0x560b12c6a330 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5ba000640 10 client.5784 handle_command_reply: tid=0 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 client.5784 shutdown 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 msgr2=0x560b12c22100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80051f0 tx=0x7ff5c8004330 comp rx=0 tx=0).stop 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 shutdown_connections 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x560b12c6a330 0x560b12c7ff70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b12882270 0x560b12c53690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b12861590 0x560b12cf0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 --2- 172.21.15.37:0/4044715939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b128593b0 0x560b12c22100 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5ba000640 0 client.5784 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.623+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 >> 172.21.15.37:0/4044715939 conn(0x560b128763c0 msgr2=0x560b12c951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 shutdown_connections 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5e2522740 1 -- 172.21.15.37:0/4044715939 wait complete. 2024-09-21T06:30:44.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5e2522740 20 client.5784 trim_cache size 0 max 16384 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5574, 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 13.319085075, 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 38, 2024-09-21T06:30:44.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 56, 2024-09-21T06:30:44.630 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 56, 2024-09-21T06:30:44.630 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 18.307408128999999 2024-09-21T06:30:44.630 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:44.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 msgr2=0x7ff5b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8033000 tx=0x7ff5c8004730 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 msgr2=0x7ff5d4143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000c390 tx=0x7ff5c000c850 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 shutdown_connections 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5b4078b90 0x7ff5b407b0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5d40cf6d0 0x7ff5d4143560 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5d415b390 0x7ff5d4159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 --2- 172.21.15.37:0/595599640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d40cc0e0 0x7ff5d40c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.624+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 >> 172.21.15.37:0/595599640 conn(0x7ff5d40d05c0 msgr2=0x7ff5d41457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.625+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 shutdown_connections 2024-09-21T06:30:44.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.625+0000 7ff5da200640 1 -- 172.21.15.37:0/595599640 wait complete. 2024-09-21T06:30:44.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 -- 172.21.15.37:0/2887575537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb33006ddf0 msgr2=0x7fb33006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 --2- 172.21.15.37:0/2887575537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb33006ddf0 0x7fb33006e200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb33006f100 tx=0x7fb31802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 -- 172.21.15.37:0/2887575537 shutdown_connections 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 --2- 172.21.15.37:0/2887575537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb33006ddf0 0x7fb33006e200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 --2- 172.21.15.37:0/2887575537 >> v2:172.21.15.150:3300/0 conn(0x7fb3300ceb20 0x7fb330074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 -- 172.21.15.37:0/2887575537 >> 172.21.15.37:0/2887575537 conn(0x7fb330073400 msgr2=0x7fb330073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 -- 172.21.15.37:0/2887575537 shutdown_connections 2024-09-21T06:30:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.876+0000 7fb32f200640 1 -- 172.21.15.37:0/2887575537 wait complete. 2024-09-21T06:30:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.879+0000 7fb32f200640 1 Processor -- start 2024-09-21T06:30:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.879+0000 7fb32f200640 1 -- start start 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb33006ddf0 0x7fb33014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3300ceb20 0x7fb330160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3300ae8d0 con 0x7fb3300addb0 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3300cc7e0 con 0x7fb33006ddf0 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb32f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb3300aea50 con 0x7fb3300ceb20 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb327e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.880+0000 7fb327e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41958/0 (socket says 172.21.15.37:41958) 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 -- 172.21.15.37:0/1424428616 learned_addr learned my addr 172.21.15.37:0/1424428616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb32de00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb33006ddf0 0x7fb33014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb32e800640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3300ceb20 0x7fb330160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 -- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3300ceb20 msgr2=0x7fb330160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3300ceb20 0x7fb330160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 -- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb33006ddf0 msgr2=0x7fb33014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb33006ddf0 0x7fb33014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 -- 172.21.15.37:0/1424428616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb330161aa0 con 0x7fb3300addb0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.881+0000 7fb327e00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb318034040 tx=0x7fb3180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.882+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31803e030 con 0x7fb3300addb0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.882+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb318003db0 con 0x7fb3300addb0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.882+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318035040 con 0x7fb3300addb0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.882+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3300c6aa0 con 0x7fb3300addb0 2024-09-21T06:30:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.882+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3300c7060 con 0x7fb3300addb0 2024-09-21T06:30:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.883+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3300c6810 con 0x7fb3300addb0 2024-09-21T06:30:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.884+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb318041030 con 0x7fb3300addb0 2024-09-21T06:30:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.885+0000 7fb326000640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 0x7fb30c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.885+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fb3180bb9e0 con 0x7fb3300addb0 2024-09-21T06:30:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.885+0000 7fb32de00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 0x7fb30c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.886+0000 7fb32de00640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 0x7fb30c07b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b30 tx=0x7fb320005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:44.890+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb318083580 con 0x7fb3300addb0 2024-09-21T06:30:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.124+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33006e200 con 0x7fb3300addb0 2024-09-21T06:30:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.124+0000 7fb326000640 1 -- 172.21.15.37:0/1424428616 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5267 (secure 0 0 0) 0x7fb318087fb0 con 0x7fb3300addb0 2024-09-21T06:30:45.127 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:30:45.127 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":38,"btime":"2024-09-21T06:30:42:894731+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:30:42.894719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5500,5506,5574]},"id":3}]} 2024-09-21T06:30:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 38 2024-09-21T06:30:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 msgr2=0x7fb30c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 0x7fb30c07b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b30 tx=0x7fb320005f90 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 msgr2=0x7fb330160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb318034040 tx=0x7fb3180038a0 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 shutdown_connections 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb30c078b90 0x7fb30c07b0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3300addb0 0x7fb330160210 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3300ceb20 0x7fb330160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 --2- 172.21.15.37:0/1424428616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb33006ddf0 0x7fb33014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 >> 172.21.15.37:0/1424428616 conn(0x7fb330073400 msgr2=0x7fb330073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 shutdown_connections 2024-09-21T06:30:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:45.127+0000 7fb32f200640 1 -- 172.21.15.37:0/1424428616 wait complete. 2024-09-21T06:30:45.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:45.151 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:30:45.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:45.199 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:30:45.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:45.255 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:30:45.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:45.321 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:30:45.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:45.397 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:30:45.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:30:45.758 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:30:45.759 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:30:45.759 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:30:47.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:30:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:30:48.017 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:30:48.017 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:30:49.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:30:49.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:30:50.018 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:30:50.272 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:30:50.272 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:30:50.273 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:30:52.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:30:52.531 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:30:52.532 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:30:52.532 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:30:52.532 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:30:52.777 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:30:52.777 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:30:52.777 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:30:54.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:30:55.041 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:30:55.041 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:30:55.042 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:30:57.042 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:30:57.298 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T06:30:57.298 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:30:57.299 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:30:57.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:30:57.548 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:30:57.549 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:30:57.549 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:30:57.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 -- 172.21.15.37:0/480578751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7464156c60 msgr2=0x7f74640cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 --2- 172.21.15.37:0/480578751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7464156c60 0x7f74640cddc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f745c005fd0 tx=0x7f745c02a8a0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 -- 172.21.15.37:0/480578751 shutdown_connections 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 --2- 172.21.15.37:0/480578751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7464156c60 0x7f74640cddc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 --2- 172.21.15.37:0/480578751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640ccfc0 0x7f74640b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 -- 172.21.15.37:0/480578751 >> 172.21.15.37:0/480578751 conn(0x7f746406f920 msgr2=0x7f74640b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.779+0000 7f7463e00640 1 -- 172.21.15.37:0/480578751 shutdown_connections 2024-09-21T06:30:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.780+0000 7f7463e00640 1 -- 172.21.15.37:0/480578751 wait complete. 2024-09-21T06:30:57.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.782+0000 7f7463e00640 1 Processor -- start 2024-09-21T06:30:57.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.782+0000 7f7463e00640 1 -- start start 2024-09-21T06:30:57.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 0x7f74641f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640c7ea0 0x7f74640ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f746406d510 con 0x7f7464156c60 2024-09-21T06:30:57.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f746406d390 con 0x7f746406e370 2024-09-21T06:30:57.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.783+0000 7f7463e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f746406d690 con 0x7f74640c7ea0 2024-09-21T06:30:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51514/0 (socket says 172.21.15.37:51514) 2024-09-21T06:30:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 -- 172.21.15.37:0/3651161137 learned_addr learned my addr 172.21.15.37:0/3651161137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f7462a00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 0x7f74641f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f7463400640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640c7ea0 0x7f74640ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 -- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640c7ea0 msgr2=0x7f74640ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640c7ea0 0x7f74640ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 -- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 msgr2=0x7f74641f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 0x7f74641f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 -- 172.21.15.37:0/3651161137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74640c8680 con 0x7f7464156c60 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f7462a00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 0x7f74641f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:57.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.784+0000 7f745be00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f744c00e790 tx=0x7f744c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.785+0000 7f745aa00640 1 -- 172.21.15.37:0/3651161137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00eea0 con 0x7f7464156c60 2024-09-21T06:30:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.785+0000 7f745aa00640 1 -- 172.21.15.37:0/3651161137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f744c00cdb0 con 0x7f7464156c60 2024-09-21T06:30:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.785+0000 7f745aa00640 1 -- 172.21.15.37:0/3651161137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c010690 con 0x7f7464156c60 2024-09-21T06:30:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.785+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74640c8960 con 0x7f7464156c60 2024-09-21T06:30:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.785+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74640c8f20 con 0x7f7464156c60 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.787+0000 7f745aa00640 1 -- 172.21.15.37:0/3651161137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f744c010830 con 0x7f7464156c60 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.788+0000 7f745aa00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 0x7f744007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.788+0000 7f745aa00640 1 -- 172.21.15.37:0/3651161137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f744c09b880 con 0x7f7464156c60 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.789+0000 7f7462a00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 0x7f744007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.789+0000 7f7470c90740 1 Processor -- start 2024-09-21T06:30:57.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.789+0000 7f7470c90740 1 -- start start 2024-09-21T06:30:57.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7462a00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 0x7f744007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7454007f20 tx=0x7f7454007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 0x56167bdb4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bd97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56167bccd1c0 con 0x56167b99d4d0 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56167bce0870 con 0x56167b9c6390 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.790+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56167bccfca0 con 0x56167b9a56b0 2024-09-21T06:30:57.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 0x56167bdb4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51530/0 (socket says 172.21.15.37:51530) 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f745be00640 1 -- 172.21.15.37:0/2630733802 learned_addr learned my addr 172.21.15.37:0/2630733802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7463400640 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bd97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 -- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 msgr2=0x56167bd97940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bd97940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 -- 172.21.15.37:0/2630733802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 msgr2=0x56167bd97400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f7462a00640 1 -- 172.21.15.37:0/2630733802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167baa87d0 con 0x56167b9c6390 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.791+0000 7f745be00640 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.792+0000 7f7462a00640 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 0x56167bdb4730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f745400fd50 tx=0x7f7454012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668b0183c5f66bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.792+0000 7f7453400640 1 -- 172.21.15.37:0/2630733802 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745401b050 con 0x56167b9c6390 2024-09-21T06:30:57.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.792+0000 7f7453400640 1 -- 172.21.15.37:0/2630733802 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7454025070 con 0x56167b9c6390 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.792+0000 7f7453400640 1 -- 172.21.15.37:0/2630733802 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454020790 con 0x56167b9c6390 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 -- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 msgr2=0x56167bdb4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 0x56167bdb4730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f745400fd50 tx=0x7f7454012000 comp rx=0 tx=0).stop 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 -- 172.21.15.37:0/2630733802 shutdown_connections 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b99d4d0 0x56167bd97400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bd97940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 --2- 172.21.15.37:0/2630733802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9c6390 0x56167bdb4730 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 -- 172.21.15.37:0/2630733802 >> 172.21.15.37:0/2630733802 conn(0x56167b9ba4e0 msgr2=0x56167bd6d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 -- 172.21.15.37:0/2630733802 shutdown_connections 2024-09-21T06:30:57.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.793+0000 7f7470c90740 1 -- 172.21.15.37:0/2630733802 wait complete. 2024-09-21T06:30:57.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.794+0000 7f7470c90740 1 -- start start 2024-09-21T06:30:57.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.794+0000 7f7470c90740 1 Processor -- start 2024-09-21T06:30:57.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.795+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b99d4d0 0x56167bd943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9a56b0 0x56167bd85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56167bc76410 con 0x56167b9c6390 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56167bcd3ab0 con 0x56167b9a56b0 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56167ba40740 con 0x56167b99d4d0 2024-09-21T06:30:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7470c90740 10 client.0 _ll_register_callbacks cb 0x56167bc53360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51546/0 (socket says 172.21.15.37:51546) 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 -- 172.21.15.37:0/1150928440 learned_addr learned my addr 172.21.15.37:0/1150928440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f745be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9a56b0 0x56167bd85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7462a00640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b99d4d0 0x56167bd943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 -- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b99d4d0 msgr2=0x56167bd943f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b99d4d0 0x56167bd943f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 -- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9a56b0 msgr2=0x56167bd85f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9a56b0 0x56167bd85f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.796+0000 7f7463400640 1 -- 172.21.15.37:0/1150928440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167ba31970 con 0x56167b9c6390 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7463400640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f745c02ae90 tx=0x7f745c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc6e6e5f9fcea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7447e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c04a060 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7454025070 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f745c0045f0 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f745c050050 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7470c90740 10 client.5805 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56167ba0f6a0 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.797+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7454025070 con 0x56167b9c6390 2024-09-21T06:30:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f745c053020 con 0x56167b9c6390 2024-09-21T06:30:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 10 client.5805 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 10 client.5805 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:30:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 20 client.5805 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167b984f60 0x56167bd6cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 4 client.5805 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56167bce27c0 con 0x56167b984f60 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7462a00640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167b984f60 0x56167bd6cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7447e00640 10 client.5805 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.798+0000 7f7462a00640 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167b984f60 0x56167bd6cdd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67862e421676d783 server_cookie=82c07e5e4c54485 in_seq=0 out_seq=0 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.799+0000 7f7447e00640 1 -- 172.21.15.37:0/1150928440 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56167bce27c0 con 0x56167b984f60 2024-09-21T06:30:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.799+0000 7f7447e00640 10 client.5805 handle_command_reply: tid=0 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.799+0000 7f7470c90740 1 client.5805 shutdown 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.799+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 msgr2=0x56167bdc5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.799+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f745c02ae90 tx=0x7f745c004270 comp rx=0 tx=0).stop 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 shutdown_connections 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdc5150 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167b984f60 0x56167bd6cdd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b9a56b0 0x56167bd85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 --2- 172.21.15.37:0/1150928440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b99d4d0 0x56167bd943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7447e00640 0 client.5805 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 >> 172.21.15.37:0/1150928440 conn(0x56167b9ba4e0 msgr2=0x56167bd66830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 shutdown_connections 2024-09-21T06:30:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 1 -- 172.21.15.37:0/1150928440 wait complete. 2024-09-21T06:30:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.800+0000 7f7470c90740 20 client.5805 trim_cache size 0 max 16384 2024-09-21T06:30:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.816+0000 7f7470c90740 1 Processor -- start 2024-09-21T06:30:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.816+0000 7f7470c90740 1 -- start start 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bdd4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74642024e0 con 0x56167b9c6390 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74642016b0 con 0x56167b99d4d0 2024-09-21T06:30:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7464201db0 con 0x56167b9a56b0 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.817+0000 7f7463400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51550/0 (socket says 172.21.15.37:51550) 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 -- 172.21.15.37:0/897580356 learned_addr learned my addr 172.21.15.37:0/897580356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f745be00640 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bdd4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7462a00640 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 -- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 msgr2=0x56167bdd4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bdd4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 -- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 msgr2=0x56167bd975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 -- 172.21.15.37:0/897580356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167bda9640 con 0x56167b9c6390 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7463400640 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f745c004780 tx=0x7f745c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262b76b54686fde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7452a00640 1 -- 172.21.15.37:0/897580356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c031430 con 0x56167b9c6390 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7452a00640 1 -- 172.21.15.37:0/897580356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745c0315d0 con 0x56167b9c6390 2024-09-21T06:30:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.818+0000 7f7452a00640 1 -- 172.21.15.37:0/897580356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c031800 con 0x56167b9c6390 2024-09-21T06:30:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 -- 172.21.15.37:0/897580356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 msgr2=0x56167bdd5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f745c004780 tx=0x7f745c0047b0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 -- 172.21.15.37:0/897580356 shutdown_connections 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9c6390 0x56167bdd5450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9a56b0 0x56167bdd4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 --2- 172.21.15.37:0/897580356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 -- 172.21.15.37:0/897580356 >> 172.21.15.37:0/897580356 conn(0x56167b9ba4e0 msgr2=0x56167bdab610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 -- 172.21.15.37:0/897580356 shutdown_connections 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.819+0000 7f7470c90740 1 -- 172.21.15.37:0/897580356 wait complete. 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.820+0000 7f7470c90740 1 -- start start 2024-09-21T06:30:57.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.820+0000 7f7470c90740 1 Processor -- start 2024-09-21T06:30:57.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 0x56167be34df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56167bd676c0 con 0x56167b9a56b0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74640aa220 con 0x56167b99d4d0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56167bd67cb0 con 0x56167b9c6390 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f7470c90740 10 client.0 _ll_register_callbacks cb 0x7f745c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f745be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51554/0 (socket says 172.21.15.37:51554) 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.822+0000 7f745be00640 1 -- 172.21.15.37:0/720925954 learned_addr learned my addr 172.21.15.37:0/720925954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7463400640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 0x56167be34df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7462a00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 -- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 msgr2=0x56167be34df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 0x56167be34df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 -- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 msgr2=0x56167bd6cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd6cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 -- 172.21.15.37:0/720925954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167bd6e4a0 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7463400640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 0x56167be34df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7462a00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd6cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f745be00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c6a0 tx=0x7f744c0635f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5082f6f7ca3da4df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7447e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c010b20 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f744c07ad90 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c0315d0 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f744c010cc0 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7470c90740 10 client.5811 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.823+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56167be356b0 con 0x56167b9a56b0 2024-09-21T06:30:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745c0315d0 con 0x56167b9a56b0 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f744c06a0f0 con 0x56167b9a56b0 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 10 client.5811 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 10 client.5811 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 20 client.5811 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167bdb4420 0x56167bdaa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 4 client.5811 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56167bd92880 con 0x56167bdb4420 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.824+0000 7f7462a00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167bdb4420 0x56167bdaa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.825+0000 7f7462a00640 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167bdb4420 0x56167bdaa1a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e805fdee22b23fd server_cookie=de93d044fa50f7be in_seq=0 out_seq=0 2024-09-21T06:30:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.825+0000 7f7447e00640 10 client.5811 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7447e00640 1 -- 172.21.15.37:0/720925954 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56167bd92880 con 0x56167bdb4420 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7447e00640 10 client.5811 handle_command_reply: tid=0 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 client.5811 shutdown 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 msgr2=0x56167be1a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c6a0 tx=0x7f744c0635f0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 shutdown_connections 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56167b9a56b0 0x56167be1a8f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x56167bdb4420 0x56167bdaa1a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56167b9c6390 0x56167be34df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 --2- 172.21.15.37:0/720925954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56167b99d4d0 0x56167bd6cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7447e00640 0 client.5811 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.843+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 >> 172.21.15.37:0/720925954 conn(0x56167b9ba4e0 msgr2=0x56167bda9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.844+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 shutdown_connections 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.844+0000 7f7470c90740 1 -- 172.21.15.37:0/720925954 wait complete. 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.844+0000 7f7470c90740 20 client.5811 trim_cache size 0 max 16384 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:30:57.846 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.844+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 msgr2=0x7f744007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.844+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 0x7f744007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7454007f20 tx=0x7f7454007eb0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 msgr2=0x7f74640b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f744c00e790 tx=0x7f744c00ec50 comp rx=0 tx=0).stop 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 shutdown_connections 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7440078bc0 0x7f744007b0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7464156c60 0x7f74640b1a30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74640c7ea0 0x7f74640ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 --2- 172.21.15.37:0/3651161137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f746406e370 0x7f74641f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 >> 172.21.15.37:0/3651161137 conn(0x7f746406f920 msgr2=0x7f746415c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 shutdown_connections 2024-09-21T06:30:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:57.845+0000 7f7463e00640 1 -- 172.21.15.37:0/3651161137 wait complete. 2024-09-21T06:30:57.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2024-09-21T06:30:58.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 -- 172.21.15.37:0/1354741063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9040b5bd0 msgr2=0x7ff90415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 --2- 172.21.15.37:0/1354741063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9040b5bd0 0x7ff90415ead0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009860 tx=0x7ff8f402a510 comp rx=0 tx=0).stop 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 -- 172.21.15.37:0/1354741063 shutdown_connections 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 --2- 172.21.15.37:0/1354741063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9040b5bd0 0x7ff90415ead0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 -- 172.21.15.37:0/1354741063 >> 172.21.15.37:0/1354741063 conn(0x7ff9040b46e0 msgr2=0x7ff9040d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 -- 172.21.15.37:0/1354741063 shutdown_connections 2024-09-21T06:30:58.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.101+0000 7ff909600640 1 -- 172.21.15.37:0/1354741063 wait complete. 2024-09-21T06:30:58.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.103+0000 7ff909600640 1 Processor -- start 2024-09-21T06:30:58.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.103+0000 7ff909600640 1 -- start start 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9040b5bd0 0x7ff904151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff904162710 0x7ff904158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040ce0e0 con 0x7ff904163230 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040cdf60 con 0x7ff9040b5bd0 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff909600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040ce260 con 0x7ff904162710 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51562/0 (socket says 172.21.15.37:51562) 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff903400640 1 -- 172.21.15.37:0/3754725614 learned_addr learned my addr 172.21.15.37:0/3754725614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff902a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9040b5bd0 0x7ff904151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.105+0000 7ff8fa000640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff904162710 0x7ff904158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 -- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff904162710 msgr2=0x7ff904158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff904162710 0x7ff904158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 -- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9040b5bd0 msgr2=0x7ff904151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9040b5bd0 0x7ff904151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 -- 172.21.15.37:0/3754725614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9041f6580 con 0x7ff904163230 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff903400640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cb10 tx=0x7ff8fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff8fb400640 1 -- 172.21.15.37:0/3754725614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc016020 con 0x7ff904163230 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff8fb400640 1 -- 172.21.15.37:0/3754725614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8fc00ad80 con 0x7ff904163230 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff8fb400640 1 -- 172.21.15.37:0/3754725614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc012680 con 0x7ff904163230 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.106+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff904158890 con 0x7ff904163230 2024-09-21T06:30:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.107+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041f5aa0 con 0x7ff904163230 2024-09-21T06:30:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.109+0000 7ff8fb400640 1 -- 172.21.15.37:0/3754725614 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8fc019020 con 0x7ff904163230 2024-09-21T06:30:58.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.110+0000 7ff8fb400640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 0x7ff8e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.110+0000 7ff8fb400640 1 -- 172.21.15.37:0/3754725614 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff8fc09b450 con 0x7ff904163230 2024-09-21T06:30:58.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.110+0000 7ff902a00640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 0x7ff8e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.110+0000 7ff9107e0740 1 Processor -- start 2024-09-21T06:30:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.111+0000 7ff9107e0740 1 -- start start 2024-09-21T06:30:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.111+0000 7ff902a00640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 0x7ff8e007b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4034000 tx=0x7ff8f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c7231c0 con 0x55ca6c41c390 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c736870 con 0x55ca6c3f34d0 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55ca6c725ca0 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff902a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff902a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51570/0 (socket says 172.21.15.37:51570) 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff902a00640 1 -- 172.21.15.37:0/1687263977 learned_addr learned my addr 172.21.15.37:0/1687263977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff903400640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.112+0000 7ff8fa000640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 -- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 msgr2=0x55ca6c7ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 -- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 msgr2=0x55ca6c7ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 -- 172.21.15.37:0/1687263977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6c4fe7d0 con 0x55ca6c41c390 2024-09-21T06:30:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff903400640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff902a00640 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009950 tx=0x7ff8f403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae26e2303d2191f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff8f3000640 1 -- 172.21.15.37:0/1687263977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40039f0 con 0x55ca6c41c390 2024-09-21T06:30:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff8f3000640 1 -- 172.21.15.37:0/1687263977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f4003c80 con 0x55ca6c41c390 2024-09-21T06:30:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.113+0000 7ff8f3000640 1 -- 172.21.15.37:0/1687263977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40506a0 con 0x55ca6c41c390 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 -- 172.21.15.37:0/1687263977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 msgr2=0x55ca6c80a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009950 tx=0x7ff8f403e040 comp rx=0 tx=0).stop 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 -- 172.21.15.37:0/1687263977 shutdown_connections 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 --2- 172.21.15.37:0/1687263977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c80a550 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 -- 172.21.15.37:0/1687263977 >> 172.21.15.37:0/1687263977 conn(0x55ca6c4104e0 msgr2=0x55ca6c7c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 -- 172.21.15.37:0/1687263977 shutdown_connections 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.114+0000 7ff9107e0740 1 -- 172.21.15.37:0/1687263977 wait complete. 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.115+0000 7ff9107e0740 1 -- start start 2024-09-21T06:30:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.116+0000 7ff9107e0740 1 Processor -- start 2024-09-21T06:30:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c6cc410 con 0x55ca6c41c390 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c729ab0 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55ca6c496740 con 0x55ca6c3f34d0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff9107e0740 10 client.0 _ll_register_callbacks cb 0x55ca6c6a9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff902a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff903400640 1 -- 172.21.15.37:0/3082769054 learned_addr learned my addr 172.21.15.37:0/3082769054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.117+0000 7ff8fa000640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 -- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3fb6b0 msgr2=0x55ca6c7e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 -- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 msgr2=0x55ca6c7e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 -- 172.21.15.37:0/3082769054 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6c487970 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff903400640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff902a00640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7e9f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8f404f000 tx=0x7ff8f4003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9841caa12db79ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff8e7400640 10 client.0 ms_handle_connect on v2:172.21.15.150:3301/0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4050950 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f4003c80 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.118+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mon.2 v2:172.21.15.150:3301/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff8f4050af0 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff8f4061050 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 10 client.5414 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55ca6c4656a0 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mon.2 v2:172.21.15.150:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f4003c80 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mon.2 v2:172.21.15.150:3301/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7ff8f4064020 con 0x55ca6c3f34d0 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 10 client.5414 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 10 client.5414 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:30:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 20 client.5414 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c3daf60 0x55ca6c816bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 4 client.5414 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca6c7387c0 con 0x55ca6c3daf60 2024-09-21T06:30:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.119+0000 7ff8fa000640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c3daf60 0x55ca6c816bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.120+0000 7ff8e7400640 10 client.5414 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.120+0000 7ff8fa000640 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c3daf60 0x55ca6c816bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4a6a41f5e2a040a server_cookie=202e57db845b5c42 in_seq=0 out_seq=0 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff8e7400640 1 -- 172.21.15.37:0/3082769054 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca6c7387c0 con 0x55ca6c3daf60 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff8e7400640 10 client.5414 handle_command_reply: tid=0 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 client.5414 shutdown 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 msgr2=0x55ca6c7e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7e9f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8f404f000 tx=0x7ff8f4003ec0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 shutdown_connections 2024-09-21T06:30:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c3daf60 0x55ca6c816bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 --2- 172.21.15.37:0/3082769054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7e9f30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 >> 172.21.15.37:0/3082769054 conn(0x55ca6c4104e0 msgr2=0x55ca6c7ed220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff8e7400640 0 client.5414 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 shutdown_connections 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.121+0000 7ff9107e0740 1 -- 172.21.15.37:0/3082769054 wait complete. 2024-09-21T06:30:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.122+0000 7ff9107e0740 20 client.5414 trim_cache size 0 max 16384 2024-09-21T06:30:58.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.137+0000 7ff9107e0740 1 Processor -- start 2024-09-21T06:30:58.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.137+0000 7ff9107e0740 1 -- start start 2024-09-21T06:30:58.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c8020b0 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9042027e0 con 0x55ca6c41c390 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff9042019b0 con 0x55ca6c3f34d0 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51586/0 (socket says 172.21.15.37:51586) 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 -- 172.21.15.37:0/2600307712 learned_addr learned my addr 172.21.15.37:0/2600307712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff903400640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff902a00640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 -- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 msgr2=0x55ca6c7ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 -- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 msgr2=0x55ca6c7c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff8fa000640 1 -- 172.21.15.37:0/2600307712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6c7bd220 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.139+0000 7ff903400640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.140+0000 7ff8fa000640 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff9040b1e70 tx=0x7ff8ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe31aaf6e4dba746 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.140+0000 7ff8f2600640 1 -- 172.21.15.37:0/2600307712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec001c30 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.140+0000 7ff8f2600640 1 -- 172.21.15.37:0/2600307712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8ec016e50 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.140+0000 7ff8f2600640 1 -- 172.21.15.37:0/2600307712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec005350 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 -- 172.21.15.37:0/2600307712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 msgr2=0x55ca6c7ed760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff9040b1e70 tx=0x7ff8ec009040 comp rx=0 tx=0).stop 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 -- 172.21.15.37:0/2600307712 shutdown_connections 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c3fb6b0 0x55ca6c7ed760 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2600307712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3f34d0 0x55ca6c7ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 -- 172.21.15.37:0/2600307712 >> 172.21.15.37:0/2600307712 conn(0x55ca6c4104e0 msgr2=0x55ca6c7bdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 -- 172.21.15.37:0/2600307712 shutdown_connections 2024-09-21T06:30:58.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.141+0000 7ff9107e0740 1 -- 172.21.15.37:0/2600307712 wait complete. 2024-09-21T06:30:58.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.142+0000 7ff9107e0740 1 -- start start 2024-09-21T06:30:58.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.142+0000 7ff9107e0740 1 Processor -- start 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c86ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff904153210 con 0x55ca6c41c390 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9042020b0 con 0x55ca6c3f34d0 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55ca6c86f820 con 0x55ca6c3fb6b0 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.143+0000 7ff9107e0740 10 client.0 _ll_register_callbacks cb 0x7ff8ec001210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51596/0 (socket says 172.21.15.37:51596) 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 -- 172.21.15.37:0/2572491798 learned_addr learned my addr 172.21.15.37:0/2572491798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff902a00640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c86ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 -- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 msgr2=0x55ca6c7e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 -- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 msgr2=0x55ca6c86ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c86ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 -- 172.21.15.37:0/2572491798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6c86f9f0 con 0x55ca6c41c390 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff903400640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc099700 tx=0x7ff8fc062c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2e11a7625b26af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.144+0000 7ff8e7400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc099d50 con 0x55ca6c41c390 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff8fc07ecf0 con 0x55ca6c41c390 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec016e50 con 0x55ca6c41c390 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff8fc089360 con 0x55ca6c41c390 2024-09-21T06:30:58.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 10 client.5829 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55ca6c81a4d0 con 0x55ca6c41c390 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8ec016e50 con 0x55ca6c41c390 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7ff8fc0622e0 con 0x55ca6c41c390 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 10 client.5829 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 10 client.5829 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 20 client.5829 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c7c4660 0x55ca6c809ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 4 client.5829 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.145+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca6c734270 con 0x55ca6c7c4660 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.146+0000 7ff902a00640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c7c4660 0x55ca6c809ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.146+0000 7ff902a00640 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c7c4660 0x55ca6c809ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0f45a5830dfe823 server_cookie=1410f8427bf4b18 in_seq=0 out_seq=0 2024-09-21T06:30:58.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.146+0000 7ff8e7400640 10 client.5829 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff8e7400640 1 -- 172.21.15.37:0/2572491798 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca6c734270 con 0x55ca6c7c4660 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff8e7400640 10 client.5829 handle_command_reply: tid=0 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 client.5829 shutdown 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 msgr2=0x55ca6c7c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc099700 tx=0x7ff8fc062c60 comp rx=0 tx=0).stop 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 shutdown_connections 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55ca6c7c4660 0x55ca6c809ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca6c41c390 0x55ca6c7c4120 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ca6c3fb6b0 0x55ca6c7e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff8e7400640 0 client.5829 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 --2- 172.21.15.37:0/2572491798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6c3f34d0 0x55ca6c86ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 >> 172.21.15.37:0/2572491798 conn(0x55ca6c4104e0 msgr2=0x55ca6c803930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 shutdown_connections 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 1 -- 172.21.15.37:0/2572491798 wait complete. 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.157+0000 7ff9107e0740 20 client.5829 trim_cache size 0 max 16384 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:30:58.160 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 msgr2=0x7ff8e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 0x7ff8e007b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4034000 tx=0x7ff8f4004730 comp rx=0 tx=0).stop 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 msgr2=0x7ff904161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cb10 tx=0x7ff8fc00a060 comp rx=0 tx=0).stop 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 shutdown_connections 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8e0078bc0 0x7ff8e007b0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff904163230 0x7ff904161400 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff904162710 0x7ff904158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 --2- 172.21.15.37:0/3754725614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9040b5bd0 0x7ff904151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 >> 172.21.15.37:0/3754725614 conn(0x7ff9040b46e0 msgr2=0x7ff90415df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 shutdown_connections 2024-09-21T06:30:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.158+0000 7ff909600640 1 -- 172.21.15.37:0/3754725614 wait complete. 2024-09-21T06:30:58.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2024-09-21T06:30:58.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 -- 172.21.15.37:0/3152623943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73100cc8d0 msgr2=0x7f73100ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 --2- 172.21.15.37:0/3152623943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73100cc8d0 0x7f73100ccce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7308005fd0 tx=0x7f730802aa90 comp rx=0 tx=0).stop 2024-09-21T06:30:58.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 -- 172.21.15.37:0/3152623943 shutdown_connections 2024-09-21T06:30:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 --2- 172.21.15.37:0/3152623943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73100cc8d0 0x7f73100ccce0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 --2- 172.21.15.37:0/3152623943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100b4f90 0x7f731006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 -- 172.21.15.37:0/3152623943 >> 172.21.15.37:0/3152623943 conn(0x7f731006f920 msgr2=0x7f73100b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 -- 172.21.15.37:0/3152623943 shutdown_connections 2024-09-21T06:30:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.412+0000 7f7314a00640 1 -- 172.21.15.37:0/3152623943 wait complete. 2024-09-21T06:30:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.414+0000 7f7314a00640 1 Processor -- start 2024-09-21T06:30:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.414+0000 7f7314a00640 1 -- start start 2024-09-21T06:30:58.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73100b4f90 0x7f731015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 0x7f73101f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73100ce150 con 0x7f73101456e0 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73100cdfd0 con 0x7f73100b4f90 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f7314a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f73100ce2d0 con 0x7f73100cc8d0 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51612/0 (socket says 172.21.15.37:51612) 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 -- 172.21.15.37:0/2549694979 learned_addr learned my addr 172.21.15.37:0/2549694979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730f400640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73100b4f90 0x7f731015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730ea00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 0x7f73101f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 -- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 msgr2=0x7f73101f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 0x7f73101f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 -- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73100b4f90 msgr2=0x7f731015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73100b4f90 0x7f731015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730fe00640 1 -- 172.21.15.37:0/2549694979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731015c210 con 0x7f73101456e0 2024-09-21T06:30:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.416+0000 7f730ea00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 0x7f73101f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f730fe00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f7308003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f7307400640 1 -- 172.21.15.37:0/2549694979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308044070 con 0x7f73101456e0 2024-09-21T06:30:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f7307400640 1 -- 172.21.15.37:0/2549694979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7308004040 con 0x7f73101456e0 2024-09-21T06:30:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73100a95e0 con 0x7f73101456e0 2024-09-21T06:30:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f7307400640 1 -- 172.21.15.37:0/2549694979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004430 con 0x7f73101456e0 2024-09-21T06:30:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.417+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73100a9ba0 con 0x7f73101456e0 2024-09-21T06:30:58.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.420+0000 7f7307400640 1 -- 172.21.15.37:0/2549694979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7308047070 con 0x7f73101456e0 2024-09-21T06:30:58.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.421+0000 7f7307400640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 0x7f72f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.421+0000 7f7307400640 1 -- 172.21.15.37:0/2549694979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f73080bbd70 con 0x7f73101456e0 2024-09-21T06:30:58.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.422+0000 7f731cfd1740 1 Processor -- start 2024-09-21T06:30:58.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.422+0000 7f730f400640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 0x7f72f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.422+0000 7f731cfd1740 1 -- start start 2024-09-21T06:30:58.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f730f400640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 0x7f72f007b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7300001720 tx=0x7f7300009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e58f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 0x56424e572940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56424e4a81c0 con 0x56424e1784d0 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56424e4bb870 con 0x56424e1a1390 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.423+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56424e4aaca0 con 0x56424e1806b0 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e58f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51624/0 (socket says 172.21.15.37:51624) 2024-09-21T06:30:58.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 -- 172.21.15.37:0/3908089504 learned_addr learned my addr 172.21.15.37:0/3908089504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730fe00640 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 0x56424e572940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 -- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 msgr2=0x56424e572940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 0x56424e572940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 -- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 msgr2=0x56424e58f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e58f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730ea00640 1 -- 172.21.15.37:0/3908089504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424e2837d0 con 0x56424e1784d0 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.424+0000 7f730fe00640 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 0x56424e572940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.425+0000 7f730ea00640 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009c30 tx=0x7f72fc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2b3a2dc147e11d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.425+0000 7f72fb400640 1 -- 172.21.15.37:0/3908089504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc004030 con 0x56424e1784d0 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.425+0000 7f72fb400640 1 -- 172.21.15.37:0/3908089504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72fc016070 con 0x56424e1784d0 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.425+0000 7f72fb400640 1 -- 172.21.15.37:0/3908089504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0035f0 con 0x56424e1784d0 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 -- 172.21.15.37:0/3908089504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 msgr2=0x56424e572400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009c30 tx=0x7f72fc0117b0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 -- 172.21.15.37:0/3908089504 shutdown_connections 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1784d0 0x56424e572400 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1806b0 0x56424e572940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 --2- 172.21.15.37:0/3908089504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e58f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 -- 172.21.15.37:0/3908089504 >> 172.21.15.37:0/3908089504 conn(0x56424e1954e0 msgr2=0x56424e548620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 -- 172.21.15.37:0/3908089504 shutdown_connections 2024-09-21T06:30:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.426+0000 7f731cfd1740 1 -- 172.21.15.37:0/3908089504 wait complete. 2024-09-21T06:30:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.427+0000 7f731cfd1740 1 -- start start 2024-09-21T06:30:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.427+0000 7f731cfd1740 1 Processor -- start 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1784d0 0x56424e56f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1a1390 0x56424e5856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56424e451410 con 0x56424e1806b0 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56424e4aeab0 con 0x56424e1784d0 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56424e21b740 con 0x56424e1a1390 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f731cfd1740 10 client.0 _ll_register_callbacks cb 0x56424e42e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51636/0 (socket says 172.21.15.37:51636) 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 -- 172.21.15.37:0/315322182 learned_addr learned my addr 172.21.15.37:0/315322182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730f400640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1784d0 0x56424e56f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730fe00640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1a1390 0x56424e5856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 -- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1a1390 msgr2=0x56424e5856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1a1390 0x56424e5856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 -- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1784d0 msgr2=0x56424e56f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1784d0 0x56424e56f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.429+0000 7f730ea00640 1 -- 172.21.15.37:0/315322182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424e20c970 con 0x56424e1806b0 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f730ea00640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f72fc000c00 tx=0x7f72fc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee52965126cf720 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f72efe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:58.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc029060 con 0x56424e1806b0 2024-09-21T06:30:58.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72fc016070 con 0x56424e1806b0 2024-09-21T06:30:58.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f72fc0035f0 con 0x56424e1806b0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f72fc02f050 con 0x56424e1806b0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f731cfd1740 10 client.5841 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.430+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56424e1ea6a0 con 0x56424e1806b0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72fc016070 con 0x56424e1806b0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f72fc032020 con 0x56424e1806b0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 10 client.5841 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 10 client.5841 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 20 client.5841 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e15ff60 0x56424e56e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 4 client.5841 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.431+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56424e4bd7c0 con 0x56424e15ff60 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.432+0000 7f730f400640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e15ff60 0x56424e56e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.432+0000 7f730f400640 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e15ff60 0x56424e56e1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=863be908f5934cd6 server_cookie=78cd55e3fe51091 in_seq=0 out_seq=0 2024-09-21T06:30:58.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.432+0000 7f72efe00640 10 client.5841 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:58.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.433+0000 7f72efe00640 1 -- 172.21.15.37:0/315322182 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56424e4bd7c0 con 0x56424e15ff60 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.433+0000 7f72efe00640 10 client.5841 handle_command_reply: tid=0 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.433+0000 7f731cfd1740 1 client.5841 shutdown 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.433+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 msgr2=0x56424e5869e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.433+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f72fc000c00 tx=0x7f72fc0042e0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 shutdown_connections 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e15ff60 0x56424e56e1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e5869e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1a1390 0x56424e5856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 --2- 172.21.15.37:0/315322182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1784d0 0x56424e56f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 >> 172.21.15.37:0/315322182 conn(0x56424e1954e0 msgr2=0x56424e58db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f72efe00640 0 client.5841 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 shutdown_connections 2024-09-21T06:30:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 1 -- 172.21.15.37:0/315322182 wait complete. 2024-09-21T06:30:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.434+0000 7f731cfd1740 20 client.5841 trim_cache size 0 max 16384 2024-09-21T06:30:58.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.450+0000 7f731cfd1740 1 Processor -- start 2024-09-21T06:30:58.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.450+0000 7f731cfd1740 1 -- start start 2024-09-21T06:30:58.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e58e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e549840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56424e541f10 con 0x56424e1806b0 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7310200470 con 0x56424e1a1390 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f73101ff640 con 0x56424e1784d0 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51644/0 (socket says 172.21.15.37:51644) 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f730ea00640 1 -- 172.21.15.37:0/268227174 learned_addr learned my addr 172.21.15.37:0/268227174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.451+0000 7f730f400640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e58e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730fe00640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e549840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 -- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 msgr2=0x56424e58e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e58e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 -- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 msgr2=0x56424e549840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e549840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 -- 172.21.15.37:0/268227174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424e545180 con 0x56424e1806b0 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730f400640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e58e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f730ea00640 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72fc004180 tx=0x7f72fc003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1decb38cc64d6e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f72faa00640 1 -- 172.21.15.37:0/268227174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc029020 con 0x56424e1806b0 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f72faa00640 1 -- 172.21.15.37:0/268227174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72fc016070 con 0x56424e1806b0 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.452+0000 7f72faa00640 1 -- 172.21.15.37:0/268227174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc003ba0 con 0x56424e1806b0 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 -- 172.21.15.37:0/268227174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 msgr2=0x56424e549300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72fc004180 tx=0x7f72fc003940 comp rx=0 tx=0).stop 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 -- 172.21.15.37:0/268227174 shutdown_connections 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e549300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e549840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 --2- 172.21.15.37:0/268227174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e58e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 -- 172.21.15.37:0/268227174 >> 172.21.15.37:0/268227174 conn(0x56424e1954e0 msgr2=0x56424e572400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 -- 172.21.15.37:0/268227174 shutdown_connections 2024-09-21T06:30:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.453+0000 7f731cfd1740 1 -- 172.21.15.37:0/268227174 wait complete. 2024-09-21T06:30:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.454+0000 7f731cfd1740 1 -- start start 2024-09-21T06:30:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.454+0000 7f731cfd1740 1 Processor -- start 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e603060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e59e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73100aacb0 con 0x56424e1806b0 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ffd40 con 0x56424e1a1390 2024-09-21T06:30:58.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56424e544c00 con 0x56424e1784d0 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f731cfd1740 10 client.0 _ll_register_callbacks cb 0x7f7300017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51648/0 (socket says 172.21.15.37:51648) 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 -- 172.21.15.37:0/926178228 learned_addr learned my addr 172.21.15.37:0/926178228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730fe00640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e59e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730f400640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e603060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 -- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 msgr2=0x56424e603060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e603060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 -- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 msgr2=0x56424e59e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.456+0000 7f730ea00640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e59e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f730ea00640 1 -- 172.21.15.37:0/926178228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424e60d230 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f730ea00640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72fc01c040 tx=0x7f72fc012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b1270a203326af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f72efe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f72fc01f040 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72fc016070 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f72fc042050 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f731cfd1740 10 client.5847 fetch_fsmap learned FSMap version 40 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.457+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56424e58f410 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72fc016070 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f72fc045020 con 0x56424e1806b0 2024-09-21T06:30:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 10 client.5847 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 10 client.5847 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 20 client.5847 populate_metadata read hostname 'smithi037' 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e5867a0 0x56424e59ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 4 client.5847 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56424e544580 con 0x56424e5867a0 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.458+0000 7f730f400640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e5867a0 0x56424e59ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.459+0000 7f730f400640 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e5867a0 0x56424e59ecb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f201f6e4f96151 server_cookie=1fe50ac36cb996a in_seq=0 out_seq=0 2024-09-21T06:30:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.459+0000 7f72efe00640 10 client.5847 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:58.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f72efe00640 1 -- 172.21.15.37:0/926178228 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56424e544580 con 0x56424e5867a0 2024-09-21T06:30:58.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f72efe00640 10 client.5847 handle_command_reply: tid=0 2024-09-21T06:30:58.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 client.5847 shutdown 2024-09-21T06:30:58.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 msgr2=0x56424e58eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72fc01c040 tx=0x7f72fc012680 comp rx=0 tx=0).stop 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 shutdown_connections 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x56424e5867a0 0x56424e59ecb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f72efe00640 0 client.5847 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56424e1806b0 0x56424e58eed0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56424e1a1390 0x56424e59e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 --2- 172.21.15.37:0/926178228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56424e1784d0 0x56424e603060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 >> 172.21.15.37:0/926178228 conn(0x56424e1954e0 msgr2=0x56424e59fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 shutdown_connections 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 1 -- 172.21.15.37:0/926178228 wait complete. 2024-09-21T06:30:58.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.472+0000 7f731cfd1740 20 client.5847 trim_cache size 0 max 16384 2024-09-21T06:30:58.475 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:30:58.475 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:30:58.475 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:30:58.475 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 msgr2=0x7f72f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 0x7f72f007b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7300001720 tx=0x7f7300009040 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 msgr2=0x7f73101f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f7308003ce0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 shutdown_connections 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f72f0078b90 0x7f72f007b0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73101456e0 0x7f73101f72f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73100cc8d0 0x7f73101f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 --2- 172.21.15.37:0/2549694979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73100b4f90 0x7f731015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:58.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 >> 172.21.15.37:0/2549694979 conn(0x7f731006f920 msgr2=0x7f73100b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:58.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 shutdown_connections 2024-09-21T06:30:58.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58.473+0000 7f7314a00640 1 -- 172.21.15.37:0/2549694979 wait complete. 2024-09-21T06:30:58.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:30:58.495 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/ 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-21T06:30:58.539 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-21T06:30:58.540 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:58.540 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:58.540 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:58.540 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:58.541 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-21T06:30:58.541 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:58.681 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-09-21T06:30:58.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-21T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:58.832 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-09-21T06:30:58.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-21T06:30:58.844 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:58.984 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-09-21T06:30:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-21T06:30:58.996 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:59.136 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627781 2024-09-21T06:30:59.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:59.147 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:59.147 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:59.147 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:59.147 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:59.148 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-21T06:30:59.148 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:59.289 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627782 2024-09-21T06:30:59.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:59.299 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:30:59.300 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:30:59.300 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:30:59.300 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:30:59.300 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-21T06:30:59.300 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:30:59.440 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627783 2024-09-21T06:30:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:30:59.451 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-21T06:30:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.633+0000 7f773421ef40 1 Processor -- start 2024-09-21T06:30:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.634+0000 7f773421ef40 1 -- start start 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be3544630 0x564be34fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 0x564be35b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564be3448c60 con 0x564be35415a0 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564be34a71d0 con 0x564be35b38b0 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f773421ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564be3544110 con 0x564be3544630 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f772c800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f772c800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51664/0 (socket says 172.21.15.37:51664) 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f772c800640 1 -- 172.21.15.37:0/3190309164 learned_addr learned my addr 172.21.15.37:0/3190309164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f772d200640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 0x564be35b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.635+0000 7f772be00640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be3544630 0x564be34fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 -- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be3544630 msgr2=0x564be34fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be3544630 0x564be34fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 -- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 msgr2=0x564be35b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 0x564be35b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 -- 172.21.15.37:0/3190309164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be35b4eb0 con 0x564be35415a0 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772d200640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 0x564be35b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772c800640 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f77200465a0 tx=0x7f7720002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a4eb62442135f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772aa00640 1 -- 172.21.15.37:0/3190309164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772004d900 con 0x564be35415a0 2024-09-21T06:30:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772aa00640 1 -- 172.21.15.37:0/3190309164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772004dd90 con 0x564be35415a0 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.636+0000 7f772aa00640 1 -- 172.21.15.37:0/3190309164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200566d0 con 0x564be35415a0 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 -- 172.21.15.37:0/3190309164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 msgr2=0x564be365b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f77200465a0 tx=0x7f7720002fb0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 -- 172.21.15.37:0/3190309164 shutdown_connections 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be35b38b0 0x564be35b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be3544630 0x564be34fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 --2- 172.21.15.37:0/3190309164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35415a0 0x564be365b0c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 -- 172.21.15.37:0/3190309164 >> 172.21.15.37:0/3190309164 conn(0x564be3419ac0 msgr2=0x564be35d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 -- 172.21.15.37:0/3190309164 shutdown_connections 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.637+0000 7f773421ef40 1 -- 172.21.15.37:0/3190309164 wait complete. 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.638+0000 7f773421ef40 1 Processor -- start 2024-09-21T06:30:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.638+0000 7f773421ef40 1 -- start start 2024-09-21T06:30:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be35415a0 0x564be35cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be3544630 0x564be34f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 0x564be34f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564be365a970 con 0x564be35b38b0 2024-09-21T06:30:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564be35b5340 con 0x564be3544630 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f773421ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564be35d0ec0 con 0x564be35415a0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 0x564be34f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 0x564be34f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51676/0 (socket says 172.21.15.37:51676) 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 -- 172.21.15.37:0/2550163877 learned_addr learned my addr 172.21.15.37:0/2550163877 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772c800640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be35415a0 0x564be35cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 -- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be35415a0 msgr2=0x564be35cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772be00640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be3544630 0x564be34f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be35415a0 0x564be35cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 -- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be3544630 msgr2=0x564be34f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564be3544630 0x564be34f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772d200640 1 -- 172.21.15.37:0/2550163877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be35b5590 con 0x564be35b38b0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.640+0000 7f772c800640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564be35415a0 0x564be35cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f772d200640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 0x564be34f92c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f772400c670 tx=0x7f772400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f7729600640 1 -- 172.21.15.37:0/2550163877 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724014070 con 0x564be35b38b0 2024-09-21T06:30:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f7729600640 1 -- 172.21.15.37:0/2550163877 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772400ce70 con 0x564be35b38b0 2024-09-21T06:30:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564be35d0be0 con 0x564be35b38b0 2024-09-21T06:30:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f7729600640 1 -- 172.21.15.37:0/2550163877 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772400aa20 con 0x564be35b38b0 2024-09-21T06:30:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.641+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be35d1240 con 0x564be35b38b0 2024-09-21T06:30:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.643+0000 7f7729600640 1 -- 172.21.15.37:0/2550163877 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f772400abc0 con 0x564be35b38b0 2024-09-21T06:30:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.644+0000 7f7729600640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f770c0775a0 0x7f770c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.644+0000 7f7729600640 1 -- 172.21.15.37:0/2550163877 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f772409ad60 con 0x564be35b38b0 2024-09-21T06:30:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.645+0000 7f773421ef40 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x564be35b47f0 0x564be35b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.645+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x564be3540b90 con 0x564be35b47f0 2024-09-21T06:30:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.645+0000 7f772c800640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f770c0775a0 0x7f770c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.645+0000 7f772be00640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x564be35b47f0 0x564be35b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.646+0000 7f772be00640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x564be35b47f0 0x564be35b24e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.646+0000 7f772c800640 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f770c0775a0 0x7f770c079ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7720000c00 tx=0x7f7720004940 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f772be00640 1 -- 172.21.15.37:0/2550163877 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v56'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7718010350 con 0x564be35b47f0 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x564be35b47f0 msgr2=0x564be35b24e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x564be35b47f0 0x564be35b24e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f770c0775a0 msgr2=0x7f770c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f770c0775a0 0x7f770c079ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7720000c00 tx=0x7f7720004940 comp rx=0 tx=0).stop 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 msgr2=0x564be34f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 --2- 172.21.15.37:0/2550163877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564be35b38b0 0x564be34f92c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f772400c670 tx=0x7f772400cb30 comp rx=0 tx=0).stop 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f772d200640 1 -- 172.21.15.37:0/2550163877 reap_dead start 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 shutdown_connections 2024-09-21T06:30:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 >> 172.21.15.37:0/2550163877 conn(0x564be3419ac0 msgr2=0x564be35cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 shutdown_connections 2024-09-21T06:30:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.648+0000 7f773421ef40 1 -- 172.21.15.37:0/2550163877 wait complete. 2024-09-21T06:30:59.661 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-21T06:30:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.803+0000 7f301a163f40 1 Processor -- start 2024-09-21T06:30:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.803+0000 7f301a163f40 1 -- start start 2024-09-21T06:30:59.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde0224b80 0x55bde0385820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde026c360 0x55bde02fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bde0170c60 con 0x55bde02f2280 2024-09-21T06:30:59.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bde01cf1d0 con 0x55bde0224b80 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.804+0000 7f301a163f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55bde02f2690 con 0x55bde026c360 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51694/0 (socket says 172.21.15.37:51694) 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 -- 172.21.15.37:0/3760541364 learned_addr learned my addr 172.21.15.37:0/3760541364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013200640 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde0224b80 0x55bde0385820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3012800640 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde026c360 0x55bde02fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 -- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde026c360 msgr2=0x55bde02fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde026c360 0x55bde02fa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 -- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde0224b80 msgr2=0x55bde0385820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde0224b80 0x55bde0385820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 -- 172.21.15.37:0/3760541364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde02faa60 con 0x55bde02f2280 2024-09-21T06:30:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.805+0000 7f3013c00640 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f300c057040 tx=0x7f300c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15aab5bd9e58d927 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f3011400640 1 -- 172.21.15.37:0/3760541364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c064030 con 0x55bde02f2280 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f3011400640 1 -- 172.21.15.37:0/3760541364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f300c004020 con 0x55bde02f2280 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f3011400640 1 -- 172.21.15.37:0/3760541364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c004270 con 0x55bde02f2280 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f301a163f40 1 -- 172.21.15.37:0/3760541364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 msgr2=0x55bde0250770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f301a163f40 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f300c057040 tx=0x7f300c003330 comp rx=0 tx=0).stop 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.806+0000 7f301a163f40 1 -- 172.21.15.37:0/3760541364 shutdown_connections 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde02f2280 0x55bde0250770 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde026c360 0x55bde02fa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 --2- 172.21.15.37:0/3760541364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde0224b80 0x55bde0385820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 -- 172.21.15.37:0/3760541364 >> 172.21.15.37:0/3760541364 conn(0x55bde0141ac0 msgr2=0x55bde02fafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 -- 172.21.15.37:0/3760541364 shutdown_connections 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.807+0000 7f301a163f40 1 -- 172.21.15.37:0/3760541364 wait complete. 2024-09-21T06:30:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.808+0000 7f301a163f40 1 Processor -- start 2024-09-21T06:30:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.808+0000 7f301a163f40 1 -- start start 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde0224b80 0x55bde02f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 0x55bde0268510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde02f2280 0x55bde038f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bde024e530 con 0x55bde026c360 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bde02f18c0 con 0x55bde02f2280 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f301a163f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55bde02facb0 con 0x55bde0224b80 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f3012800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 0x55bde0268510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f3012800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 0x55bde0268510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51710/0 (socket says 172.21.15.37:51710) 2024-09-21T06:30:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.809+0000 7f3012800640 1 -- 172.21.15.37:0/3663937300 learned_addr learned my addr 172.21.15.37:0/3663937300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3013200640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde0224b80 0x55bde02f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3013c00640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde02f2280 0x55bde038f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 -- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde0224b80 msgr2=0x55bde02f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bde0224b80 0x55bde02f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 -- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde02f2280 msgr2=0x55bde038f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde02f2280 0x55bde038f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 -- 172.21.15.37:0/3663937300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde02f11b0 con 0x55bde026c360 2024-09-21T06:30:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3013c00640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bde02f2280 0x55bde038f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f3012800640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 0x55bde0268510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3000002cd0 tx=0x7f300000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f300be00640 1 -- 172.21.15.37:0/3663937300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000015020 con 0x55bde026c360 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f300be00640 1 -- 172.21.15.37:0/3663937300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3000011450 con 0x55bde026c360 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bde0385750 con 0x55bde026c360 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.810+0000 7f300be00640 1 -- 172.21.15.37:0/3663937300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000011840 con 0x55bde026c360 2024-09-21T06:30:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.811+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde024e020 con 0x55bde026c360 2024-09-21T06:30:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.813+0000 7f300be00640 1 -- 172.21.15.37:0/3663937300 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3000018020 con 0x55bde026c360 2024-09-21T06:30:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f300be00640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2ff40775a0 0x7f2ff4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f300be00640 1 -- 172.21.15.37:0/3663937300 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f3000099c50 con 0x55bde026c360 2024-09-21T06:30:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f301a163f40 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55bde024cd20 0x55bde0390070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 7.12 7:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x55bde03905b0 con 0x55bde024cd20 2024-09-21T06:30:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f3013200640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2ff40775a0 0x7f2ff4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.814+0000 7f3013c00640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55bde024cd20 0x55bde0390070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.815+0000 7f3013200640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2ff40775a0 0x7f2ff4079ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3004004fd0 tx=0x7f300400e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.815+0000 7f3013c00640 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55bde024cd20 0x55bde0390070 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f3013c00640 1 -- 172.21.15.37:0/3663937300 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v56'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f300c00b040 con 0x55bde024cd20 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55bde024cd20 msgr2=0x55bde0390070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55bde024cd20 0x55bde0390070 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2ff40775a0 msgr2=0x7f2ff4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2ff40775a0 0x7f2ff4079ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3004004fd0 tx=0x7f300400e040 comp rx=0 tx=0).stop 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 msgr2=0x55bde0268510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 --2- 172.21.15.37:0/3663937300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bde026c360 0x55bde0268510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3000002cd0 tx=0x7f300000c620 comp rx=0 tx=0).stop 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f3013c00640 1 -- 172.21.15.37:0/3663937300 reap_dead start 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 shutdown_connections 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 >> 172.21.15.37:0/3663937300 conn(0x55bde0141ac0 msgr2=0x55bde02faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 shutdown_connections 2024-09-21T06:30:59.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.817+0000 7f301a163f40 1 -- 172.21.15.37:0/3663937300 wait complete. 2024-09-21T06:30:59.832 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-21T06:30:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.976+0000 7f8a35bd1f40 1 Processor -- start 2024-09-21T06:30:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.976+0000 7f8a35bd1f40 1 -- start start 2024-09-21T06:30:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d54a360 0x56468d544950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d544e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56468d44ec60 con 0x56468d5029b0 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56468d4ad1d0 con 0x56468d5d0280 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56468d502dc0 con 0x56468d54a360 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51732/0 (socket says 172.21.15.37:51732) 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.977+0000 7f8a2ee00640 1 -- 172.21.15.37:0/3965428132 learned_addr learned my addr 172.21.15.37:0/3965428132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2e400640 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d54a360 0x56468d544950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2f800640 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d544e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 -- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d54a360 msgr2=0x56468d544950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d54a360 0x56468d544950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 -- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 msgr2=0x56468d544e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:30:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d544e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 -- 172.21.15.37:0/3965428132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d5d7620 con 0x56468d5029b0 2024-09-21T06:30:59.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c630 tx=0x7f8a2000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf184749e9b4ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2d000640 1 -- 172.21.15.37:0/3965428132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000aa80 con 0x56468d5029b0 2024-09-21T06:30:59.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2d000640 1 -- 172.21.15.37:0/3965428132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a2000ac20 con 0x56468d5029b0 2024-09-21T06:30:59.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.978+0000 7f8a2d000640 1 -- 172.21.15.37:0/3965428132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20015440 con 0x56468d5029b0 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/3965428132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 msgr2=0x56468d52ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c630 tx=0x7f8a2000caf0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/3965428132 shutdown_connections 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d544e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d54a360 0x56468d544950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/3965428132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d5029b0 0x56468d52ddb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/3965428132 >> 172.21.15.37:0/3965428132 conn(0x56468d41fac0 msgr2=0x56468d548b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/3965428132 shutdown_connections 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.979+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/3965428132 wait complete. 2024-09-21T06:30:59.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.980+0000 7f8a35bd1f40 1 Processor -- start 2024-09-21T06:30:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.981+0000 7f8a35bd1f40 1 -- start start 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d5029b0 0x56468d544b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 0x56468d5c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d5cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56468d5d5930 con 0x56468d54a360 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56468d546070 con 0x56468d5d0280 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a35bd1f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56468d5d5b80 con 0x56468d5029b0 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a2e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 0x56468d5c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a2e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 0x56468d5c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51736/0 (socket says 172.21.15.37:51736) 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.982+0000 7f8a2e400640 1 -- 172.21.15.37:0/2119628473 learned_addr learned my addr 172.21.15.37:0/2119628473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d5029b0 0x56468d544b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2f800640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d5cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 -- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d5029b0 msgr2=0x56468d544b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d5029b0 0x56468d544b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 -- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 msgr2=0x56468d5cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d5cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 -- 172.21.15.37:0/2119628473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d5cfad0 con 0x56468d54a360 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56468d5029b0 0x56468d544b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2f800640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56468d5d0280 0x56468d5cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a2e400640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 0x56468d5c20c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0465a0 tx=0x7f8a1c04d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a27400640 1 -- 172.21.15.37:0/2119628473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c067070 con 0x56468d54a360 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a27400640 1 -- 172.21.15.37:0/2119628473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a1c056820 con 0x56468d54a360 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a27400640 1 -- 172.21.15.37:0/2119628473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c056c70 con 0x56468d54a360 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56468d500350 con 0x56468d54a360 2024-09-21T06:30:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.983+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d5c9e00 con 0x56468d54a360 2024-09-21T06:30:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.986+0000 7f8a27400640 1 -- 172.21.15.37:0/2119628473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a1c06a070 con 0x56468d54a360 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a27400640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a100775a0 0x7f8a10079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a27400640 1 -- 172.21.15.37:0/2119628473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f8a1c0dd860 con 0x56468d54a360 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x56468d548c50 0x56468d5c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a100775a0 0x7f8a10079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- osd_op(unknown.0.0:1 7.5 7:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x56468d5c32f0 con 0x56468d548c50 2024-09-21T06:30:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.987+0000 7f8a2f800640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x56468d548c50 0x56468d5c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:59.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.988+0000 7f8a2ee00640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a100775a0 0x7f8a10079ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000c00 tx=0x7f8a20017040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.988+0000 7f8a2f800640 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x56468d548c50 0x56468d5c2db0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:59.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a2f800640 1 -- 172.21.15.37:0/2119628473 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v56'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8a2800e1d0 con 0x56468d548c50 2024-09-21T06:30:59.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x56468d548c50 msgr2=0x56468d5c2db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x56468d548c50 0x56468d5c2db0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:59.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a100775a0 msgr2=0x7f8a10079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8a100775a0 0x7f8a10079ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000c00 tx=0x7f8a20017040 comp rx=0 tx=0).stop 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 msgr2=0x56468d5c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 --2- 172.21.15.37:0/2119628473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56468d54a360 0x56468d5c20c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0465a0 tx=0x7f8a1c04d860 comp rx=0 tx=0).stop 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a2f800640 1 -- 172.21.15.37:0/2119628473 reap_dead start 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 shutdown_connections 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 >> 172.21.15.37:0/2119628473 conn(0x56468d41fac0 msgr2=0x56468d5c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 shutdown_connections 2024-09-21T06:30:59.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:30:59.990+0000 7f8a35bd1f40 1 -- 172.21.15.37:0/2119628473 wait complete. 2024-09-21T06:31:00.002 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-21T06:31:00.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.144+0000 7ff7053e1f40 1 Processor -- start 2024-09-21T06:31:00.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.144+0000 7ff7053e1f40 1 -- start start 2024-09-21T06:31:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141c31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141cb8380 0x564141bef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564141b3bc60 con 0x564141bef9b0 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564141b9a1d0 con 0x564141cb8380 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.145+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564141cbd690 con 0x564141cbd280 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51746/0 (socket says 172.21.15.37:51746) 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 -- 172.21.15.37:0/868666205 learned_addr learned my addr 172.21.15.37:0/868666205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fdc00640 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141c31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6ff000640 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141cb8380 0x564141bef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 -- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 msgr2=0x564141c31e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141c31e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 -- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141cb8380 msgr2=0x564141bef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141cb8380 0x564141bef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 -- 172.21.15.37:0/868666205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564141c31830 con 0x564141bef9b0 2024-09-21T06:31:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.146+0000 7ff6fe600640 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0001d10 tx=0x7ff6f000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca07c71c6146919 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff6f7e00640 1 -- 172.21.15.37:0/868666205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0001e60 con 0x564141bef9b0 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff6f7e00640 1 -- 172.21.15.37:0/868666205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f000bd80 con 0x564141bef9b0 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff6f7e00640 1 -- 172.21.15.37:0/868666205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000f660 con 0x564141bef9b0 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 -- 172.21.15.37:0/868666205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 msgr2=0x564141cba710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0001d10 tx=0x7ff6f000b5b0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 -- 172.21.15.37:0/868666205 shutdown_connections 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141cb8380 0x564141bef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141c31e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/868666205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141bef9b0 0x564141cba710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.147+0000 7ff7053e1f40 1 -- 172.21.15.37:0/868666205 >> 172.21.15.37:0/868666205 conn(0x564141b0cac0 msgr2=0x564141c30f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.148+0000 7ff7053e1f40 1 -- 172.21.15.37:0/868666205 shutdown_connections 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.148+0000 7ff7053e1f40 1 -- 172.21.15.37:0/868666205 wait complete. 2024-09-21T06:31:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.149+0000 7ff7053e1f40 1 Processor -- start 2024-09-21T06:31:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.149+0000 7ff7053e1f40 1 -- start start 2024-09-21T06:31:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141bef9b0 0x564141c35310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 0x564141caa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141caaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564141cb9fc0 con 0x564141cb8380 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564141c1c050 con 0x564141bef9b0 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff7053e1f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564141c1bd20 con 0x564141cbd280 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff6fdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 0x564141caa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff6fdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 0x564141caa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51758/0 (socket says 172.21.15.37:51758) 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.150+0000 7ff6fdc00640 1 -- 172.21.15.37:0/2026130821 learned_addr learned my addr 172.21.15.37:0/2026130821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fe600640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141bef9b0 0x564141c35310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6ff000640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141caaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 -- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 msgr2=0x564141caaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564141cbd280 0x564141caaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 -- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141bef9b0 msgr2=0x564141c35310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564141bef9b0 0x564141c35310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 -- 172.21.15.37:0/2026130821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564141c35850 con 0x564141cb8380 2024-09-21T06:31:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.151+0000 7ff6fdc00640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 0x564141caa4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec025e70 tx=0x7ff6ec04d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.152+0000 7ff6f6a00640 1 -- 172.21.15.37:0/2026130821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec068070 con 0x564141cb8380 2024-09-21T06:31:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.152+0000 7ff6f6a00640 1 -- 172.21.15.37:0/2026130821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6ec058280 con 0x564141cb8380 2024-09-21T06:31:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.152+0000 7ff6f6a00640 1 -- 172.21.15.37:0/2026130821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec058770 con 0x564141cb8380 2024-09-21T06:31:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.152+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564141caa9e0 con 0x564141cb8380 2024-09-21T06:31:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.152+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564141cbc710 con 0x564141cb8380 2024-09-21T06:31:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.154+0000 7ff6f6a00640 1 -- 172.21.15.37:0/2026130821 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6ec06b070 con 0x564141cb8380 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff6f6a00640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff6e00775a0 0x7ff6e0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff6f6a00640 1 -- 172.21.15.37:0/2026130821 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff6ec0dd910 con 0x564141cb8380 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff6fe600640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff6e00775a0 0x7ff6e0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x564141cb8bf0 0x564141cb9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- osd_op(unknown.0.0:1 7.9 7:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x564141cab440 con 0x564141cb8bf0 2024-09-21T06:31:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.155+0000 7ff6ff000640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x564141cb8bf0 0x564141cb9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.156+0000 7ff6fe600640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff6e00775a0 0x7ff6e0079ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00012d0 tx=0x7ff6f0000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.156+0000 7ff6ff000640 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x564141cb8bf0 0x564141cb9000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff6ff000640 1 -- 172.21.15.37:0/2026130821 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v56'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6f8003040 con 0x564141cb8bf0 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x564141cb8bf0 msgr2=0x564141cb9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x564141cb8bf0 0x564141cb9000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff6e00775a0 msgr2=0x7ff6e0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff6e00775a0 0x7ff6e0079ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00012d0 tx=0x7ff6f0000ed0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 msgr2=0x564141caa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 --2- 172.21.15.37:0/2026130821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564141cb8380 0x564141caa4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec025e70 tx=0x7ff6ec04d8e0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff6ff000640 1 -- 172.21.15.37:0/2026130821 reap_dead start 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 shutdown_connections 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 >> 172.21.15.37:0/2026130821 conn(0x564141b0cac0 msgr2=0x564141bef010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 shutdown_connections 2024-09-21T06:31:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.158+0000 7ff7053e1f40 1 -- 172.21.15.37:0/2026130821 wait complete. 2024-09-21T06:31:00.173 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-21T06:31:00.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.315+0000 7fd55f4f3f40 1 Processor -- start 2024-09-21T06:31:00.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.315+0000 7fd55f4f3f40 1 -- start start 2024-09-21T06:31:00.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.316+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291fd630 0x55ce291b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291b5110 0x55ce2931bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ce29101c60 con 0x55ce291fa5a0 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ce291601d0 con 0x55ce291fd630 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55ce291fd110 con 0x55ce291b5110 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51780/0 (socket says 172.21.15.37:51780) 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 -- 172.21.15.37:0/439008475 learned_addr learned my addr 172.21.15.37:0/439008475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 -- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291b5110 msgr2=0x55ce2931bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291b5110 0x55ce2931bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 -- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291fd630 msgr2=0x55ce291b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291fd630 0x55ce291b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 -- 172.21.15.37:0/439008475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce2926afd0 con 0x55ce291fa5a0 2024-09-21T06:31:00.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.317+0000 7fd558600640 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b5d0 tx=0x7fd54800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d149049cdf0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd556800640 1 -- 172.21.15.37:0/439008475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54800a8c0 con 0x55ce291fa5a0 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd556800640 1 -- 172.21.15.37:0/439008475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd54800aa60 con 0x55ce291fa5a0 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd556800640 1 -- 172.21.15.37:0/439008475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548015460 con 0x55ce291fa5a0 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/439008475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 msgr2=0x55ce291e2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b5d0 tx=0x7fd54800ba90 comp rx=0 tx=0).stop 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/439008475 shutdown_connections 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291b5110 0x55ce2931bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291fd630 0x55ce291b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/439008475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fa5a0 0x55ce291e2700 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/439008475 >> 172.21.15.37:0/439008475 conn(0x55ce290d2ac0 msgr2=0x55ce29289c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/439008475 shutdown_connections 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.318+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/439008475 wait complete. 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.319+0000 7fd55f4f3f40 1 Processor -- start 2024-09-21T06:31:00.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.320+0000 7fd55f4f3f40 1 -- start start 2024-09-21T06:31:00.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291b5110 0x55ce29320b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291fa5a0 0x55ce29269b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 0x55ce2926c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ce291b2bc0 con 0x55ce291fd630 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ce291b5630 con 0x55ce291b5110 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd55f4f3f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55ce291b5d00 con 0x55ce291fa5a0 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd559000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 0x55ce2926c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd559000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 0x55ce2926c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51786/0 (socket says 172.21.15.37:51786) 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd559000640 1 -- 172.21.15.37:0/2001259820 learned_addr learned my addr 172.21.15.37:0/2001259820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd558600640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291b5110 0x55ce29320b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.321+0000 7fd557c00640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291fa5a0 0x55ce29269b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 -- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291fa5a0 msgr2=0x55ce29269b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ce291fa5a0 0x55ce29269b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 -- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291b5110 msgr2=0x55ce29320b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce291b5110 0x55ce29320b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 -- 172.21.15.37:0/2001259820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce291b49a0 con 0x55ce291fd630 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd559000640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 0x55ce2926c320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd550057040 tx=0x7fd550004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd555400640 1 -- 172.21.15.37:0/2001259820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550064070 con 0x55ce291fd630 2024-09-21T06:31:00.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd555400640 1 -- 172.21.15.37:0/2001259820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd55004d7f0 con 0x55ce291fd630 2024-09-21T06:31:00.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce291b4ce0 con 0x55ce291fd630 2024-09-21T06:31:00.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.322+0000 7fd555400640 1 -- 172.21.15.37:0/2001259820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550004280 con 0x55ce291fd630 2024-09-21T06:31:00.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.323+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce2926a890 con 0x55ce291fd630 2024-09-21T06:31:00.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.325+0000 7fd555400640 1 -- 172.21.15.37:0/2001259820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd55005e040 con 0x55ce291fd630 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd555400640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd5380775a0 0x7fd538079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd555400640 1 -- 172.21.15.37:0/2001259820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fd5500ddae0 con 0x55ce291fd630 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x55ce2926ab70 0x55ce2926cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- osd_op(unknown.0.0:1 7.5 7:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x55ce2932d150 con 0x55ce2926ab70 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd558600640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd5380775a0 0x7fd538079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.326+0000 7fd557c00640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x55ce2926ab70 0x55ce2926cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.327+0000 7fd557c00640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x55ce2926ab70 0x55ce2926cd70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.327+0000 7fd558600640 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd5380775a0 0x7fd538079ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd548002040 tx=0x7fd548002fb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd557c00640 1 -- 172.21.15.37:0/2001259820 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v56'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd5440104c0 con 0x55ce2926ab70 2024-09-21T06:31:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x55ce2926ab70 msgr2=0x55ce2926cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x55ce2926ab70 0x55ce2926cd70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd5380775a0 msgr2=0x7fd538079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd5380775a0 0x7fd538079ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd548002040 tx=0x7fd548002fb0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 msgr2=0x55ce2926c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 --2- 172.21.15.37:0/2001259820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ce291fd630 0x55ce2926c320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd550057040 tx=0x7fd550004ab0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd559000640 1 -- 172.21.15.37:0/2001259820 reap_dead start 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 shutdown_connections 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 >> 172.21.15.37:0/2001259820 conn(0x55ce290d2ac0 msgr2=0x55ce29289c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 shutdown_connections 2024-09-21T06:31:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.329+0000 7fd55f4f3f40 1 -- 172.21.15.37:0/2001259820 wait complete. 2024-09-21T06:31:00.341 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-21T06:31:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.485+0000 7fddf5e0ef40 1 Processor -- start 2024-09-21T06:31:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.485+0000 7fddf5e0ef40 1 -- start start 2024-09-21T06:31:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd4003a40 0x55de6f10a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 0x55de6f05ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de6ef80c60 con 0x55de6f060510 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de6efdf1d0 con 0x55de6f05fbc0 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55de6f102690 con 0x7fddd4003a40 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddeee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddeee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51802/0 (socket says 172.21.15.37:51802) 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.486+0000 7fddeee00640 1 -- 172.21.15.37:0/2820724176 learned_addr learned my addr 172.21.15.37:0/2820724176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddee400640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd4003a40 0x55de6f10a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeda00640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 0x55de6f05ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 -- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd4003a40 msgr2=0x55de6f10a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd4003a40 0x55de6f10a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 -- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 msgr2=0x55de6f05ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 0x55de6f05ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 -- 172.21.15.37:0/2820724176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6f05dae0 con 0x55de6f060510 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeda00640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 0x55de6f05ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fddeee00640 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdde8056040 tx=0x7fdde80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4171952c0a64572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fdde7e00640 1 -- 172.21.15.37:0/2820724176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800a3f0 con 0x55de6f060510 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fdde7e00640 1 -- 172.21.15.37:0/2820724176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde8004130 con 0x55de6f060510 2024-09-21T06:31:00.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.487+0000 7fdde7e00640 1 -- 172.21.15.37:0/2820724176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde80546f0 con 0x55de6f060510 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/2820724176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 msgr2=0x55de6f19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdde8056040 tx=0x7fdde80033a0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/2820724176 shutdown_connections 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f060510 0x55de6f19aab0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de6f05fbc0 0x55de6f05ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/2820724176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd4003a40 0x55de6f10a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/2820724176 >> 172.21.15.37:0/2820724176 conn(0x55de6ef51ac0 msgr2=0x55de6f05e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/2820724176 shutdown_connections 2024-09-21T06:31:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.488+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/2820724176 wait complete. 2024-09-21T06:31:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.489+0000 7fddf5e0ef40 1 Processor -- start 2024-09-21T06:31:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.489+0000 7fddf5e0ef40 1 -- start start 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 0x55de6f0f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de6f060510 0x55de6f07b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd4003a40 0x55de6f07be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de6f10aff0 con 0x55de6f05fbc0 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de6f0fd260 con 0x7fddd4003a40 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddf5e0ef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55de6f0f5b40 con 0x55de6f060510 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddee400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 0x55de6f0f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddee400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 0x55de6f0f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51808/0 (socket says 172.21.15.37:51808) 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddee400640 1 -- 172.21.15.37:0/1208998173 learned_addr learned my addr 172.21.15.37:0/1208998173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddeda00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de6f060510 0x55de6f07b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.491+0000 7fddeee00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd4003a40 0x55de6f07be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 -- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de6f060510 msgr2=0x55de6f07b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de6f060510 0x55de6f07b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 -- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd4003a40 msgr2=0x55de6f07be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd4003a40 0x55de6f07be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 -- 172.21.15.37:0/1208998173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6f0f8890 con 0x55de6f05fbc0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddeda00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de6f060510 0x55de6f07b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddeee00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd4003a40 0x55de6f07be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddee400640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 0x55de6f0f3500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdde0010640 tx=0x7fdde0010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fdde6a00640 1 -- 172.21.15.37:0/1208998173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000ea70 con 0x55de6f05fbc0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fdde6a00640 1 -- 172.21.15.37:0/1208998173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde000ec10 con 0x55de6f05fbc0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de6f0f8b70 con 0x55de6f05fbc0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fdde6a00640 1 -- 172.21.15.37:0/1208998173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde001a400 con 0x55de6f05fbc0 2024-09-21T06:31:00.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.492+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6f0f9130 con 0x55de6f05fbc0 2024-09-21T06:31:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.494+0000 7fdde6a00640 1 -- 172.21.15.37:0/1208998173 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdde00193b0 con 0x55de6f05fbc0 2024-09-21T06:31:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.495+0000 7fdde6a00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fddd00775a0 0x7fddd0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.495+0000 7fdde6a00640 1 -- 172.21.15.37:0/1208998173 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fdde009a7b0 con 0x55de6f05fbc0 2024-09-21T06:31:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.496+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55de6f0fc3c0 0x55de6f0fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.496+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- osd_op(unknown.0.0:1 7.1d 7:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x55de6f1aba10 con 0x55de6f0fc3c0 2024-09-21T06:31:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.496+0000 7fddeda00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fddd00775a0 0x7fddd0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.496+0000 7fddeee00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55de6f0fc3c0 0x55de6f0fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.497+0000 7fddeda00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fddd00775a0 0x7fddd0079ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55de6f100d20 tx=0x7fdddc004880 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.497+0000 7fddeee00640 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55de6f0fc3c0 0x55de6f0fc7d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddeee00640 1 -- 172.21.15.37:0/1208998173 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v56'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdde800b040 con 0x55de6f0fc3c0 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55de6f0fc3c0 msgr2=0x55de6f0fc7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55de6f0fc3c0 0x55de6f0fc7d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fddd00775a0 msgr2=0x7fddd0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fddd00775a0 0x7fddd0079ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55de6f100d20 tx=0x7fdddc004880 comp rx=0 tx=0).stop 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 msgr2=0x55de6f0f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 --2- 172.21.15.37:0/1208998173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de6f05fbc0 0x55de6f0f3500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdde0010640 tx=0x7fdde0010b00 comp rx=0 tx=0).stop 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddeee00640 1 -- 172.21.15.37:0/1208998173 reap_dead start 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 shutdown_connections 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 >> 172.21.15.37:0/1208998173 conn(0x55de6ef51ac0 msgr2=0x55de6f05e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 shutdown_connections 2024-09-21T06:31:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.499+0000 7fddf5e0ef40 1 -- 172.21.15.37:0/1208998173 wait complete. 2024-09-21T06:31:00.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:00.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.742+0000 7fc4f4800640 1 -- 172.21.15.37:0/600979012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160810 msgr2=0x7fc4ec06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.742+0000 7fc4f4800640 1 --2- 172.21.15.37:0/600979012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160810 0x7fc4ec06d2e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009830 tx=0x7fc4dc02a510 comp rx=0 tx=0).stop 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.742+0000 7fc4f4800640 1 -- 172.21.15.37:0/600979012 shutdown_connections 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.742+0000 7fc4f4800640 1 --2- 172.21.15.37:0/600979012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160810 0x7fc4ec06d2e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.742+0000 7fc4f4800640 1 -- 172.21.15.37:0/600979012 >> 172.21.15.37:0/600979012 conn(0x7fc4ec0ccac0 msgr2=0x7fc4ec0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.743+0000 7fc4f4800640 1 -- 172.21.15.37:0/600979012 shutdown_connections 2024-09-21T06:31:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.743+0000 7fc4f4800640 1 -- 172.21.15.37:0/600979012 wait complete. 2024-09-21T06:31:00.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.745+0000 7fc4f4800640 1 Processor -- start 2024-09-21T06:31:00.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.745+0000 7fc4f4800640 1 -- start start 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.746+0000 7fc4f4800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4ec0ca690 0x7fc4ec0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.746+0000 7fc4f4800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec160810 0x7fc4ec150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.746+0000 7fc4f4800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.746+0000 7fc4f4800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec06e2c0 con 0x7fc4ec160c20 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f4800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec06e140 con 0x7fc4ec160810 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f4800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec06e440 con 0x7fc4ec0ca690 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51836/0 (socket says 172.21.15.37:51836) 2024-09-21T06:31:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 -- 172.21.15.37:0/3077704119 learned_addr learned my addr 172.21.15.37:0/3077704119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4ec0ca690 0x7fc4ec0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec160810 0x7fc4ec150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 -- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4ec0ca690 msgr2=0x7fc4ec0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4ec0ca690 0x7fc4ec0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 -- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec160810 msgr2=0x7fc4ec150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec160810 0x7fc4ec150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.747+0000 7fc4f2800640 1 -- 172.21.15.37:0/3077704119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0b3e90 con 0x7fc4ec160c20 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800be60 tx=0x7fc4e800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4e6000640 1 -- 172.21.15.37:0/3077704119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8019070 con 0x7fc4ec160c20 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec0b38d0 con 0x7fc4ec160c20 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec09ad20 con 0x7fc4ec160c20 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4e6000640 1 -- 172.21.15.37:0/3077704119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e8003970 con 0x7fc4ec160c20 2024-09-21T06:31:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.748+0000 7fc4e6000640 1 -- 172.21.15.37:0/3077704119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8003da0 con 0x7fc4ec160c20 2024-09-21T06:31:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.751+0000 7fc4e6000640 1 -- 172.21.15.37:0/3077704119 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4e801c070 con 0x7fc4ec160c20 2024-09-21T06:31:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.752+0000 7fc4e6000640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 0x7fc4d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.752+0000 7fc4e6000640 1 -- 172.21.15.37:0/3077704119 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc4e809adc0 con 0x7fc4ec160c20 2024-09-21T06:31:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.752+0000 7fc4fb812740 1 Processor -- start 2024-09-21T06:31:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.752+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 0x7fc4d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.752+0000 7fc4fb812740 1 -- start start 2024-09-21T06:31:00.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 0x7fc4d007b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0000f60 tx=0x7fc4e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfaf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcf91e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2dcec70c0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d2dcedaf90 con 0x55d2dcb977c0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.753+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d2dceca3c0 con 0x55d2dcbc0680 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51840/0 (socket says 172.21.15.37:51840) 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 -- 172.21.15.37:0/2639727368 learned_addr learned my addr 172.21.15.37:0/2639727368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfaf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f1400640 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcf91e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 -- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 msgr2=0x55d2dcfaf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfaf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 -- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 msgr2=0x55d2dcf91e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcf91e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.754+0000 7fc4f2800640 1 -- 172.21.15.37:0/2639727368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dcca0a50 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.755+0000 7fc4f2800640 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8098fe0 tx=0x7fc4e80628f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63402b4006afee8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.755+0000 7fc4da600640 1 -- 172.21.15.37:0/2639727368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e806c2d0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.755+0000 7fc4da600640 1 -- 172.21.15.37:0/2639727368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e805ec60 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.755+0000 7fc4da600640 1 -- 172.21.15.37:0/2639727368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e806c080 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 -- 172.21.15.37:0/2639727368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 msgr2=0x55d2dcf92350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8098fe0 tx=0x7fc4e80628f0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 -- 172.21.15.37:0/2639727368 shutdown_connections 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf92350 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcf91e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2639727368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 -- 172.21.15.37:0/2639727368 >> 172.21.15.37:0/2639727368 conn(0x55d2dcbb47d0 msgr2=0x55d2dcf68030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 -- 172.21.15.37:0/2639727368 shutdown_connections 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.756+0000 7fc4fb812740 1 -- 172.21.15.37:0/2639727368 wait complete. 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.757+0000 7fc4fb812740 1 -- start start 2024-09-21T06:31:00.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.757+0000 7fc4fb812740 1 Processor -- start 2024-09-21T06:31:00.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcfa7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfa4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2dce6faf0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d2dcece1d0 con 0x55d2dcb977c0 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.758+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d2dcc3aa30 con 0x55d2dcbc0680 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4fb812740 10 client.0 _ll_register_callbacks cb 0x55d2dce75b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51852/0 (socket says 172.21.15.37:51852) 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 -- 172.21.15.37:0/3921761553 learned_addr learned my addr 172.21.15.37:0/3921761553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcfa7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfa4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 -- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 msgr2=0x55d2dcfa4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfa4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 -- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 msgr2=0x55d2dcfa7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcfa7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.759+0000 7fc4f1400640 1 -- 172.21.15.37:0/3921761553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dcc09990 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005eb0 tx=0x7fc4dc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36bde71f0061cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4cea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc003da0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc4dc005020 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e805ec60 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc4dc04e050 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4fb812740 10 client.5922 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55d2dcd011e0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.760+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e805ec60 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7fc4dc051020 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 10 client.5922 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:00.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 10 client.5922 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 20 client.5922 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcb7f250 0x55d2dcfa59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 4 client.5922 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2dcedcee0 con 0x55d2dcb7f250 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcb7f250 0x55d2dcfa59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcb7f250 0x55d2dcfa59a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f7418b2a04b4a1 server_cookie=885186d21f90c613 in_seq=0 out_seq=0 2024-09-21T06:31:00.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.761+0000 7fc4cea00640 10 client.5922 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.762+0000 7fc4cea00640 1 -- 172.21.15.37:0/3921761553 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2dcedcee0 con 0x55d2dcb7f250 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.762+0000 7fc4cea00640 10 client.5922 handle_command_reply: tid=0 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.762+0000 7fc4fb812740 1 client.5922 shutdown 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.762+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 msgr2=0x55d2dcfa8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.762+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005eb0 tx=0x7fc4dc003960 comp rx=0 tx=0).stop 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 shutdown_connections 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcfa8b60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcb7f250 0x55d2dcfa59a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dcfa4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3921761553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dcfa7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4cea00640 0 client.5922 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 >> 172.21.15.37:0/3921761553 conn(0x55d2dcbb47d0 msgr2=0x55d2dcfb01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 shutdown_connections 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 1 -- 172.21.15.37:0/3921761553 wait complete. 2024-09-21T06:31:00.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.763+0000 7fc4fb812740 20 client.5922 trim_cache size 0 max 16384 2024-09-21T06:31:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.782+0000 7fc4fb812740 1 Processor -- start 2024-09-21T06:31:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.782+0000 7fc4fb812740 1 -- start start 2024-09-21T06:31:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcb977c0 0x55d2dcf62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 0x55d2dcf91e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800d040 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800c210 con 0x55d2dcbc0680 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c800c910 con 0x55d2dcb977c0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51866/0 (socket says 172.21.15.37:51866) 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.783+0000 7fc4f1400640 1 -- 172.21.15.37:0/3349737159 learned_addr learned my addr 172.21.15.37:0/3349737159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcb977c0 0x55d2dcf62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 0x55d2dcf91e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 -- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcb977c0 msgr2=0x55d2dcf62c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcb977c0 0x55d2dcf62c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 -- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 msgr2=0x55d2dcf91e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 0x55d2dcf91e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 -- 172.21.15.37:0/3349737159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dcc9ed50 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f2800640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 0x55d2dcf91e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4f1400640 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02acc0 tx=0x7fc4dc02aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344ce96679aba413 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4d9c00640 1 -- 172.21.15.37:0/3349737159 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc04b070 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4d9c00640 1 -- 172.21.15.37:0/3349737159 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4dc0055f0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.784+0000 7fc4d9c00640 1 -- 172.21.15.37:0/3349737159 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc004a00 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 -- 172.21.15.37:0/3349737159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 msgr2=0x55d2dcf5dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02acc0 tx=0x7fc4dc02aa50 comp rx=0 tx=0).stop 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 -- 172.21.15.37:0/3349737159 shutdown_connections 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf5dca0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcbc0680 0x55d2dcf91e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 --2- 172.21.15.37:0/3349737159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcb977c0 0x55d2dcf62c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 -- 172.21.15.37:0/3349737159 >> 172.21.15.37:0/3349737159 conn(0x55d2dcbb47d0 msgr2=0x55d2dcf61240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 -- 172.21.15.37:0/3349737159 shutdown_connections 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.785+0000 7fc4fb812740 1 -- 172.21.15.37:0/3349737159 wait complete. 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.786+0000 7fc4fb812740 1 -- start start 2024-09-21T06:31:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.786+0000 7fc4fb812740 1 Processor -- start 2024-09-21T06:31:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dd027090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dd033cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800b1b0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d2dcfa3dd0 con 0x55d2dcb977c0 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d2dd034800 con 0x55d2dcbc0680 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4fb812740 10 client.0 _ll_register_callbacks cb 0x7fc4dc0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51876/0 (socket says 172.21.15.37:51876) 2024-09-21T06:31:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.788+0000 7fc4f1400640 1 -- 172.21.15.37:0/2063281712 learned_addr learned my addr 172.21.15.37:0/2063281712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 -- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 msgr2=0x55d2dd033cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f2800640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dd033cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dd027090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dd033cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 -- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 msgr2=0x55d2dd027090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dd027090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 -- 172.21.15.37:0/2063281712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dcf7e1c0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dd027090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f2800640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dd033cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4f1400640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000c00 tx=0x7fc4dc0036c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b91be6cecb01a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4cea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc04b040 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc4dc057070 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4dc0055f0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc4dc0050d0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4fb812740 10 client.5928 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.789+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55d2dcf7e500 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4dc0055f0 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7fc4dc063050 con 0x55d2dcb9f9a0 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 10 client.5928 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 10 client.5928 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 20 client.5928 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcf65a90 0x55d2dcf91e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 4 client.5928 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55d2dced8990 con 0x55d2dcf65a90 2024-09-21T06:31:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.790+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcf65a90 0x55d2dcf91e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.791+0000 7fc4f1e00640 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcf65a90 0x55d2dcf91e10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457e5f09412a617b server_cookie=ed51feb68f0ec9d2 in_seq=0 out_seq=0 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.791+0000 7fc4cea00640 10 client.5928 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.791+0000 7fc4cea00640 1 -- 172.21.15.37:0/2063281712 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2dced8990 con 0x55d2dcf65a90 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.791+0000 7fc4cea00640 10 client.5928 handle_command_reply: tid=0 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.791+0000 7fc4fb812740 1 client.5928 shutdown 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 msgr2=0x55d2dcf66580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000c00 tx=0x7fc4dc0036c0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.795 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:31:00.792+0000 7f9349a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1) n(v3 rc2024-09-21T06:30:45.469447+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ece5b99600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:30:45.469447+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 shutdown_connections 2024-09-21T06:31:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dcb9f9a0 0x55d2dcf66580 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55d2dcf65a90 0x55d2dcf91e10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d2dcbc0680 0x55d2dd033cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 --2- 172.21.15.37:0/2063281712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dcb977c0 0x55d2dd027090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4cea00640 0 client.5928 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 >> 172.21.15.37:0/2063281712 conn(0x55d2dcbb47d0 msgr2=0x55d2dcf67310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 shutdown_connections 2024-09-21T06:31:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 1 -- 172.21.15.37:0/2063281712 wait complete. 2024-09-21T06:31:00.796 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:31:00.792+0000 7f87e6a00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1) n(v2 rc2024-09-21T06:30:45.469447+0000 4=0+4) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a1db340c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:30:45.469447+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.792+0000 7fc4fb812740 20 client.5928 trim_cache size 0 max 16384 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "69a4999b-afdb-4309-9e01-a1f19558c26d", 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-21T06:31:00.797 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 msgr2=0x7fc4d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 0x7fc4d007b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0000f60 tx=0x7fc4e0009040 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 msgr2=0x7fc4ec073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800be60 tx=0x7fc4e800bf50 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 shutdown_connections 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4d0078b90 0x7fc4d007b0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ec160c20 0x7fc4ec073070 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec160810 0x7fc4ec150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 --2- 172.21.15.37:0/3077704119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4ec0ca690 0x7fc4ec0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.793+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 >> 172.21.15.37:0/3077704119 conn(0x7fc4ec0ccac0 msgr2=0x7fc4ec0aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.794+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 shutdown_connections 2024-09-21T06:31:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:00.794+0000 7fc4f4800640 1 -- 172.21.15.37:0/3077704119 wait complete. 2024-09-21T06:31:00.799 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:31:00.794+0000 7f9351c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1) n(v4 rc2024-09-21T06:30:45.469447+0000 5=0+5) (inest sync->mix g=1,2) caps={5327=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55ece5b99080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:30:45.469447+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.799 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:31:00.795+0000 7f87eec00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1) n(v1 rc2024-09-21T06:30:45.469447+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5327=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a1db086580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:30:45.469447+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.799 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:31:00.792+0000 7f97e6c00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1) n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5617ead6e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.800 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:31:00.793+0000 7f97e6c00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/AsLsXsFsx@2},l=5327 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5617ead6eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:31:00.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:00.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub abort 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.045+0000 7f3460400640 1 -- 172.21.15.37:0/3465525352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580d0d80 msgr2=0x7f3458167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.045+0000 7f3460400640 1 --2- 172.21.15.37:0/3465525352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580d0d80 0x7f3458167550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3450005fd0 tx=0x7f345002a820 comp rx=0 tx=0).stop 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.046+0000 7f3460400640 1 -- 172.21.15.37:0/3465525352 shutdown_connections 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.046+0000 7f3460400640 1 --2- 172.21.15.37:0/3465525352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580d0d80 0x7f3458167550 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.046+0000 7f3460400640 1 -- 172.21.15.37:0/3465525352 >> 172.21.15.37:0/3465525352 conn(0x7f34580b46e0 msgr2=0x7f34580d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.046+0000 7f3460400640 1 -- 172.21.15.37:0/3465525352 shutdown_connections 2024-09-21T06:31:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.046+0000 7f3460400640 1 -- 172.21.15.37:0/3465525352 wait complete. 2024-09-21T06:31:01.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.048+0000 7f3460400640 1 Processor -- start 2024-09-21T06:31:01.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.048+0000 7f3460400640 1 -- start start 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.049+0000 7f3460400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 0x7f34580cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.049+0000 7f3460400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345815ca90 0x7f345815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.049+0000 7f3460400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f3460400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3458167ca0 con 0x7f345815afe0 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f3460400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34580b6f00 con 0x7f345815ca90 2024-09-21T06:31:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f3460400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3458167e20 con 0x7f34580ccc40 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51898/0 (socket says 172.21.15.37:51898) 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 -- 172.21.15.37:0/3997234579 learned_addr learned my addr 172.21.15.37:0/3997234579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345d000640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345815ca90 0x7f345815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345da00640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 0x7f34580cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 -- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 msgr2=0x7f34580cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 0x7f34580cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 -- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345815ca90 msgr2=0x7f345815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345815ca90 0x7f345815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345e400640 1 -- 172.21.15.37:0/3997234579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345815ef10 con 0x7f345815afe0 2024-09-21T06:31:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.050+0000 7f345da00640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 0x7f34580cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f345e400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3450035040 tx=0x7f3450032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f3455600640 1 -- 172.21.15.37:0/3997234579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450037040 con 0x7f345815afe0 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f3455600640 1 -- 172.21.15.37:0/3997234579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3450042070 con 0x7f345815afe0 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345815f1f0 con 0x7f345815afe0 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f3455600640 1 -- 172.21.15.37:0/3997234579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450004260 con 0x7f345815afe0 2024-09-21T06:31:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.051+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34580b38b0 con 0x7f345815afe0 2024-09-21T06:31:01.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.054+0000 7f3455600640 1 -- 172.21.15.37:0/3997234579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3450004400 con 0x7f345815afe0 2024-09-21T06:31:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.055+0000 7f3455600640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 0x7f343807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.055+0000 7f3455600640 1 -- 172.21.15.37:0/3997234579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f34500bbdf0 con 0x7f345815afe0 2024-09-21T06:31:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.055+0000 7f345da00640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 0x7f343807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.055+0000 7f3467636740 1 Processor -- start 2024-09-21T06:31:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.055+0000 7f3467636740 1 -- start start 2024-09-21T06:31:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.056+0000 7f345da00640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 0x7f343807b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f60 tx=0x7f344c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee55370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dcbed3ec10 con 0x55dcbea626b0 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcbed9d9f0 con 0x55dcbea83390 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dcbed8ce20 con 0x55dcbea5a4d0 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51914/0 (socket says 172.21.15.37:51914) 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f345e400640 1 -- 172.21.15.37:0/4143136565 learned_addr learned my addr 172.21.15.37:0/4143136565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f345da00640 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.057+0000 7f345d000640 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee55370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 -- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 msgr2=0x55dcbee55370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee55370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 -- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 msgr2=0x55dcbee726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 -- 172.21.15.37:0/4143136565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbeb657d0 con 0x55dcbea626b0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f345e400640 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f34500ba0b0 tx=0x7f345002a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5b72ff0849838a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f344a000640 1 -- 172.21.15.37:0/4143136565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500834a0 con 0x55dcbea626b0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f344a000640 1 -- 172.21.15.37:0/4143136565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34500a7070 con 0x55dcbea626b0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.058+0000 7f344a000640 1 -- 172.21.15.37:0/4143136565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450009280 con 0x55dcbea626b0 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 -- 172.21.15.37:0/4143136565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 msgr2=0x55dcbee558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f34500ba0b0 tx=0x7f345002a820 comp rx=0 tx=0).stop 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 -- 172.21.15.37:0/4143136565 shutdown_connections 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee558b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee55370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 --2- 172.21.15.37:0/4143136565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 -- 172.21.15.37:0/4143136565 >> 172.21.15.37:0/4143136565 conn(0x55dcbea774e0 msgr2=0x55dcbee2b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 -- 172.21.15.37:0/4143136565 shutdown_connections 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.059+0000 7f3467636740 1 -- 172.21.15.37:0/4143136565 wait complete. 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.060+0000 7f3467636740 1 -- start start 2024-09-21T06:31:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.061+0000 7f3467636740 1 Processor -- start 2024-09-21T06:31:01.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee80380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dcbed32790 con 0x55dcbea626b0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcbed90c30 con 0x55dcbea83390 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dcbeafd740 con 0x55dcbea5a4d0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f3467636740 10 client.0 _ll_register_callbacks cb 0x55dcbed42260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f345d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f345d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51924/0 (socket says 172.21.15.37:51924) 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.062+0000 7f345d000640 1 -- 172.21.15.37:0/2303019310 learned_addr learned my addr 172.21.15.37:0/2303019310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345e400640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345da00640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee80380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 -- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 msgr2=0x55dcbee80380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee80380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 -- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 msgr2=0x55dcbee2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee2ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 -- 172.21.15.37:0/2303019310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbeacc6a0 con 0x55dcbea626b0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f345d000640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f34440102d0 tx=0x7f3444010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bb47361595eab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f343ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344401a020 con 0x55dcbea626b0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f3444016690 con 0x55dcbea626b0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.063+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34500a7070 con 0x55dcbea626b0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3444024050 con 0x55dcbea626b0 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 10 client.5943 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55dcbebc5f60 con 0x55dcbea626b0 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34500a7070 con 0x55dcbea626b0 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f3444004110 con 0x55dcbea626b0 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 10 client.5943 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 20 client.5943 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbea41f60 0x55dcbee2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 4 client.5943 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcbed9f940 con 0x55dcbea41f60 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.064+0000 7f345da00640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbea41f60 0x55dcbee2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.065+0000 7f345da00640 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbea41f60 0x55dcbee2b370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e775f73e462479 server_cookie=4d5d481a98ddbd1d in_seq=0 out_seq=0 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.065+0000 7f343ea00640 10 client.5943 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f343ea00640 1 -- 172.21.15.37:0/2303019310 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcbed9f940 con 0x55dcbea41f60 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f343ea00640 10 client.5943 handle_command_reply: tid=0 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 client.5943 shutdown 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 msgr2=0x55dcbee69050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f34440102d0 tx=0x7f3444010790 comp rx=0 tx=0).stop 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 shutdown_connections 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee69050 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbea41f60 0x55dcbee2b370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea83390 0x55dcbee2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 --2- 172.21.15.37:0/2303019310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea5a4d0 0x55dcbee80380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f343ea00640 0 client.5943 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 >> 172.21.15.37:0/2303019310 conn(0x55dcbea774e0 msgr2=0x55dcbee55370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 shutdown_connections 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 1 -- 172.21.15.37:0/2303019310 wait complete. 2024-09-21T06:31:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.066+0000 7f3467636740 20 client.5943 trim_cache size 0 max 16384 2024-09-21T06:31:01.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.085+0000 7f3467636740 1 Processor -- start 2024-09-21T06:31:01.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.085+0000 7f3467636740 1 -- start start 2024-09-21T06:31:01.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbee6a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 0x55dcbee72160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34582027b0 con 0x55dcbea626b0 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3458201980 con 0x55dcbea5a4d0 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3458202080 con 0x55dcbea83390 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f345d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f345d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51932/0 (socket says 172.21.15.37:51932) 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f345d000640 1 -- 172.21.15.37:0/923516049 learned_addr learned my addr 172.21.15.37:0/923516049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f345e400640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 0x55dcbee72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.086+0000 7f345da00640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbee6a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 -- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 msgr2=0x55dcbee72160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 0x55dcbee72160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 -- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 msgr2=0x55dcbee6a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbee6a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 -- 172.21.15.37:0/923516049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbeb63ad0 con 0x55dcbea626b0 2024-09-21T06:31:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345e400640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 0x55dcbee72160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345da00640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbee6a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f345d000640 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f344400b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbcaa164f957bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f3449600640 1 -- 172.21.15.37:0/923516049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344402d070 con 0x55dcbea626b0 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f3449600640 1 -- 172.21.15.37:0/923516049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34440167a0 con 0x55dcbea626b0 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.087+0000 7f3449600640 1 -- 172.21.15.37:0/923516049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344401a030 con 0x55dcbea626b0 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 -- 172.21.15.37:0/923516049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 msgr2=0x55dcbee71c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f344400b280 comp rx=0 tx=0).stop 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 -- 172.21.15.37:0/923516049 shutdown_connections 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea626b0 0x55dcbee71c20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea83390 0x55dcbee72160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 --2- 172.21.15.37:0/923516049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbee6a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 -- 172.21.15.37:0/923516049 >> 172.21.15.37:0/923516049 conn(0x55dcbea774e0 msgr2=0x55dcbee2c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 -- 172.21.15.37:0/923516049 shutdown_connections 2024-09-21T06:31:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.088+0000 7f3467636740 1 -- 172.21.15.37:0/923516049 wait complete. 2024-09-21T06:31:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.089+0000 7f3467636740 1 -- start start 2024-09-21T06:31:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.089+0000 7f3467636740 1 Processor -- start 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbeeef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 0x55dcbee6c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3458200a50 con 0x55dcbea83390 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcbee257e0 con 0x55dcbea5a4d0 2024-09-21T06:31:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dcbee87720 con 0x55dcbea626b0 2024-09-21T06:31:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f3467636740 10 client.0 _ll_register_callbacks cb 0x7f344400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51938/0 (socket says 172.21.15.37:51938) 2024-09-21T06:31:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 -- 172.21.15.37:0/1004360577 learned_addr learned my addr 172.21.15.37:0/1004360577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345da00640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbeeef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345d000640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 0x55dcbee6c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 -- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 msgr2=0x55dcbee6c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 0x55dcbee6c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 -- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 msgr2=0x55dcbeeef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbeeef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.091+0000 7f345e400640 1 -- 172.21.15.37:0/1004360577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbee6c1a0 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f345da00640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbeeef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f345d000640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 0x55dcbee6c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f345e400640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3450003fd0 tx=0x7f3450032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3869a4fa5f4f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345009d020 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f3450004b40 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34440167a0 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3450009280 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f3467636740 10 client.5949 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55dcbee87380 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34440167a0 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.092+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f345003a030 con 0x55dcbea83390 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 10 client.5949 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 10 client.5949 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 20 client.5949 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbee2c270 0x55dcbee71c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 4 client.5949 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-21T06:31:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55dcbed9b3f0 con 0x55dcbee2c270 2024-09-21T06:31:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f345da00640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbee2c270 0x55dcbee71c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f345da00640 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbee2c270 0x55dcbee71c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736651ceb7725174 server_cookie=7759c0106d3bb8f0 in_seq=0 out_seq=0 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.093+0000 7f343ea00640 10 client.5949 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f343ea00640 1 -- 172.21.15.37:0/1004360577 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcbed9b3f0 con 0x55dcbee2c270 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f343ea00640 10 client.5949 handle_command_reply: tid=0 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 client.5949 shutdown 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 msgr2=0x55dcbee86e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3450003fd0 tx=0x7f3450032e80 comp rx=0 tx=0).stop 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 shutdown_connections 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcbea83390 0x55dcbee86e40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55dcbee2c270 0x55dcbee71c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dcbea626b0 0x55dcbee6c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 --2- 172.21.15.37:0/1004360577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcbea5a4d0 0x55dcbeeef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f343ea00640 0 client.5949 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 >> 172.21.15.37:0/1004360577 conn(0x55dcbea774e0 msgr2=0x55dcbee25a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 shutdown_connections 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 1 -- 172.21.15.37:0/1004360577 wait complete. 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.094+0000 7f3467636740 20 client.5949 trim_cache size 0 max 16384 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:31:01.098 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 msgr2=0x7f343807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 0x7f343807b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f60 tx=0x7f344c009040 comp rx=0 tx=0).stop 2024-09-21T06:31:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 msgr2=0x7f345815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3450035040 tx=0x7f3450032d10 comp rx=0 tx=0).stop 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 shutdown_connections 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3438078b90 0x7f343807b0a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345815afe0 0x7f345815f6d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345815ca90 0x7f345815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 --2- 172.21.15.37:0/3997234579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34580ccc40 0x7f34580cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 >> 172.21.15.37:0/3997234579 conn(0x7f34580b46e0 msgr2=0x7f34580b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 shutdown_connections 2024-09-21T06:31:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.095+0000 7f3460400640 1 -- 172.21.15.37:0/3997234579 wait complete. 2024-09-21T06:31:01.118 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:31:01.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-21T06:31:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.349+0000 7f6b37200640 1 -- 172.21.15.37:0/3891260017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300cc530 msgr2=0x7f6b3015ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.349+0000 7f6b37200640 1 --2- 172.21.15.37:0/3891260017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300cc530 0x7f6b3015ead0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b30072a30 tx=0x7f6b2002a560 comp rx=0 tx=0).stop 2024-09-21T06:31:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 -- 172.21.15.37:0/3891260017 shutdown_connections 2024-09-21T06:31:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 --2- 172.21.15.37:0/3891260017 >> v2:172.21.15.150:3300/0 conn(0x7f6b3006e330 0x7f6b300ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 --2- 172.21.15.37:0/3891260017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300af5b0 0x7f6b3006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 --2- 172.21.15.37:0/3891260017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300cc530 0x7f6b3015ead0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 -- 172.21.15.37:0/3891260017 >> 172.21.15.37:0/3891260017 conn(0x7f6b300b1b90 msgr2=0x7f6b300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 -- 172.21.15.37:0/3891260017 shutdown_connections 2024-09-21T06:31:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.350+0000 7f6b37200640 1 -- 172.21.15.37:0/3891260017 wait complete. 2024-09-21T06:31:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.352+0000 7f6b37200640 1 Processor -- start 2024-09-21T06:31:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.352+0000 7f6b37200640 1 -- start start 2024-09-21T06:31:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.353+0000 7f6b37200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b3006e330 0x7f6b300b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.353+0000 7f6b37200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b37200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300cc530 0x7f6b301ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b37200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300cadb0 con 0x7f6b300af5b0 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b37200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300cac30 con 0x7f6b3006e330 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b37200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6b300caf30 con 0x7f6b300cc530 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51956/0 (socket says 172.21.15.37:51956) 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 -- 172.21.15.37:0/1766673829 learned_addr learned my addr 172.21.15.37:0/1766673829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b3006e330 0x7f6b300b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b35200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300cc530 0x7f6b301ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 -- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300cc530 msgr2=0x7f6b301ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300cc530 0x7f6b301ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 -- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b3006e330 msgr2=0x7f6b300b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b3006e330 0x7f6b300b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.354+0000 7f6b27400640 1 -- 172.21.15.37:0/1766673829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300b3000 con 0x7f6b300af5b0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b27400640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c970 tx=0x7f6b1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b2cc00640 1 -- 172.21.15.37:0/1766673829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18007bb0 con 0x7f6b300af5b0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b300b32e0 con 0x7f6b300af5b0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b2cc00640 1 -- 172.21.15.37:0/1766673829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b18007e40 con 0x7f6b300af5b0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b2cc00640 1 -- 172.21.15.37:0/1766673829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800f6b0 con 0x7f6b300af5b0 2024-09-21T06:31:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.355+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b300b38a0 con 0x7f6b300af5b0 2024-09-21T06:31:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.357+0000 7f6b2cc00640 1 -- 172.21.15.37:0/1766673829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6b1800f850 con 0x7f6b300af5b0 2024-09-21T06:31:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.358+0000 7f6b2cc00640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 0x7f6b0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.358+0000 7f6b2cc00640 1 -- 172.21.15.37:0/1766673829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f6b1809b7a0 con 0x7f6b300af5b0 2024-09-21T06:31:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.359+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 0x7f6b0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.359+0000 7f6b3e275740 1 Processor -- start 2024-09-21T06:31:01.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.359+0000 7f6b3e275740 1 -- start start 2024-09-21T06:31:01.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 0x7f6b0c07b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6b20033000 tx=0x7f6b20004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d544550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d527760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55629d45d1c0 con 0x55629d12d4d0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55629d470870 con 0x55629d1356b0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.360+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55629d45fca0 con 0x55629d156390 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51960/0 (socket says 172.21.15.37:51960) 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 -- 172.21.15.37:0/497459835 learned_addr learned my addr 172.21.15.37:0/497459835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d544550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b35200640 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d527760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 -- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 msgr2=0x55629d544550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d544550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 -- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 msgr2=0x55629d527760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d527760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 -- 172.21.15.37:0/497459835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d2387d0 con 0x55629d12d4d0 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.361+0000 7f6b27400640 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b18062c90 tx=0x7f6b18062a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827fd649239d9d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.362+0000 7f6b24c00640 1 -- 172.21.15.37:0/497459835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1801d040 con 0x55629d12d4d0 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.362+0000 7f6b24c00640 1 -- 172.21.15.37:0/497459835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b18063210 con 0x55629d12d4d0 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.362+0000 7f6b24c00640 1 -- 172.21.15.37:0/497459835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18067cf0 con 0x55629d12d4d0 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.362+0000 7f6b3e275740 1 -- 172.21.15.37:0/497459835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 msgr2=0x55629d527220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.362+0000 7f6b3e275740 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b18062c90 tx=0x7f6b18062a40 comp rx=0 tx=0).stop 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 -- 172.21.15.37:0/497459835 shutdown_connections 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d527220 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d527760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 --2- 172.21.15.37:0/497459835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d544550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 -- 172.21.15.37:0/497459835 >> 172.21.15.37:0/497459835 conn(0x55629d14a4e0 msgr2=0x55629d4fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 -- 172.21.15.37:0/497459835 shutdown_connections 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.363+0000 7f6b3e275740 1 -- 172.21.15.37:0/497459835 wait complete. 2024-09-21T06:31:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.364+0000 7f6b3e275740 1 -- start start 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.364+0000 7f6b3e275740 1 Processor -- start 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d12d4d0 0x55629d524470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55629d406410 con 0x55629d1356b0 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55629d463ab0 con 0x55629d156390 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55629d1d0740 con 0x55629d12d4d0 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.365+0000 7f6b3e275740 10 client.0 _ll_register_callbacks cb 0x55629d3e3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51974/0 (socket says 172.21.15.37:51974) 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 -- 172.21.15.37:0/699625607 learned_addr learned my addr 172.21.15.37:0/699625607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d12d4d0 0x55629d524470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b35200640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 -- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d12d4d0 msgr2=0x55629d524470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d12d4d0 0x55629d524470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 -- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 msgr2=0x55629d53ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 -- 172.21.15.37:0/699625607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629d1c1970 con 0x55629d1356b0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b35200640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b27400640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6b18063870 tx=0x7f6b1806c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ba84afc7d0bdb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1801d070 con 0x55629d1356b0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f6b1804ad60 con 0x55629d1356b0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.366+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b18063210 con 0x55629d1356b0 2024-09-21T06:31:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b18083050 con 0x55629d1356b0 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 10 client.5961 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55629d19f6a0 con 0x55629d1356b0 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b18063210 con 0x55629d1356b0 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f6b18062470 con 0x55629d1356b0 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 10 client.5961 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 10 client.5961 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 20 client.5961 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d114f60 0x55629d53c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 4 client.5961 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55629d4727c0 con 0x55629d114f60 2024-09-21T06:31:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.367+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d114f60 0x55629d53c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.368+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d114f60 0x55629d53c5f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df8bb9df9e84d08b server_cookie=cf3d45a3425bef2e in_seq=0 out_seq=0 2024-09-21T06:31:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.368+0000 7f6b1d600640 10 client.5961 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b1d600640 1 -- 172.21.15.37:0/699625607 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55629d4727c0 con 0x55629d114f60 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b1d600640 10 client.5961 handle_command_reply: tid=0 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 client.5961 shutdown 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 msgr2=0x55629d550b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6b18063870 tx=0x7f6b1806c9b0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 shutdown_connections 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d114f60 0x55629d53c5f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d1356b0 0x55629d550b10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 --2- 172.21.15.37:0/699625607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d12d4d0 0x55629d524470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b1d600640 0 client.5961 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 >> 172.21.15.37:0/699625607 conn(0x55629d14a4e0 msgr2=0x55629d517380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 shutdown_connections 2024-09-21T06:31:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 1 -- 172.21.15.37:0/699625607 wait complete. 2024-09-21T06:31:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.369+0000 7f6b3e275740 20 client.5961 trim_cache size 0 max 16384 2024-09-21T06:31:01.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.388+0000 7f6b3e275740 1 Processor -- start 2024-09-21T06:31:01.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.388+0000 7f6b3e275740 1 -- start start 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.389+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.389+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d523f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d524440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b3e275740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55629d4f9810 con 0x55629d12d4d0 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30202760 con 0x55629d1356b0 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6b30201950 con 0x55629d156390 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51976/0 (socket says 172.21.15.37:51976) 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 -- 172.21.15.37:0/3515597111 learned_addr learned my addr 172.21.15.37:0/3515597111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b27400640 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d523f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b35200640 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d524440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 -- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 msgr2=0x55629d524440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d524440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 -- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 msgr2=0x55629d523f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d523f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 -- 172.21.15.37:0/3515597111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d5450e0 con 0x55629d12d4d0 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.390+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6b2003e040 tx=0x7f6b2003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c6c634eae328e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b1fe00640 1 -- 172.21.15.37:0/3515597111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20035040 con 0x55629d12d4d0 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b1fe00640 1 -- 172.21.15.37:0/3515597111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b20003df0 con 0x55629d12d4d0 2024-09-21T06:31:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b1fe00640 1 -- 172.21.15.37:0/3515597111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200049c0 con 0x55629d12d4d0 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 -- 172.21.15.37:0/3515597111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 msgr2=0x55629d53cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6b2003e040 tx=0x7f6b2003c000 comp rx=0 tx=0).stop 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 -- 172.21.15.37:0/3515597111 shutdown_connections 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d156390 0x55629d524440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d1356b0 0x55629d523f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 --2- 172.21.15.37:0/3515597111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d53cbf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.391+0000 7f6b3e275740 1 -- 172.21.15.37:0/3515597111 >> 172.21.15.37:0/3515597111 conn(0x55629d14a4e0 msgr2=0x55629d4f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.392+0000 7f6b3e275740 1 -- 172.21.15.37:0/3515597111 shutdown_connections 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.392+0000 7f6b3e275740 1 -- 172.21.15.37:0/3515597111 wait complete. 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.393+0000 7f6b3e275740 1 -- start start 2024-09-21T06:31:01.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.393+0000 7f6b3e275740 1 Processor -- start 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 0x55629d53b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301f96e0 con 0x55629d12d4d0 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30202030 con 0x55629d156390 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55629d4fd9a0 con 0x55629d1356b0 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b3e275740 10 client.0 _ll_register_callbacks cb 0x7f6b18062b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51980/0 (socket says 172.21.15.37:51980) 2024-09-21T06:31:01.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.394+0000 7f6b2fe00640 1 -- 172.21.15.37:0/973721592 learned_addr learned my addr 172.21.15.37:0/973721592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b27400640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 0x55629d53b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b35200640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 -- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 msgr2=0x55629d53b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 0x55629d53b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 -- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 msgr2=0x55629d53ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 -- 172.21.15.37:0/973721592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629d53c280 con 0x55629d12d4d0 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b27400640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 0x55629d53b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b2fe00640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6b2002ab50 tx=0x7f6b20005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ff699bc6e063be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:01.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:01.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20035050 con 0x55629d12d4d0 2024-09-21T06:31:01.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f6b20066c90 con 0x55629d12d4d0 2024-09-21T06:31:01.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.395+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b20003df0 con 0x55629d12d4d0 2024-09-21T06:31:01.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b200047a0 con 0x55629d12d4d0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 10 client.5967 fetch_fsmap learned FSMap version 40 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55629d5c1830 con 0x55629d12d4d0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b20003df0 con 0x55629d12d4d0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 3554+0+0 (secure 0 0 0) 0x7f6b2003a040 con 0x55629d12d4d0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 10 client.5967 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 10 client.5967 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 20 client.5967 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d4fd9a0 0x55629d563c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 4 client.5967 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55629d46e270 con 0x55629d4fd9a0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.396+0000 7f6b27400640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d4fd9a0 0x55629d563c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b27400640 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d4fd9a0 0x55629d563c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6443a2fe17f6c683 server_cookie=7d6c4116a69b5e8c in_seq=0 out_seq=0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b1d600640 10 client.5967 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b1d600640 1 -- 172.21.15.37:0/973721592 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55629d46e270 con 0x55629d4fd9a0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b1d600640 10 client.5967 handle_command_reply: tid=0 2024-09-21T06:31:01.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 client.5967 shutdown 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 msgr2=0x55629d5c00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6b2002ab50 tx=0x7f6b20005d70 comp rx=0 tx=0).stop 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 shutdown_connections 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x55629d4fd9a0 0x55629d563c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55629d156390 0x55629d53ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55629d1356b0 0x55629d53b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 --2- 172.21.15.37:0/973721592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55629d12d4d0 0x55629d5c00d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b1d600640 0 client.5967 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.397+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 >> 172.21.15.37:0/973721592 conn(0x55629d14a4e0 msgr2=0x55629d4f6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 shutdown_connections 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b3e275740 1 -- 172.21.15.37:0/973721592 wait complete. 2024-09-21T06:31:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b3e275740 20 client.5967 trim_cache size 0 max 16384 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "69a4999b-afdb-4309-9e01-a1f19558c26d": { 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "69a4999b-afdb-4309-9e01-a1f19558c26d", 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:01.402 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:01.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 msgr2=0x7f6b0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 0x7f6b0c07b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6b20033000 tx=0x7f6b20004730 comp rx=0 tx=0).stop 2024-09-21T06:31:01.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 msgr2=0x7f6b3009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c970 tx=0x7f6b1800ce30 comp rx=0 tx=0).stop 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 shutdown_connections 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6b0c078b90 0x7f6b0c07b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b300af5b0 0x7f6b3009c480 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b300cc530 0x7f6b301ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 --2- 172.21.15.37:0/1766673829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b3006e330 0x7f6b300b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.398+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 >> 172.21.15.37:0/1766673829 conn(0x7f6b300b1b90 msgr2=0x7f6b3009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.399+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 shutdown_connections 2024-09-21T06:31:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:01.399+0000 7f6b37200640 1 -- 172.21.15.37:0/1766673829 wait complete. 2024-09-21T06:31:02.215 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:31:02.214+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:31:06.420 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:31:06.421 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 -- 172.21.15.37:0/632071407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd968003c10 msgr2=0x7fd968004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 --2- 172.21.15.37:0/632071407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd968003c10 0x7fd968004060 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c4a0 tx=0x7fd968005830 comp rx=0 tx=0).stop 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd986200640 1 -- 172.21.15.37:0/632071407 reap_dead start 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 -- 172.21.15.37:0/632071407 shutdown_connections 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 -- 172.21.15.37:0/632071407 >> 172.21.15.37:0/632071407 conn(0x7fd9800b1b90 msgr2=0x7fd9800b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 -- 172.21.15.37:0/632071407 shutdown_connections 2024-09-21T06:31:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.659+0000 7fd988200640 1 -- 172.21.15.37:0/632071407 wait complete. 2024-09-21T06:31:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.662+0000 7fd988200640 1 Processor -- start 2024-09-21T06:31:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.662+0000 7fd988200640 1 -- start start 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd968003c10 0x7fd9800c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9800ce8b0 0x7fd9801f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd980160fd0 con 0x7fd9800c5f20 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd980160e50 con 0x7fd968003c10 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.663+0000 7fd988200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd98006e200 con 0x7fd9800ce8b0 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd985800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd968003c10 0x7fd9800c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56974/0 (socket says 172.21.15.37:56974) 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 -- 172.21.15.37:0/57854216 learned_addr learned my addr 172.21.15.37:0/57854216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd984e00640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9800ce8b0 0x7fd9801f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 -- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9800ce8b0 msgr2=0x7fd9801f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9800ce8b0 0x7fd9801f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 -- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd968003c10 msgr2=0x7fd9800c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd968003c10 0x7fd9800c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 -- 172.21.15.37:0/57854216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9800ce3b0 con 0x7fd9800c5f20 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd986200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd978001330 tx=0x7fd978001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.664+0000 7fd97d600640 1 -- 172.21.15.37:0/57854216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978011870 con 0x7fd9800c5f20 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.665+0000 7fd97d600640 1 -- 172.21.15.37:0/57854216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd978017070 con 0x7fd9800c5f20 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.665+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801f7260 con 0x7fd9800c5f20 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.665+0000 7fd97d600640 1 -- 172.21.15.37:0/57854216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978011dc0 con 0x7fd9800c5f20 2024-09-21T06:31:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.665+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9800b2ed0 con 0x7fd9800c5f20 2024-09-21T06:31:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.667+0000 7fd97d600640 1 -- 172.21.15.37:0/57854216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd97801c460 con 0x7fd9800c5f20 2024-09-21T06:31:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.668+0000 7fd97d600640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.668+0000 7fd97d600640 1 -- 172.21.15.37:0/57854216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fd97809b730 con 0x7fd9800c5f20 2024-09-21T06:31:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.669+0000 7fd985800640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.669+0000 7fd98f1f5740 1 Processor -- start 2024-09-21T06:31:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.669+0000 7fd98f1f5740 1 -- start start 2024-09-21T06:31:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.669+0000 7fd985800640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 0x7fd96007b0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd974033000 tx=0x7fd97402d070 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117f50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559117e861c0 con 0x559117b7f390 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559117e99870 con 0x559117b564d0 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.670+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x559117e88ca0 con 0x559117b5e6b0 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56982/0 (socket says 172.21.15.37:56982) 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 -- 172.21.15.37:0/4102911060 learned_addr learned my addr 172.21.15.37:0/4102911060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd984e00640 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117f50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd986200640 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 -- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 msgr2=0x559117f50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 -- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 msgr2=0x559117f50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117f50220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 -- 172.21.15.37:0/4102911060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559117c617d0 con 0x559117b7f390 2024-09-21T06:31:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.671+0000 7fd985800640 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd9740062c0 tx=0x7fd974038070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6196c8145e2bbfa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.672+0000 7fd972000640 1 -- 172.21.15.37:0/4102911060 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9740036b0 con 0x559117b7f390 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.672+0000 7fd972000640 1 -- 172.21.15.37:0/4102911060 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd974003850 con 0x559117b7f390 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.672+0000 7fd972000640 1 -- 172.21.15.37:0/4102911060 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974003c80 con 0x559117b7f390 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.672+0000 7fd98f1f5740 1 -- 172.21.15.37:0/4102911060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 msgr2=0x559117f6d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.672+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd9740062c0 tx=0x7fd974038070 comp rx=0 tx=0).stop 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 -- 172.21.15.37:0/4102911060 shutdown_connections 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117f50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/4102911060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f6d550 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 -- 172.21.15.37:0/4102911060 >> 172.21.15.37:0/4102911060 conn(0x559117b734e0 msgr2=0x559117f26440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 -- 172.21.15.37:0/4102911060 shutdown_connections 2024-09-21T06:31:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.673+0000 7fd98f1f5740 1 -- 172.21.15.37:0/4102911060 wait complete. 2024-09-21T06:31:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.674+0000 7fd98f1f5740 1 -- start start 2024-09-21T06:31:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.674+0000 7fd98f1f5740 1 Processor -- start 2024-09-21T06:31:06.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b564d0 0x559117f4b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 0x559117f79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559117e2f410 con 0x559117b7f390 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559117e8cab0 con 0x559117b5e6b0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.675+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x559117bf9740 con 0x559117b564d0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd98f1f5740 10 client.0 _ll_register_callbacks cb 0x559117e0c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56990/0 (socket says 172.21.15.37:56990) 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 -- 172.21.15.37:0/872831892 learned_addr learned my addr 172.21.15.37:0/872831892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd985800640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b564d0 0x559117f4b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd984e00640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 0x559117f79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 -- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b564d0 msgr2=0x559117f4b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b564d0 0x559117f4b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 -- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 msgr2=0x559117f79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 0x559117f79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 -- 172.21.15.37:0/872831892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559117bea970 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd984e00640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 0x559117f79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd986200640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd978062f20 tx=0x7fd978062a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3eb15012dc35eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.676+0000 7fd966a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978063220 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fd97804ab60 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd974003850 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd978067d00 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 10 client.5979 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x559117bc86a0 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd974003850 con 0x559117b7f390 2024-09-21T06:31:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7fd97805eb70 con 0x559117b7f390 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 10 client.5979 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 10 client.5979 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 20 client.5979 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117b3df60 0x559117f4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 4 client.5979 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.677+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559117e9b7c0 con 0x559117b3df60 2024-09-21T06:31:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.678+0000 7fd985800640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117b3df60 0x559117f4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.678+0000 7fd985800640 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117b3df60 0x559117f4cbf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9f05acf9df63094 server_cookie=c47ffb8e4abbbb18 in_seq=0 out_seq=0 2024-09-21T06:31:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.678+0000 7fd966a00640 10 client.5979 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd966a00640 1 -- 172.21.15.37:0/872831892 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559117e9b7c0 con 0x559117b3df60 2024-09-21T06:31:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd966a00640 10 client.5979 handle_command_reply: tid=0 2024-09-21T06:31:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 client.5979 shutdown 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 msgr2=0x559117f49620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd978062f20 tx=0x7fd978062a00 comp rx=0 tx=0).stop 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 shutdown_connections 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117b3df60 0x559117f4cbf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f49620 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b5e6b0 0x559117f79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/872831892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b564d0 0x559117f4b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd966a00640 0 client.5979 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 >> 172.21.15.37:0/872831892 conn(0x559117b734e0 msgr2=0x559117f3f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 shutdown_connections 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.679+0000 7fd98f1f5740 1 -- 172.21.15.37:0/872831892 wait complete. 2024-09-21T06:31:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.680+0000 7fd98f1f5740 20 client.5979 trim_cache size 0 max 16384 2024-09-21T06:31:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.698+0000 7fd98f1f5740 1 Processor -- start 2024-09-21T06:31:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.698+0000 7fd98f1f5740 1 -- start start 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f25850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b7f390 0x559117f1bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd980200ac0 con 0x559117b564d0 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801ffc60 con 0x559117b7f390 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.699+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd980200390 con 0x559117b5e6b0 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56998/0 (socket says 172.21.15.37:56998) 2024-09-21T06:31:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 -- 172.21.15.37:0/855312535 learned_addr learned my addr 172.21.15.37:0/855312535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd984e00640 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd986200640 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b7f390 0x559117f1bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 -- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 msgr2=0x559117f25850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f25850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 -- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b7f390 msgr2=0x559117f1bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b7f390 0x559117f1bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 -- 172.21.15.37:0/855312535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559117f20aa0 con 0x559117b564d0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.700+0000 7fd985800640 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c500 tx=0x7fd974042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5157736343549f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.701+0000 7fd971600640 1 -- 172.21.15.37:0/855312535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974004850 con 0x559117b564d0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.701+0000 7fd971600640 1 -- 172.21.15.37:0/855312535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9740049f0 con 0x559117b564d0 2024-09-21T06:31:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.701+0000 7fd971600640 1 -- 172.21.15.37:0/855312535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974004e20 con 0x559117b564d0 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.701+0000 7fd98f1f5740 1 -- 172.21.15.37:0/855312535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 msgr2=0x559117f4ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.701+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c500 tx=0x7fd974042040 comp rx=0 tx=0).stop 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 -- 172.21.15.37:0/855312535 shutdown_connections 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b7f390 0x559117f1bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f25850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/855312535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b564d0 0x559117f4ae60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 -- 172.21.15.37:0/855312535 >> 172.21.15.37:0/855312535 conn(0x559117b734e0 msgr2=0x559117f6b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 -- 172.21.15.37:0/855312535 shutdown_connections 2024-09-21T06:31:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.702+0000 7fd98f1f5740 1 -- 172.21.15.37:0/855312535 wait complete. 2024-09-21T06:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.703+0000 7fd98f1f5740 1 -- start start 2024-09-21T06:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.703+0000 7fd98f1f5740 1 Processor -- start 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117fed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f3f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801fecb0 con 0x559117b7f390 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559117f3d7d0 con 0x559117b564d0 2024-09-21T06:31:06.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x559117f4a800 con 0x559117b5e6b0 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.704+0000 7fd98f1f5740 10 client.0 _ll_register_callbacks cb 0x7fd978014040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57008/0 (socket says 172.21.15.37:57008) 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 -- 172.21.15.37:0/1735379439 learned_addr learned my addr 172.21.15.37:0/1735379439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd984e00640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f3f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd985800640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117fed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 -- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 msgr2=0x559117f3f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f3f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 -- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 msgr2=0x559117fed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117fed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 -- 172.21.15.37:0/1735379439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559117f4a5e0 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd986200640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd9780069d0 tx=0x7fd97804ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383fac54d7c00dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.705+0000 7fd966a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978063220 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fd978065db0 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9740049f0 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd978062b70 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 10 client.5985 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x559117f4a5e0 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9740049f0 con 0x559117b7f390 2024-09-21T06:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7fd978062100 con 0x559117b7f390 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 10 client.5985 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 10 client.5985 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 20 client.5985 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117f646a0 0x559117f7c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 4 client.5985 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.706+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559117f4a3d0 con 0x559117f646a0 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd985800640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117f646a0 0x559117f7c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd985800640 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117f646a0 0x559117f7c4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11d6de8b1bfdc921 server_cookie=5811f89a276a4d19 in_seq=0 out_seq=0 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd966a00640 10 client.5985 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd966a00640 1 -- 172.21.15.37:0/1735379439 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x559117f4a3d0 con 0x559117f646a0 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd966a00640 10 client.5985 handle_command_reply: tid=0 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.707+0000 7fd98f1f5740 1 client.5985 shutdown 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 msgr2=0x559117f3fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd9780069d0 tx=0x7fd97804ad70 comp rx=0 tx=0).stop 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 shutdown_connections 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x559117f646a0 0x559117f7c4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559117b7f390 0x559117f3fc70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559117b5e6b0 0x559117f3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 --2- 172.21.15.37:0/1735379439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559117b564d0 0x559117fed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd966a00640 0 client.5985 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 >> 172.21.15.37:0/1735379439 conn(0x559117b734e0 msgr2=0x559117f1f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 shutdown_connections 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 1 -- 172.21.15.37:0/1735379439 wait complete. 2024-09-21T06:31:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.708+0000 7fd98f1f5740 20 client.5985 trim_cache size 0 max 16384 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "69a4999b-afdb-4309-9e01-a1f19558c26d": { 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "69a4999b-afdb-4309-9e01-a1f19558c26d", 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:06.713 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 msgr2=0x7fd96007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 0x7fd96007b0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd974033000 tx=0x7fd97402d070 comp rx=0 tx=0).stop 2024-09-21T06:31:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 msgr2=0x7fd9801f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd978001330 tx=0x7fd978001b80 comp rx=0 tx=0).stop 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 shutdown_connections 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9800c5f20 0x7fd9801f6af0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9800ce8b0 0x7fd9801f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 --2- 172.21.15.37:0/57854216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd968003c10 0x7fd9800c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 >> 172.21.15.37:0/57854216 conn(0x7fd9800b1b90 msgr2=0x7fd98015fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 shutdown_connections 2024-09-21T06:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:06.709+0000 7fd988200640 1 -- 172.21.15.37:0/57854216 wait complete. 2024-09-21T06:31:11.732 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:31:11.733 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-21T06:31:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.973+0000 7f2150800640 1 -- 172.21.15.37:0/2868038422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 msgr2=0x7f2130005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.973+0000 7f2150800640 1 --2- 172.21.15.37:0/2868038422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f2130005220 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f214400c740 tx=0x7f2130005830 comp rx=0 tx=0).stop 2024-09-21T06:31:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.973+0000 7f214e800640 1 -- 172.21.15.37:0/2868038422 reap_dead start 2024-09-21T06:31:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.973+0000 7f2150800640 1 -- 172.21.15.37:0/2868038422 shutdown_connections 2024-09-21T06:31:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.973+0000 7f2150800640 1 -- 172.21.15.37:0/2868038422 >> 172.21.15.37:0/2868038422 conn(0x7f21480b1b90 msgr2=0x7f21480b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:11.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.974+0000 7f2150800640 1 -- 172.21.15.37:0/2868038422 shutdown_connections 2024-09-21T06:31:11.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.974+0000 7f2150800640 1 -- 172.21.15.37:0/2868038422 wait complete. 2024-09-21T06:31:11.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.976+0000 7f2150800640 1 Processor -- start 2024-09-21T06:31:11.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.976+0000 7f2150800640 1 -- start start 2024-09-21T06:31:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.977+0000 7f2150800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f2150800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21480a0d70 0x7f21480b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f2150800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 0x7f214809e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f2150800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f214806ddf0 con 0x7f2130004d90 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f2150800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21480ca990 con 0x7f214809e170 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f2150800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f214806df70 con 0x7f21480a0d70 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57018/0 (socket says 172.21.15.37:57018) 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 -- 172.21.15.37:0/3460425363 learned_addr learned my addr 172.21.15.37:0/3460425363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214e800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 0x7f214809e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214d400640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21480a0d70 0x7f21480b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 -- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21480a0d70 msgr2=0x7f21480b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21480a0d70 0x7f21480b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 -- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 msgr2=0x7f214809e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 0x7f214809e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214de00640 1 -- 172.21.15.37:0/3460425363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21480b4c50 con 0x7f2130004d90 2024-09-21T06:31:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.978+0000 7f214e800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 0x7f214809e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:11.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f214de00640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f213c011ae0 tx=0x7f213c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:11.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f2142000640 1 -- 172.21.15.37:0/3460425363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c01e070 con 0x7f2130004d90 2024-09-21T06:31:11.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f2142000640 1 -- 172.21.15.37:0/3460425363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f213c00fe70 con 0x7f2130004d90 2024-09-21T06:31:11.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f2142000640 1 -- 172.21.15.37:0/3460425363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c004290 con 0x7f2130004d90 2024-09-21T06:31:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214809eac0 con 0x7f2130004d90 2024-09-21T06:31:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.979+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21481f5790 con 0x7f2130004d90 2024-09-21T06:31:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.981+0000 7f2142000640 1 -- 172.21.15.37:0/3460425363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f213c021070 con 0x7f2130004d90 2024-09-21T06:31:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.983+0000 7f2142000640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 0x7f212c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.983+0000 7f2142000640 1 -- 172.21.15.37:0/3460425363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f213c09b990 con 0x7f2130004d90 2024-09-21T06:31:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.984+0000 7f214d400640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 0x7f212c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.984+0000 7f2157897740 1 Processor -- start 2024-09-21T06:31:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.984+0000 7f2157897740 1 -- start start 2024-09-21T06:31:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.984+0000 7f214d400640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 0x7f212c07b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f21480cec20 tx=0x7f2138006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e8e390 0x5596f227c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f225f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596f21951c0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5596f21a8870 con 0x5596f1e8e390 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.985+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5596f2197ca0 con 0x5596f1e654d0 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e8e390 0x5596f227c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57030/0 (socket says 172.21.15.37:57030) 2024-09-21T06:31:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 -- 172.21.15.37:0/1337447165 learned_addr learned my addr 172.21.15.37:0/1337447165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214d400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f225f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 -- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 msgr2=0x5596f225f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f225f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 -- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e8e390 msgr2=0x5596f227c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e8e390 0x5596f227c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 -- 172.21.15.37:0/1337447165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f1f707d0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.986+0000 7f214e800640 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f214402b5c0 tx=0x7f21480cade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c0cf348bc68632 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2136600640 1 -- 172.21.15.37:0/1337447165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144001a40 con 0x5596f1e6d6b0 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2136600640 1 -- 172.21.15.37:0/1337447165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2144001be0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2136600640 1 -- 172.21.15.37:0/1337447165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144004da0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 -- 172.21.15.37:0/1337447165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 msgr2=0x5596f225f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:11.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f214402b5c0 tx=0x7f21480cade0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 -- 172.21.15.37:0/1337447165 shutdown_connections 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f225f760 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f225f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 --2- 172.21.15.37:0/1337447165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e8e390 0x5596f227c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.987+0000 7f2157897740 1 -- 172.21.15.37:0/1337447165 >> 172.21.15.37:0/1337447165 conn(0x5596f1e824e0 msgr2=0x5596f2235440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.988+0000 7f2157897740 1 -- 172.21.15.37:0/1337447165 shutdown_connections 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.988+0000 7f2157897740 1 -- 172.21.15.37:0/1337447165 wait complete. 2024-09-21T06:31:11.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.989+0000 7f2157897740 1 -- start start 2024-09-21T06:31:11.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.989+0000 7f2157897740 1 Processor -- start 2024-09-21T06:31:11.991 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:31:11.990+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:31:11.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f22349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e8e390 0x5596f224d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596f213e410 con 0x5596f1e6d6b0 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5596f219bab0 con 0x5596f1e654d0 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5596f1f08740 con 0x5596f1e8e390 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f2157897740 10 client.0 _ll_register_callbacks cb 0x5596f211b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57036/0 (socket says 172.21.15.37:57036) 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 -- 172.21.15.37:0/3210545692 learned_addr learned my addr 172.21.15.37:0/3210545692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214de00640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f22349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214e800640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e8e390 0x5596f224d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 -- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e8e390 msgr2=0x5596f224d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e8e390 0x5596f224d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 -- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 msgr2=0x5596f22349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f22349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.991+0000 7f214d400640 1 -- 172.21.15.37:0/3210545692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f1ef9970 con 0x5596f1e6d6b0 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f214d400640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2138001dc0 tx=0x7f2138001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e989f3e719b649a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f212aa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213801f400 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f2138018af0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2144001be0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f213802f050 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f2157897740 10 client.5997 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5596f1ed76a0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.992+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2144001be0 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f2138032020 con 0x5596f1e6d6b0 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 10 client.5997 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 10 client.5997 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 20 client.5997 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:11.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f1e4cf60 0x5596f2275f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 4 client.5997 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f21aa7c0 con 0x5596f1e4cf60 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f214de00640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f1e4cf60 0x5596f2275f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f214de00640 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f1e4cf60 0x5596f2275f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb4ff5098f84852a server_cookie=a4b03db84a89b54e in_seq=0 out_seq=0 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.993+0000 7f212aa00640 10 client.5997 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.994+0000 7f212aa00640 1 -- 172.21.15.37:0/3210545692 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596f21aa7c0 con 0x5596f1e4cf60 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.994+0000 7f212aa00640 10 client.5997 handle_command_reply: tid=0 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.994+0000 7f2157897740 1 client.5997 shutdown 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.994+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 msgr2=0x5596f2278b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.994+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2138001dc0 tx=0x7f2138001360 comp rx=0 tx=0).stop 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 shutdown_connections 2024-09-21T06:31:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f1e4cf60 0x5596f2275f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e6d6b0 0x5596f2278b70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e8e390 0x5596f224d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 --2- 172.21.15.37:0/3210545692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f22349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 >> 172.21.15.37:0/3210545692 conn(0x5596f1e824e0 msgr2=0x5596f225c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f212aa00640 0 client.5997 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 shutdown_connections 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 1 -- 172.21.15.37:0/3210545692 wait complete. 2024-09-21T06:31:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:11.995+0000 7f2157897740 20 client.5997 trim_cache size 0 max 16384 2024-09-21T06:31:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.014+0000 7f2157897740 1 Processor -- start 2024-09-21T06:31:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.014+0000 7f2157897740 1 -- start start 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f2236670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e6d6b0 0x5596f227d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2148202dd0 con 0x5596f1e8e390 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2148201f70 con 0x5596f1e654d0 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.015+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f21482026a0 con 0x5596f1e6d6b0 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57050/0 (socket says 172.21.15.37:57050) 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 -- 172.21.15.37:0/1484890627 learned_addr learned my addr 172.21.15.37:0/1484890627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214de00640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f2236670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214d400640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e6d6b0 0x5596f227d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 -- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e6d6b0 msgr2=0x5596f227d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e6d6b0 0x5596f227d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 -- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 msgr2=0x5596f2236670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f2236670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 -- 172.21.15.37:0/1484890627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f224cfc0 con 0x5596f1e8e390 2024-09-21T06:31:12.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214de00640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f2236670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.016+0000 7f214e800640 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f214400c2f0 tx=0x7f2130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90ac5e66064e2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2135c00640 1 -- 172.21.15.37:0/1484890627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144004b40 con 0x5596f1e8e390 2024-09-21T06:31:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2135c00640 1 -- 172.21.15.37:0/1484890627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214402f070 con 0x5596f1e8e390 2024-09-21T06:31:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2135c00640 1 -- 172.21.15.37:0/1484890627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214402bb70 con 0x5596f1e8e390 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 -- 172.21.15.37:0/1484890627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 msgr2=0x5596f2279500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f214400c2f0 tx=0x7f2130005830 comp rx=0 tx=0).stop 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 -- 172.21.15.37:0/1484890627 shutdown_connections 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f2279500 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e6d6b0 0x5596f227d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 --2- 172.21.15.37:0/1484890627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e654d0 0x5596f2236670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.017+0000 7f2157897740 1 -- 172.21.15.37:0/1484890627 >> 172.21.15.37:0/1484890627 conn(0x5596f1e824e0 msgr2=0x5596f225b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.018+0000 7f2157897740 1 -- 172.21.15.37:0/1484890627 shutdown_connections 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.018+0000 7f2157897740 1 -- 172.21.15.37:0/1484890627 wait complete. 2024-09-21T06:31:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.018+0000 7f2157897740 1 -- start start 2024-09-21T06:31:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.019+0000 7f2157897740 1 Processor -- start 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f22f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 0x5596f227ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596f2236cd0 con 0x5596f1e8e390 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21480c93d0 con 0x5596f1e6d6b0 2024-09-21T06:31:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5596f228ce70 con 0x5596f1e654d0 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f2157897740 10 client.0 _ll_register_callbacks cb 0x7f21380014f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f214e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57062/0 (socket says 172.21.15.37:57062) 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.020+0000 7f214e800640 1 -- 172.21.15.37:0/2167692309 learned_addr learned my addr 172.21.15.37:0/2167692309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214de00640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f22f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214d400640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 0x5596f227ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 -- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 msgr2=0x5596f22f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f22f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 -- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 msgr2=0x5596f227ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 0x5596f227ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 -- 172.21.15.37:0/2167692309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f224d9b0 con 0x5596f1e8e390 2024-09-21T06:31:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214de00640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f22f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214d400640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 0x5596f227ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f214e800640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f214400c2f0 tx=0x7f2144001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b76b8ec50f3eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f212aa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144004b40 con 0x5596f1e8e390 2024-09-21T06:31:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f2144004dd0 con 0x5596f1e8e390 2024-09-21T06:31:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.021+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214402f070 con 0x5596f1e8e390 2024-09-21T06:31:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f214405b050 con 0x5596f1e8e390 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 10 client.6003 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5596f227b240 con 0x5596f1e8e390 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214402f070 con 0x5596f1e8e390 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f214405e020 con 0x5596f1e8e390 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 10 client.6003 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 10 client.6003 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 20 client.6003 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f229c470 0x5596f2274b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 4 client.6003 mds_command: new command op to 5506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 --> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596f21a6270 con 0x5596f229c470 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.022+0000 7f214de00640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f229c470 0x5596f2274b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f214de00640 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f229c470 0x5596f2274b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b0a652d02c9a02c server_cookie=e0b324ff1fbae755 in_seq=0 out_seq=0 2024-09-21T06:31:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f212aa00640 10 client.6003 ms_handle_connect on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f212aa00640 1 -- 172.21.15.37:0/2167692309 <== mds.1 v2:172.21.15.37:6833/1998484049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596f21a6270 con 0x5596f229c470 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f212aa00640 10 client.6003 handle_command_reply: tid=0 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 client.6003 shutdown 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 msgr2=0x5596f22745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f214400c2f0 tx=0x7f2144001720 comp rx=0 tx=0).stop 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 shutdown_connections 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:6833/1998484049,v1:172.21.15.37:6835/1998484049] conn(0x5596f229c470 0x5596f2274b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596f1e8e390 0x5596f22745f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596f1e6d6b0 0x5596f227ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 --2- 172.21.15.37:0/2167692309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5596f1e654d0 0x5596f22f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f212aa00640 0 client.6003 ms_handle_reset on v2:172.21.15.37:6833/1998484049 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.023+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 >> 172.21.15.37:0/2167692309 conn(0x5596f1e824e0 msgr2=0x5596f225b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 shutdown_connections 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2157897740 1 -- 172.21.15.37:0/2167692309 wait complete. 2024-09-21T06:31:12.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2157897740 20 client.6003 trim_cache size 0 max 16384 2024-09-21T06:31:12.029 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:12.029 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-21T06:31:12.029 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-21T06:31:12.029 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 msgr2=0x7f212c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 0x7f212c07b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f21480cec20 tx=0x7f2138006b10 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 msgr2=0x7f214815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f213c011ae0 tx=0x7f213c009e90 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 shutdown_connections 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f212c078b90 0x7f212c07b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f214809e170 0x7f214809e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21480a0d70 0x7f21480b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.024+0000 7f2150800640 1 --2- 172.21.15.37:0/3460425363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130004d90 0x7f214815e540 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.025+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 >> 172.21.15.37:0/3460425363 conn(0x7f21480b1b90 msgr2=0x7f214806d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.025+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 shutdown_connections 2024-09-21T06:31:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.025+0000 7f2150800640 1 -- 172.21.15.37:0/3460425363 wait complete. 2024-09-21T06:31:12.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:31:12.051 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:31:12.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:2 scrub status 2024-09-21T06:31:12.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 -- 172.21.15.37:0/3892914886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8004d90 msgr2=0x7f8fc8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3892914886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8004d90 0x7f8fc8005220 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009ed0 tx=0x7f8fc8005830 comp rx=0 tx=0).stop 2024-09-21T06:31:12.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe5600640 1 -- 172.21.15.37:0/3892914886 reap_dead start 2024-09-21T06:31:12.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 -- 172.21.15.37:0/3892914886 shutdown_connections 2024-09-21T06:31:12.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 -- 172.21.15.37:0/3892914886 >> 172.21.15.37:0/3892914886 conn(0x7f8fe006f920 msgr2=0x7f8fe00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 -- 172.21.15.37:0/3892914886 shutdown_connections 2024-09-21T06:31:12.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.281+0000 7f8fe6000640 1 -- 172.21.15.37:0/3892914886 wait complete. 2024-09-21T06:31:12.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.283+0000 7f8fe6000640 1 Processor -- start 2024-09-21T06:31:12.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.283+0000 7f8fe6000640 1 -- start start 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 0x7f8fe01e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 0x7f8fe01f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe00cd070 con 0x7f8fe00a9e50 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe00ccef0 con 0x7f8fe01f3b10 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe6000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe00cddc0 con 0x7f8fc8004d90 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57080/0 (socket says 172.21.15.37:57080) 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3011228423 learned_addr learned my addr 172.21.15.37:0/3011228423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 0x7f8fe01e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.285+0000 7f8fe5600640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 0x7f8fe01f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 msgr2=0x7f8fe01e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 0x7f8fe01e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 msgr2=0x7f8fe01f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 0x7f8fe01f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3011228423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe01f7090 con 0x7f8fe00a9e50 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 0x7f8fe01e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fe5600640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 0x7f8fe01f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc006600 tx=0x7f8fcc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdcc00640 1 -- 172.21.15.37:0/3011228423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc015070 con 0x7f8fe00a9e50 2024-09-21T06:31:12.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdcc00640 1 -- 172.21.15.37:0/3011228423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc011900 con 0x7f8fe00a9e50 2024-09-21T06:31:12.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fdcc00640 1 -- 172.21.15.37:0/3011228423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc011d30 con 0x7f8fe00a9e50 2024-09-21T06:31:12.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01f7370 con 0x7f8fe00a9e50 2024-09-21T06:31:12.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.286+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe00aa260 con 0x7f8fe00a9e50 2024-09-21T06:31:12.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.289+0000 7f8fdcc00640 1 -- 172.21.15.37:0/3011228423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fcc018070 con 0x7f8fe00a9e50 2024-09-21T06:31:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.290+0000 7f8fdcc00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 0x7f8fc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.290+0000 7f8fdcc00640 1 -- 172.21.15.37:0/3011228423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f8fcc09bc40 con 0x7f8fe00a9e50 2024-09-21T06:31:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.290+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 0x7f8fc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.290+0000 7f8fee659740 1 Processor -- start 2024-09-21T06:31:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.291+0000 7f8fee659740 1 -- start start 2024-09-21T06:31:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.291+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 0x7f8fc007b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4000c00 tx=0x7f8fd4005040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9da1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d84d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644d9cb61c0 con 0x5644d99864d0 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5644d9cc9870 con 0x5644d99af390 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5644d9cb8ca0 con 0x5644d998e6b0 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57090/0 (socket says 172.21.15.37:57090) 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.292+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3744336862 learned_addr learned my addr 172.21.15.37:0/3744336862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9da1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fe5600640 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d84d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 msgr2=0x5644d9d84d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d84d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 msgr2=0x5644d9da1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9da1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 -- 172.21.15.37:0/3744336862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d9a917d0 con 0x5644d99864d0 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc063070 tx=0x7f8fcc063360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c53a3619cc9d60b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fd1200640 1 -- 172.21.15.37:0/3744336862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01f040 con 0x5644d99864d0 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fd1200640 1 -- 172.21.15.37:0/3744336862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc068330 con 0x5644d99864d0 2024-09-21T06:31:12.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.293+0000 7f8fd1200640 1 -- 172.21.15.37:0/3744336862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc06cf50 con 0x5644d99864d0 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 -- 172.21.15.37:0/3744336862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 msgr2=0x5644d9d84810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc063070 tx=0x7f8fcc063360 comp rx=0 tx=0).stop 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 -- 172.21.15.37:0/3744336862 shutdown_connections 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d84810 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 --2- 172.21.15.37:0/3744336862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9da1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 -- 172.21.15.37:0/3744336862 >> 172.21.15.37:0/3744336862 conn(0x5644d99a34e0 msgr2=0x5644d9d5aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 -- 172.21.15.37:0/3744336862 shutdown_connections 2024-09-21T06:31:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.294+0000 7f8fee659740 1 -- 172.21.15.37:0/3744336862 wait complete. 2024-09-21T06:31:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.295+0000 7f8fee659740 1 -- start start 2024-09-21T06:31:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.295+0000 7f8fee659740 1 Processor -- start 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d97cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644d9c5f410 con 0x5644d99864d0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5644d9cbcab0 con 0x5644d99af390 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5644d9a29740 con 0x5644d998e6b0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fee659740 10 client.0 _ll_register_callbacks cb 0x5644d9c3c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fe4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fe4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57104/0 (socket says 172.21.15.37:57104) 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fe4c00640 1 -- 172.21.15.37:0/4286008806 learned_addr learned my addr 172.21.15.37:0/4286008806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.297+0000 7f8fe5600640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d97cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 -- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 msgr2=0x5644d9d98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 -- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 msgr2=0x5644d9d97cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d97cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 -- 172.21.15.37:0/4286008806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d9a1a970 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe5600640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d97cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4010f70 tx=0x7f8fd40014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e740e4df0dfd680 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fc5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4025030 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f8fd4018040 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fcc068330 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.298+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8fd4012e50 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 10 client.6015 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5644d99f86a0 con 0x5644d99864d0 2024-09-21T06:31:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc068330 con 0x5644d99864d0 2024-09-21T06:31:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f8fd4020650 con 0x5644d99864d0 2024-09-21T06:31:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 10 client.6015 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 10 client.6015 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-09-21T06:31:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 20 client.6015 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d996df60 0x5644d9d7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 4 client.6015 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644d9ccb7c0 con 0x5644d996df60 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.299+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d996df60 0x5644d9d7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.300+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d996df60 0x5644d9d7d0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6302dc28157dc95e server_cookie=92e4a8267826b0d9 in_seq=0 out_seq=0 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.300+0000 7f8fc5600640 10 client.6015 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.301+0000 7f8fc5600640 1 -- 172.21.15.37:0/4286008806 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644d9ccb7c0 con 0x5644d996df60 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.301+0000 7f8fc5600640 10 client.6015 handle_command_reply: tid=0 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.301+0000 7f8fee659740 1 client.6015 shutdown 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.301+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 msgr2=0x5644d9d80940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.301+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4010f70 tx=0x7f8fd40014c0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 shutdown_connections 2024-09-21T06:31:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d996df60 0x5644d9d7d0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d97cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d998e6b0 0x5644d9d98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 --2- 172.21.15.37:0/4286008806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d99864d0 0x5644d9d80940 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 >> 172.21.15.37:0/4286008806 conn(0x5644d99a34e0 msgr2=0x5644d9d73fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fc5600640 0 client.6015 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 shutdown_connections 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 1 -- 172.21.15.37:0/4286008806 wait complete. 2024-09-21T06:31:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.302+0000 7f8fee659740 20 client.6015 trim_cache size 0 max 16384 2024-09-21T06:31:12.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.320+0000 7f8fee659740 1 Processor -- start 2024-09-21T06:31:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.321+0000 7f8fee659740 1 -- start start 2024-09-21T06:31:12.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 0x5644d9d98f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d84d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01fe8d0 con 0x5644d998e6b0 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01fdbc0 con 0x5644d99af390 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe01fe1a0 con 0x5644d99864d0 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57114/0 (socket says 172.21.15.37:57114) 2024-09-21T06:31:12.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.322+0000 7f8fdfe00640 1 -- 172.21.15.37:0/2538324514 learned_addr learned my addr 172.21.15.37:0/2538324514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 0x5644d9d98f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fe5600640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d84d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 -- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 msgr2=0x5644d9d98f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 0x5644d9d98f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 -- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 msgr2=0x5644d9d84d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d84d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 -- 172.21.15.37:0/2538324514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d9d7c910 con 0x5644d998e6b0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 0x5644d9d98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fe5600640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc099fb0 tx=0x7f8fcc05ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cf876a224bcb5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fc7e00640 1 -- 172.21.15.37:0/2538324514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01f070 con 0x5644d998e6b0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fc7e00640 1 -- 172.21.15.37:0/2538324514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc06cd50 con 0x5644d998e6b0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.323+0000 7f8fc7e00640 1 -- 172.21.15.37:0/2538324514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc064b80 con 0x5644d998e6b0 2024-09-21T06:31:12.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 -- 172.21.15.37:0/2538324514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 msgr2=0x5644d9d84810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc099fb0 tx=0x7f8fcc05ff20 comp rx=0 tx=0).stop 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 -- 172.21.15.37:0/2538324514 shutdown_connections 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d84810 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99af390 0x5644d9d84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 --2- 172.21.15.37:0/2538324514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99864d0 0x5644d9d98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 -- 172.21.15.37:0/2538324514 >> 172.21.15.37:0/2538324514 conn(0x5644d99a34e0 msgr2=0x5644d9da01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 -- 172.21.15.37:0/2538324514 shutdown_connections 2024-09-21T06:31:12.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.324+0000 7f8fee659740 1 -- 172.21.15.37:0/2538324514 wait complete. 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.325+0000 7f8fee659740 1 -- start start 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.325+0000 7f8fee659740 1 Processor -- start 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99864d0 0x5644d9dc22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 0x5644d9e14d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0073e10 con 0x5644d998e6b0 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5644d9d5b840 con 0x5644d99864d0 2024-09-21T06:31:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5644d9d739f0 con 0x5644d99af390 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fee659740 10 client.0 _ll_register_callbacks cb 0x7f8fd4010940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57126/0 (socket says 172.21.15.37:57126) 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fdfe00640 1 -- 172.21.15.37:0/119418338 learned_addr learned my addr 172.21.15.37:0/119418338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99864d0 0x5644d9dc22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.327+0000 7f8fe5600640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 0x5644d9e14d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 -- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 msgr2=0x5644d9e14d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 0x5644d9e14d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 -- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99864d0 msgr2=0x5644d9dc22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99864d0 0x5644d9dc22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 -- 172.21.15.37:0/119418338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d9d84f50 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fe5600640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 0x5644d9e14d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fdfe00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc087010 tx=0x7f8fcc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3512eddd8731d444 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fc5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01f040 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f8fcc064b80 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fcc06cd50 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8fcc068330 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fee659740 10 client.6021 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.328+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5644d9d84cd0 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc06cd50 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f8fcc062ab0 con 0x5644d998e6b0 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 10 client.6021 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:12.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 10 client.6021 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-09-21T06:31:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 20 client.6021 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d9d72d70 0x5644d9d73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 4 client.6021 mds_command: new command op to 5500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:31:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 --> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644d9d7f6f0 con 0x5644d9d72d70 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.329+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d9d72d70 0x5644d9d73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fe4c00640 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d9d72d70 0x5644d9d73180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f51637a0b6e74513 server_cookie=89d30afad5a1b4d3 in_seq=0 out_seq=0 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fc5600640 10 client.6021 ms_handle_connect on v2:172.21.15.150:6834/1705702768 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fc5600640 1 -- 172.21.15.37:0/119418338 <== mds.2 v2:172.21.15.150:6834/1705702768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644d9d7f6f0 con 0x5644d9d72d70 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fc5600640 10 client.6021 handle_command_reply: tid=0 2024-09-21T06:31:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 client.6021 shutdown 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 msgr2=0x5644d9d81560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc087010 tx=0x7f8fcc011f10 comp rx=0 tx=0).stop 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 shutdown_connections 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:6834/1705702768,v1:172.21.15.150:6835/1705702768] conn(0x5644d9d72d70 0x5644d9d73180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644d998e6b0 0x5644d9d81560 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.330+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5644d99af390 0x5644d9e14d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fee659740 1 --2- 172.21.15.37:0/119418338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5644d99864d0 0x5644d9dc22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 >> 172.21.15.37:0/119418338 conn(0x5644d99a34e0 msgr2=0x5644d9d80700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fc5600640 0 client.6021 ms_handle_reset on v2:172.21.15.150:6834/1705702768 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 shutdown_connections 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fee659740 1 -- 172.21.15.37:0/119418338 wait complete. 2024-09-21T06:31:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fee659740 20 client.6021 trim_cache size 0 max 16384 2024-09-21T06:31:12.336 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:12.336 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-21T06:31:12.336 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-21T06:31:12.336 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 msgr2=0x7f8fc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 0x7f8fc007b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4000c00 tx=0x7f8fd4005040 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 msgr2=0x7f8fe0140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc006600 tx=0x7f8fcc0039c0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 shutdown_connections 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8fc0078b90 0x7f8fc007b0a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe00a9e50 0x7f8fe0140b80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe01f3b10 0x7f8fe01f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 --2- 172.21.15.37:0/3011228423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fc8004d90 0x7f8fe01e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.331+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 >> 172.21.15.37:0/3011228423 conn(0x7f8fe006f920 msgr2=0x7f8fe00afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.332+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 shutdown_connections 2024-09-21T06:31:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.332+0000 7f8fe6000640 1 -- 172.21.15.37:0/3011228423 wait complete. 2024-09-21T06:31:12.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:31:12.358 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:31:12.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-21T06:31:12.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 -- 172.21.15.37:0/631858730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 msgr2=0x7f43e4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 --2- 172.21.15.37:0/631858730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e4074620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009830 tx=0x7f43d402a510 comp rx=0 tx=0).stop 2024-09-21T06:31:12.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 -- 172.21.15.37:0/631858730 shutdown_connections 2024-09-21T06:31:12.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 --2- 172.21.15.37:0/631858730 >> v2:172.21.15.150:3300/0 conn(0x7f43e4074b60 0x7f43e406ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 --2- 172.21.15.37:0/631858730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e4074620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 -- 172.21.15.37:0/631858730 >> 172.21.15.37:0/631858730 conn(0x7f43e40b1b90 msgr2=0x7f43e40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 -- 172.21.15.37:0/631858730 shutdown_connections 2024-09-21T06:31:12.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.587+0000 7f43ea000640 1 -- 172.21.15.37:0/631858730 wait complete. 2024-09-21T06:31:12.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.589+0000 7f43ea000640 1 Processor -- start 2024-09-21T06:31:12.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.589+0000 7f43ea000640 1 -- start start 2024-09-21T06:31:12.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e4074b60 0x7f43e40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e41f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40ce500 con 0x7f43e406e330 2024-09-21T06:31:12.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40ce380 con 0x7f43e4074b60 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43ea000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43e40ce680 con 0x7f43e4156a50 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57138/0 (socket says 172.21.15.37:57138) 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 -- 172.21.15.37:0/4239470427 learned_addr learned my addr 172.21.15.37:0/4239470427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e2a00640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e4074b60 0x7f43e40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3e00640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e41f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 -- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 msgr2=0x7f43e41f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e41f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 -- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e4074b60 msgr2=0x7f43e40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e4074b60 0x7f43e40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.591+0000 7f43e3400640 1 -- 172.21.15.37:0/4239470427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41571e0 con 0x7f43e406e330 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43e3400640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009950 tx=0x7f43d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43db400640 1 -- 172.21.15.37:0/4239470427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4005c90 con 0x7f43e406e330 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43db400640 1 -- 172.21.15.37:0/4239470427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43d4031e50 con 0x7f43e406e330 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41574c0 con 0x7f43e406e330 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43db400640 1 -- 172.21.15.37:0/4239470427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4005260 con 0x7f43e406e330 2024-09-21T06:31:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.592+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e40b5450 con 0x7f43e406e330 2024-09-21T06:31:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.595+0000 7f43db400640 1 -- 172.21.15.37:0/4239470427 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43d4005400 con 0x7f43e406e330 2024-09-21T06:31:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.596+0000 7f43db400640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 0x7f43c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.596+0000 7f43db400640 1 -- 172.21.15.37:0/4239470427 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f43d40bbfe0 con 0x7f43e406e330 2024-09-21T06:31:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.596+0000 7f43f10d6740 1 Processor -- start 2024-09-21T06:31:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.596+0000 7f43e2a00640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 0x7f43c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.596+0000 7f43f10d6740 1 -- start start 2024-09-21T06:31:12.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.597+0000 7f43e2a00640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 0x7f43c407b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43e40b4160 tx=0x7f43d0009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.597+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c3fa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43f10d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a49c3301c0 con 0x55a49c0086b0 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a49c343870 con 0x55a49c029390 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a49c332ca0 con 0x55a49c0004d0 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57152/0 (socket says 172.21.15.37:57152) 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 -- 172.21.15.37:0/3327970011 learned_addr learned my addr 172.21.15.37:0/3327970011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e2a00640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c3fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3400640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 -- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 msgr2=0x55a49c3fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c3fa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 -- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 msgr2=0x55a49c417550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3e00640 1 -- 172.21.15.37:0/3327970011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49c10b7d0 con 0x55a49c0086b0 2024-09-21T06:31:12.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.598+0000 7f43e3400640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f43dc001ea0 tx=0x7f43dc001590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb67844844a075f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43cf400640 1 -- 172.21.15.37:0/3327970011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00fc40 con 0x55a49c0086b0 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43cf400640 1 -- 172.21.15.37:0/3327970011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc018070 con 0x55a49c0086b0 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43cf400640 1 -- 172.21.15.37:0/3327970011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc003b70 con 0x55a49c0086b0 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43f10d6740 1 -- 172.21.15.37:0/3327970011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 msgr2=0x55a49c3fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.599+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f43dc001ea0 tx=0x7f43dc001590 comp rx=0 tx=0).stop 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 -- 172.21.15.37:0/3327970011 shutdown_connections 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c3fa760 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c3fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3327970011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 -- 172.21.15.37:0/3327970011 >> 172.21.15.37:0/3327970011 conn(0x55a49c01d4e0 msgr2=0x55a49c3d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 -- 172.21.15.37:0/3327970011 shutdown_connections 2024-09-21T06:31:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.600+0000 7f43f10d6740 1 -- 172.21.15.37:0/3327970011 wait complete. 2024-09-21T06:31:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.601+0000 7f43f10d6740 1 -- start start 2024-09-21T06:31:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.601+0000 7f43f10d6740 1 Processor -- start 2024-09-21T06:31:12.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.602+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c40e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.602+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.602+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.602+0000 7f43f10d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a49c2d9410 con 0x55a49c0086b0 2024-09-21T06:31:12.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a49c336ab0 con 0x55a49c029390 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a49c0a3740 con 0x55a49c0004d0 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43f10d6740 10 client.0 _ll_register_callbacks cb 0x55a49c2b6360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57158/0 (socket says 172.21.15.37:57158) 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 -- 172.21.15.37:0/799337863 learned_addr learned my addr 172.21.15.37:0/799337863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e3400640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c40e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e3e00640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 -- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 msgr2=0x55a49c40e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c40e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 -- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 msgr2=0x55a49c417e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.603+0000 7f43e2a00640 1 -- 172.21.15.37:0/799337863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49c094970 con 0x55a49c0086b0 2024-09-21T06:31:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43e2a00640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004520 tx=0x7f43d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a121db23e6ef353b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0003dc0 con 0x55a49c0086b0 2024-09-21T06:31:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f43d0028c80 con 0x55a49c0086b0 2024-09-21T06:31:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc018070 con 0x55a49c0086b0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f43d0028e80 con 0x55a49c0086b0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43f10d6740 10 client.6030 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55a49c0726a0 con 0x55a49c0086b0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.604+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc018070 con 0x55a49c0086b0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f43d002f050 con 0x55a49c0086b0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 10 client.6030 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 10 client.6030 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 20 client.6030 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49bfe7f60 0x55a49c4183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 4 client.6030 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49c3457c0 con 0x55a49bfe7f60 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.605+0000 7f43e3400640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49bfe7f60 0x55a49c4183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.606+0000 7f43e3400640 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49bfe7f60 0x55a49c4183b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c684b2d149e6272 server_cookie=3fe427a3b83d263e in_seq=0 out_seq=0 2024-09-21T06:31:12.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.606+0000 7f43c3e00640 10 client.6030 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:12.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.606+0000 7f43c3e00640 1 -- 172.21.15.37:0/799337863 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a49c3457c0 con 0x55a49bfe7f60 2024-09-21T06:31:12.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.606+0000 7f43c3e00640 10 client.6030 handle_command_reply: tid=0 2024-09-21T06:31:12.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.606+0000 7f43f10d6740 1 client.6030 shutdown 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 msgr2=0x55a49c4275a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004520 tx=0x7f43d0004730 comp rx=0 tx=0).stop 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 shutdown_connections 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c0086b0 0x55a49c4275a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49bfe7f60 0x55a49c4183b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c029390 0x55a49c417e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 --2- 172.21.15.37:0/799337863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0004d0 0x55a49c40e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43c3e00640 0 client.6030 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 >> 172.21.15.37:0/799337863 conn(0x55a49c01d4e0 msgr2=0x55a49c3d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 shutdown_connections 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 1 -- 172.21.15.37:0/799337863 wait complete. 2024-09-21T06:31:12.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.607+0000 7f43f10d6740 20 client.6030 trim_cache size 0 max 16384 2024-09-21T06:31:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.625+0000 7f43f10d6740 1 Processor -- start 2024-09-21T06:31:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.626+0000 7f43f10d6740 1 -- start start 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c437b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c3d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e42006a0 con 0x55a49c029390 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41ff840 con 0x55a49c0004d0 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43e41fff70 con 0x55a49c0086b0 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57166/0 (socket says 172.21.15.37:57166) 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43e3e00640 1 -- 172.21.15.37:0/4254535329 learned_addr learned my addr 172.21.15.37:0/4254535329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43e3400640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c437b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.627+0000 7f43e2a00640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c3d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 -- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 msgr2=0x55a49c3d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c3d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 -- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 msgr2=0x55a49c437b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c437b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 -- 172.21.15.37:0/4254535329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49c3ccfa0 con 0x55a49c029390 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3400640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c437b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43e3e00640 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f43dc015040 tx=0x7f43dc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87c19e3b522a6de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43cea00640 1 -- 172.21.15.37:0/4254535329 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc006d50 con 0x55a49c029390 2024-09-21T06:31:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43cea00640 1 -- 172.21.15.37:0/4254535329 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc018040 con 0x55a49c029390 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.628+0000 7f43cea00640 1 -- 172.21.15.37:0/4254535329 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc003bd0 con 0x55a49c029390 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 -- 172.21.15.37:0/4254535329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 msgr2=0x55a49c3d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f43dc015040 tx=0x7f43dc001250 comp rx=0 tx=0).stop 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 -- 172.21.15.37:0/4254535329 shutdown_connections 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3d1660 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c3d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 --2- 172.21.15.37:0/4254535329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c437b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 -- 172.21.15.37:0/4254535329 >> 172.21.15.37:0/4254535329 conn(0x55a49c01d4e0 msgr2=0x55a49c3c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 -- 172.21.15.37:0/4254535329 shutdown_connections 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.629+0000 7f43f10d6740 1 -- 172.21.15.37:0/4254535329 wait complete. 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.630+0000 7f43f10d6740 1 -- start start 2024-09-21T06:31:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.630+0000 7f43f10d6740 1 Processor -- start 2024-09-21T06:31:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c48ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c436aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a49c3cc6b0 con 0x55a49c029390 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41f3f90 con 0x55a49c0004d0 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a49c4373a0 con 0x55a49c0086b0 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43f10d6740 10 client.0 _ll_register_callbacks cb 0x7f43d00041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57170/0 (socket says 172.21.15.37:57170) 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43e3e00640 1 -- 172.21.15.37:0/3800958964 learned_addr learned my addr 172.21.15.37:0/3800958964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43e3400640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c48ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.632+0000 7f43e2a00640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c436aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 -- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 msgr2=0x55a49c436aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c436aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 -- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 msgr2=0x55a49c48ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c48ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 -- 172.21.15.37:0/3800958964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49c4211d0 con 0x55a49c029390 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3400640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c48ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43e3e00640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f43dc011010 tx=0x7f43dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996240de1d9b04b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0052d0 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f43dc032c60 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc018040 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f43dc032e60 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43f10d6740 10 client.6036 fetch_fsmap learned FSMap version 43 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.633+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55a49c498e40 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc018040 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 3558+0+0 (secure 0 0 0) 0x7f43dc025020 con 0x55a49c029390 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 10 client.6036 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 10 client.6036 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 20 client.6036 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49c414b80 0x55a49c414f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 4 client.6036 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 --> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a49c341270 con 0x55a49c414b80 2024-09-21T06:31:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.634+0000 7f43e3400640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49c414b80 0x55a49c414f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43e3400640 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49c414b80 0x55a49c414f90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba439dd2f151643e server_cookie=5a42430dff78ef3a in_seq=0 out_seq=0 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43c3e00640 10 client.6036 ms_handle_connect on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43c3e00640 1 -- 172.21.15.37:0/3800958964 <== mds.0 v2:172.21.15.37:6832/3035654558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+382 (crc 0 0 0) 0x55a49c341270 con 0x55a49c414b80 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43c3e00640 10 client.6036 handle_command_reply: tid=0 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 client.6036 shutdown 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 msgr2=0x55a49c3c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f43dc011010 tx=0x7f43dc003ba0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 shutdown_connections 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a49c029390 0x55a49c3c9830 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.37:6832/3035654558,v1:172.21.15.37:6834/3035654558] conn(0x55a49c414b80 0x55a49c414f90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a49c0086b0 0x55a49c436aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 --2- 172.21.15.37:0/3800958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a49c0004d0 0x55a49c48ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43c3e00640 0 client.6036 ms_handle_reset on v2:172.21.15.37:6832/3035654558 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 >> 172.21.15.37:0/3800958964 conn(0x55a49c01d4e0 msgr2=0x55a49c3d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 shutdown_connections 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.635+0000 7f43f10d6740 1 -- 172.21.15.37:0/3800958964 wait complete. 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43f10d6740 20 client.6036 trim_cache size 0 max 16384 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {}, 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:31:12.640 INFO:teuthology.orchestra.run.smithi037.stdout: "69a4999b-afdb-4309-9e01-a1f19558c26d": { 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:31:00.791249+0000", 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:12.641 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 msgr2=0x7f43c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 0x7f43c407b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43e40b4160 tx=0x7f43d0009260 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 msgr2=0x7f43e40c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009950 tx=0x7f43d4033c20 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 shutdown_connections 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43c4078b90 0x7f43c407b0a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43e4156a50 0x7f43e41f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e4074b60 0x7f43e40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 --2- 172.21.15.37:0/4239470427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e406e330 0x7f43e40c9da0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 >> 172.21.15.37:0/4239470427 conn(0x7f43e40b1b90 msgr2=0x7f43e40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 shutdown_connections 2024-09-21T06:31:12.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.636+0000 7f43ea000640 1 -- 172.21.15.37:0/4239470427 wait complete. 2024-09-21T06:31:12.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:31:12.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 -- 172.21.15.37:0/1339796728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690815cb20 msgr2=0x7f69080cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 --2- 172.21.15.37:0/1339796728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690815cb20 0x7f69080cddc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f69080cdc70 tx=0x7f68f802a560 comp rx=0 tx=0).stop 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 -- 172.21.15.37:0/1339796728 shutdown_connections 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 --2- 172.21.15.37:0/1339796728 >> v2:172.21.15.150:3300/0 conn(0x7f69080b4990 0x7f69080b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 --2- 172.21.15.37:0/1339796728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080cc8d0 0x7f69080ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 --2- 172.21.15.37:0/1339796728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690815cb20 0x7f69080cddc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 -- 172.21.15.37:0/1339796728 >> 172.21.15.37:0/1339796728 conn(0x7f690806f940 msgr2=0x7f69080b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 -- 172.21.15.37:0/1339796728 shutdown_connections 2024-09-21T06:31:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.889+0000 7f6907200640 1 -- 172.21.15.37:0/1339796728 wait complete. 2024-09-21T06:31:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.891+0000 7f6907200640 1 Processor -- start 2024-09-21T06:31:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.892+0000 7f6907200640 1 -- start start 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080b4990 0x7f6908143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690815cb20 0x7f690813ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f690806d3f0 con 0x7f69080cc8d0 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f690806d270 con 0x7f690815cb20 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6907200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f690806d570 con 0x7f69080b4990 2024-09-21T06:31:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57178/0 (socket says 172.21.15.37:57178) 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f68ffe00640 1 -- 172.21.15.37:0/1150714936 learned_addr learned my addr 172.21.15.37:0/1150714936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.893+0000 7f6905e00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080b4990 0x7f6908143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f6906800640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690815cb20 0x7f690813ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 -- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080b4990 msgr2=0x7f6908143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080b4990 0x7f6908143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 -- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690815cb20 msgr2=0x7f690813ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690815cb20 0x7f690813ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 -- 172.21.15.37:0/1150714936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690813f930 con 0x7f69080cc8d0 2024-09-21T06:31:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68ffe00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e7c0 tx=0x7f68f000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f000cab0 con 0x7f69080cc8d0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68f000cc50 con 0x7f69080cc8d0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0005020 con 0x7f69080cc8d0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.894+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690813fc10 con 0x7f69080cc8d0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.895+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081401d0 con 0x7f69080cc8d0 2024-09-21T06:31:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.896+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690806da20 con 0x7f69080cc8d0 2024-09-21T06:31:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.897+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68f00051c0 con 0x7f69080cc8d0 2024-09-21T06:31:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.898+0000 7f68fe000640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 0x7f68e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.899+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f68f009b7a0 con 0x7f69080cc8d0 2024-09-21T06:31:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.899+0000 7f6905e00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 0x7f68e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.900+0000 7f6905e00640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 0x7f68e407b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68f8033000 tx=0x7f68f8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:12.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:12.903+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68f0063a20 con 0x7f69080cc8d0 2024-09-21T06:31:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.140+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6908154530 con 0x7f69080cc8d0 2024-09-21T06:31:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.141+0000 7f68fe000640 1 -- 172.21.15.37:0/1150714936 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7f68f0063320 con 0x7f69080cc8d0 2024-09-21T06:31:13.142 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:13.142 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 msgr2=0x7f68e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 0x7f68e407b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68f8033000 tx=0x7f68f8004730 comp rx=0 tx=0).stop 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 msgr2=0x7f69081409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e7c0 tx=0x7f68f000ebc0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 shutdown_connections 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f68e4078b90 0x7f68e407b0a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69080cc8d0 0x7f69081409d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690815cb20 0x7f690813ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 --2- 172.21.15.37:0/1150714936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69080b4990 0x7f6908143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 >> 172.21.15.37:0/1150714936 conn(0x7f690806f940 msgr2=0x7f69080d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 shutdown_connections 2024-09-21T06:31:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.144+0000 7f6907200640 1 -- 172.21.15.37:0/1150714936 wait complete. 2024-09-21T06:31:13.167 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-21T06:31:13.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:31:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 -- 172.21.15.37:0/4092836132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc15b820 msgr2=0x7ff1dc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 --2- 172.21.15.37:0/4092836132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc15b820 0x7ff1dc15e8a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc158770 tx=0x7ff1cc02a5f0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 -- 172.21.15.37:0/4092836132 shutdown_connections 2024-09-21T06:31:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 --2- 172.21.15.37:0/4092836132 >> v2:172.21.15.150:3300/0 conn(0x7ff1dc0ccf30 0x7ff1dc0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 --2- 172.21.15.37:0/4092836132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc15b820 0x7ff1dc15e8a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 -- 172.21.15.37:0/4092836132 >> 172.21.15.37:0/4092836132 conn(0x7ff1dc06f940 msgr2=0x7ff1dc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 -- 172.21.15.37:0/4092836132 shutdown_connections 2024-09-21T06:31:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.394+0000 7ff1db200640 1 -- 172.21.15.37:0/4092836132 wait complete. 2024-09-21T06:31:13.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.396+0000 7ff1db200640 1 Processor -- start 2024-09-21T06:31:13.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.396+0000 7ff1db200640 1 -- start start 2024-09-21T06:31:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc15b820 0x7ff1dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 0x7ff1dc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc070e70 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc070cf0 con 0x7ff1dc15ede0 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1db200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc0ce300 con 0x7ff1dc15b820 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1da800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1da800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57196/0 (socket says 172.21.15.37:57196) 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1da800640 1 -- 172.21.15.37:0/2028336061 learned_addr learned my addr 172.21.15.37:0/2028336061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1d9e00640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc15b820 0x7ff1dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.398+0000 7ff1d3e00640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 0x7ff1dc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 -- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc15b820 msgr2=0x7ff1dc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc15b820 0x7ff1dc0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 -- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 msgr2=0x7ff1dc153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 0x7ff1dc153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 -- 172.21.15.37:0/2028336061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc0afc50 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1d3e00640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 0x7ff1dc153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1da800640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ea70 tx=0x7ff1d400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400ce40 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1d4004c10 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4005020 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc0aff30 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.399+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc0b04f0 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.400+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1dc155020 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.401+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1d40051c0 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.402+0000 7ff1d2000640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 0x7ff1b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:13.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.403+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff1d409bd40 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.403+0000 7ff1d9e00640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 0x7ff1b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.404+0000 7ff1d9e00640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 0x7ff1b807b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc033000 tx=0x7ff1cc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.408+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1d4063f30 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.678+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff1dc0ce7b0 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.679+0000 7ff1d2000640 1 -- 172.21.15.37:0/2028336061 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1960 (secure 0 0 0) 0x7ff1d4063830 con 0x7ff1dc1f5db0 2024-09-21T06:31:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":241000448,"bytes_avail":772853112832,"bytes_total":773094113280,"write_bytes_sec":2985,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5500},{"filesystem_id":3,"rank":1,"name":"a","status":"up:active","gid":5506},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5574}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-21T06:31:13.140127+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.681+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 msgr2=0x7ff1b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.681+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 0x7ff1b807b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc033000 tx=0x7ff1cc004730 comp rx=0 tx=0).stop 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.681+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 msgr2=0x7ff1dc151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.681+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ea70 tx=0x7ff1d400ef30 comp rx=0 tx=0).stop 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 shutdown_connections 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1b8078b90 0x7ff1b807b0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1dc1f5db0 0x7ff1dc151cc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc15ede0 0x7ff1dc153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 --2- 172.21.15.37:0/2028336061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc15b820 0x7ff1dc0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 >> 172.21.15.37:0/2028336061 conn(0x7ff1dc06f940 msgr2=0x7ff1dc159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 shutdown_connections 2024-09-21T06:31:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:13.682+0000 7ff1db200640 1 -- 172.21.15.37:0/2028336061 wait complete. 2024-09-21T06:31:13.708 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:31:14.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.946+0000 7f2312000640 1 -- 172.21.15.37:0/3025944358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 msgr2=0x7f22f0004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.946+0000 7f2312000640 1 --2- 172.21.15.37:0/3025944358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f22f0004080 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f22fc001830 tx=0x7f22f0005830 comp rx=0 tx=0).stop 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.947+0000 7f2312000640 1 -- 172.21.15.37:0/3025944358 shutdown_connections 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.947+0000 7f2312000640 1 --2- 172.21.15.37:0/3025944358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f22f0004080 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.947+0000 7f2312000640 1 -- 172.21.15.37:0/3025944358 >> 172.21.15.37:0/3025944358 conn(0x7f230c0d4360 msgr2=0x7f230c0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.947+0000 7f2312000640 1 -- 172.21.15.37:0/3025944358 shutdown_connections 2024-09-21T06:31:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.947+0000 7f2312000640 1 -- 172.21.15.37:0/3025944358 wait complete. 2024-09-21T06:31:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.949+0000 7f2312000640 1 Processor -- start 2024-09-21T06:31:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.949+0000 7f2312000640 1 -- start start 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 0x7f230c0ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 0x7f230c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f230c0cf4b0 con 0x7f22f0003c30 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f230c15eac0 con 0x7f230c0aad90 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2312000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f230c0cf630 con 0x7f230c0cd2c0 2024-09-21T06:31:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:14.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54696/0 (socket says 172.21.15.37:54696) 2024-09-21T06:31:14.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 -- 172.21.15.37:0/604722197 learned_addr learned my addr 172.21.15.37:0/604722197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2303e00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 0x7f230c0ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2311600640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 0x7f230c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 -- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 msgr2=0x7f230c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 0x7f230c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 -- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 msgr2=0x7f230c0ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 0x7f230c0ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:14.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2310c00640 1 -- 172.21.15.37:0/604722197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c0b4480 con 0x7f22f0003c30 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2303e00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 0x7f230c0ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.951+0000 7f2311600640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 0x7f230c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2310c00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f22fc02aa40 tx=0x7f22fc0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc043070 con 0x7f22f0003c30 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22fc003af0 con 0x7f22f0003c30 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c0cdc10 con 0x7f22f0003c30 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0064c0 con 0x7f22f0003c30 2024-09-21T06:31:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.952+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c0ac040 con 0x7f22f0003c30 2024-09-21T06:31:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.953+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c0cfb90 con 0x7f22f0003c30 2024-09-21T06:31:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.954+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22fc046070 con 0x7f22f0003c30 2024-09-21T06:31:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.955+0000 7f2309600640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 0x7f22e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.956+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f22fc0bade0 con 0x7f22f0003c30 2024-09-21T06:31:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.956+0000 7f2303e00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 0x7f22e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.957+0000 7f2303e00640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 0x7f22e807b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f230c0aea30 tx=0x7f22f4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:14.960+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f230c0cdc10 con 0x7f22f0003c30 2024-09-21T06:31:15.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.200+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f230c162e90 con 0x7f22f0003c30 2024-09-21T06:31:15.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.200+0000 7f2309600640 1 -- 172.21.15.37:0/604722197 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7f230c0b4480 con 0x7f22f0003c30 2024-09-21T06:31:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:15.202 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:15.202 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 msgr2=0x7f22e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 0x7f22e807b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f230c0aea30 tx=0x7f22f4005f90 comp rx=0 tx=0).stop 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 msgr2=0x7f230c0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f22fc02aa40 tx=0x7f22fc0060e0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 shutdown_connections 2024-09-21T06:31:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f22e8078b90 0x7f22e807b0a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f230c0cd2c0 0x7f230c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f230c0aad90 0x7f230c0ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 --2- 172.21.15.37:0/604722197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f0003c30 0x7f230c0b3f40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 >> 172.21.15.37:0/604722197 conn(0x7f230c0d4360 msgr2=0x7f230c15d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 shutdown_connections 2024-09-21T06:31:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.203+0000 7f2312000640 1 -- 172.21.15.37:0/604722197 wait complete. 2024-09-21T06:31:15.226 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-21T06:31:15.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:31:15.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 -- 172.21.15.37:0/3550263357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 msgr2=0x7f22640cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 --2- 172.21.15.37:0/3550263357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640cb4d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f226406f100 tx=0x7f225c02aa30 comp rx=0 tx=0).stop 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 -- 172.21.15.37:0/3550263357 shutdown_connections 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 --2- 172.21.15.37:0/3550263357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640cb4d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 --2- 172.21.15.37:0/3550263357 >> v2:172.21.15.150:3300/0 conn(0x7f2264073f20 0x7f226415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 -- 172.21.15.37:0/3550263357 >> 172.21.15.37:0/3550263357 conn(0x7f2264073400 msgr2=0x7f2264073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 -- 172.21.15.37:0/3550263357 shutdown_connections 2024-09-21T06:31:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.452+0000 7f2263200640 1 -- 172.21.15.37:0/3550263357 wait complete. 2024-09-21T06:31:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.454+0000 7f2263200640 1 Processor -- start 2024-09-21T06:31:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.454+0000 7f2263200640 1 -- start start 2024-09-21T06:31:15.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226406ddf0 0x7f22640b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:15.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2264073f20 0x7f22640b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:15.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:15.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22640cbda0 con 0x7f22640cb0c0 2024-09-21T06:31:15.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22640cbc20 con 0x7f2264073f20 2024-09-21T06:31:15.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2263200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f22640cbf20 con 0x7f226406ddf0 2024-09-21T06:31:15.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2262800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:15.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2262800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54708/0 (socket says 172.21.15.37:54708) 2024-09-21T06:31:15.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f2262800640 1 -- 172.21.15.37:0/3599056773 learned_addr learned my addr 172.21.15.37:0/3599056773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.456+0000 7f225be00640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2264073f20 0x7f22640b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2261e00640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226406ddf0 0x7f22640b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 -- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226406ddf0 msgr2=0x7f22640b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226406ddf0 0x7f22640b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 -- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2264073f20 msgr2=0x7f22640b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2264073f20 0x7f22640b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 -- 172.21.15.37:0/3599056773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641f5b40 con 0x7f22640cb0c0 2024-09-21T06:31:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2262800640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f225c02b020 tx=0x7f225c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c033050 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f225c004540 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c038450 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.457+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641f5e20 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.458+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641f63e0 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.459+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226415b2f0 con 0x7f22640cb0c0 2024-09-21T06:31:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.460+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f225c036070 con 0x7f22640cb0c0 2024-09-21T06:31:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.461+0000 7f225a000640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.461+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f225c0bc330 con 0x7f22640cb0c0 2024-09-21T06:31:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.461+0000 7f2261e00640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:15.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.462+0000 7f2261e00640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 0x7f224007b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2254009ac0 tx=0x7f2254005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.465+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f225c084520 con 0x7f22640cb0c0 2024-09-21T06:31:15.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.734+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2264159590 con 0x7f22640cb0c0 2024-09-21T06:31:15.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.735+0000 7f225a000640 1 -- 172.21.15.37:0/3599056773 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1912 (secure 0 0 0) 0x7f225c083e20 con 0x7f22640cb0c0 2024-09-21T06:31:15.736 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:15.736 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":223,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":241000448,"bytes_avail":772853112832,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5500},{"filesystem_id":3,"rank":1,"name":"a","status":"up:active","gid":5506},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5574}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-21T06:31:15.140714+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 msgr2=0x7f224007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 0x7f224007b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2254009ac0 tx=0x7f2254005f90 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 msgr2=0x7f22640b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f225c02b020 tx=0x7f225c003f10 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 shutdown_connections 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22640cb0c0 0x7f22640b08c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2264073f20 0x7f22640b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 --2- 172.21.15.37:0/3599056773 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226406ddf0 0x7f22640b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 >> 172.21.15.37:0/3599056773 conn(0x7f2264073400 msgr2=0x7f22640ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.737+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 shutdown_connections 2024-09-21T06:31:15.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:15.738+0000 7f2263200640 1 -- 172.21.15.37:0/3599056773 wait complete. 2024-09-21T06:31:15.764 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:31:16.765 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 -- 172.21.15.37:0/3416188589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7815f5a0 msgr2=0x7faf7815f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3416188589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7815f5a0 0x7faf7815f9b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faf78072560 tx=0x7faf7002aac0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 -- 172.21.15.37:0/3416188589 shutdown_connections 2024-09-21T06:31:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3416188589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7815f5a0 0x7faf7815f9b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3416188589 >> v2:172.21.15.150:3300/0 conn(0x7faf7815b9f0 0x7faf780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 -- 172.21.15.37:0/3416188589 >> 172.21.15.37:0/3416188589 conn(0x7faf780cca40 msgr2=0x7faf780cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 -- 172.21.15.37:0/3416188589 shutdown_connections 2024-09-21T06:31:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:16.999+0000 7faf7fe00640 1 -- 172.21.15.37:0/3416188589 wait complete. 2024-09-21T06:31:17.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.001+0000 7faf7fe00640 1 Processor -- start 2024-09-21T06:31:17.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.001+0000 7faf7fe00640 1 -- start start 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 0x7faf781f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf7815f5a0 0x7faf780a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf7806df70 con 0x7faf780cefa0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf7806ddf0 con 0x7faf7815f5a0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7fe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faf7806e0f0 con 0x7faf7815b9f0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54722/0 (socket says 172.21.15.37:54722) 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 -- 172.21.15.37:0/3537548823 learned_addr learned my addr 172.21.15.37:0/3537548823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7de00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf7815f5a0 0x7faf780a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7ca00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 0x7faf781f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 -- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 msgr2=0x7faf781f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 0x7faf781f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 -- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf7815f5a0 msgr2=0x7faf780a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf7815f5a0 0x7faf780a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7d400640 1 -- 172.21.15.37:0/3537548823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf781f3540 con 0x7faf780cefa0 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.003+0000 7faf7ca00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 0x7faf781f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf7d400640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e790 tx=0x7faf6c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00eea0 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf6c00cdb0 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c010690 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf780b43f0 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.004+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf78156510 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.005+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7806e5a0 con 0x7faf780cefa0 2024-09-21T06:31:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.006+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf6c010830 con 0x7faf780cefa0 2024-09-21T06:31:17.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.007+0000 7faf75600640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 0x7faf5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.007+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7faf6c09b8a0 con 0x7faf780cefa0 2024-09-21T06:31:17.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.008+0000 7faf7ca00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 0x7faf5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.008+0000 7faf7ca00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 0x7faf5807b0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf780a12c0 tx=0x7faf64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:17.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.012+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf6c0634d0 con 0x7faf780cefa0 2024-09-21T06:31:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.248+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf780cd0a0 con 0x7faf780cefa0 2024-09-21T06:31:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.248+0000 7faf75600640 1 -- 172.21.15.37:0/3537548823 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7faf6c067f00 con 0x7faf780cefa0 2024-09-21T06:31:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:17.250 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:17.250 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 msgr2=0x7faf5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 0x7faf5807b0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf780a12c0 tx=0x7faf64009040 comp rx=0 tx=0).stop 2024-09-21T06:31:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 msgr2=0x7faf781f3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e790 tx=0x7faf6c00ec50 comp rx=0 tx=0).stop 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 shutdown_connections 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf58078b90 0x7faf5807b0a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf7815f5a0 0x7faf780a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf7815b9f0 0x7faf781f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 --2- 172.21.15.37:0/3537548823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf780cefa0 0x7faf781f3000 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 >> 172.21.15.37:0/3537548823 conn(0x7faf780cca40 msgr2=0x7faf7806fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 shutdown_connections 2024-09-21T06:31:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.251+0000 7faf7fe00640 1 -- 172.21.15.37:0/3537548823 wait complete. 2024-09-21T06:31:17.274 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-21T06:31:17.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:31:17.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 -- 172.21.15.37:0/281938336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320003c30 msgr2=0x7fe320004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 --2- 172.21.15.37:0/281938336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320003c30 0x7fe320004080 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009d00 tx=0x7fe320005830 comp rx=0 tx=0).stop 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 -- 172.21.15.37:0/281938336 shutdown_connections 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 --2- 172.21.15.37:0/281938336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320003c30 0x7fe320004080 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 -- 172.21.15.37:0/281938336 >> 172.21.15.37:0/281938336 conn(0x7fe33c06f940 msgr2=0x7fe33c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 -- 172.21.15.37:0/281938336 shutdown_connections 2024-09-21T06:31:17.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.500+0000 7fe33be00640 1 -- 172.21.15.37:0/281938336 wait complete. 2024-09-21T06:31:17.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.502+0000 7fe33be00640 1 Processor -- start 2024-09-21T06:31:17.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.503+0000 7fe33be00640 1 -- start start 2024-09-21T06:31:17.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 0x7fe33c0af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 0x7fe33c073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c06e560 con 0x7fe33c14dce0 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c06e3e0 con 0x7fe320003c30 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe33c0cc810 con 0x7fe33c072c00 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54732/0 (socket says 172.21.15.37:54732) 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.504+0000 7fe33a000640 1 -- 172.21.15.37:0/884685172 learned_addr learned my addr 172.21.15.37:0/884685172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33b400640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 0x7fe33c073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33aa00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 0x7fe33c0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 -- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 msgr2=0x7fe33c073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 0x7fe33c073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 -- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 msgr2=0x7fe33c0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 0x7fe33c0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 -- 172.21.15.37:0/884685172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c0ae110 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33aa00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 0x7fe33c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33b400640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 0x7fe33c073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33a000640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe328006650 tx=0x7fe328001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328011640 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe328016020 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328011a60 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.505+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1f5f30 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.506+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c0ae690 con 0x7fe33c14dce0 2024-09-21T06:31:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.507+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c0b4990 con 0x7fe33c14dce0 2024-09-21T06:31:17.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.508+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe328011c00 con 0x7fe33c14dce0 2024-09-21T06:31:17.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.509+0000 7fe332a00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 0x7fe31c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:17.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.509+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fe32809bb30 con 0x7fe33c14dce0 2024-09-21T06:31:17.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.510+0000 7fe33aa00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 0x7fe31c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:17.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.511+0000 7fe33aa00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 0x7fe31c07b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe32c035000 tx=0x7fe32c0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:17.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.514+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe328020020 con 0x7fe33c14dce0 2024-09-21T06:31:17.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.781+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe33c1538d0 con 0x7fe33c14dce0 2024-09-21T06:31:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.782+0000 7fe332a00640 1 -- 172.21.15.37:0/884685172 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1912 (secure 0 0 0) 0x7fe328063dc0 con 0x7fe33c14dce0 2024-09-21T06:31:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:17.783 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":241000448,"bytes_avail":772853112832,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5500},{"filesystem_id":3,"rank":1,"name":"a","status":"up:active","gid":5506},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5574}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-21T06:31:15.140714+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 msgr2=0x7fe31c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 0x7fe31c07b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe32c035000 tx=0x7fe32c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 msgr2=0x7fe33c150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe328006650 tx=0x7fe328001800 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 shutdown_connections 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe31c078bc0 0x7fe31c07b0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33c14dce0 0x7fe33c150d60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe33c072c00 0x7fe33c073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 --2- 172.21.15.37:0/884685172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe320003c30 0x7fe33c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.784+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 >> 172.21.15.37:0/884685172 conn(0x7fe33c06f940 msgr2=0x7fe33c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.785+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 shutdown_connections 2024-09-21T06:31:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:17.785+0000 7fe33be00640 1 -- 172.21.15.37:0/884685172 wait complete. 2024-09-21T06:31:17.811 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:31:18.812 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 -- 172.21.15.37:0/189286304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7104073dd0 msgr2=0x7f710406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 --2- 172.21.15.37:0/189286304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7104073dd0 0x7f710406ddf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009830 tx=0x7f70f002a5c0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 -- 172.21.15.37:0/189286304 shutdown_connections 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 --2- 172.21.15.37:0/189286304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7104073dd0 0x7f710406ddf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 -- 172.21.15.37:0/189286304 >> 172.21.15.37:0/189286304 conn(0x7f71040cca40 msgr2=0x7f71040cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 -- 172.21.15.37:0/189286304 shutdown_connections 2024-09-21T06:31:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.050+0000 7f710bc00640 1 -- 172.21.15.37:0/189286304 wait complete. 2024-09-21T06:31:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.053+0000 7f710bc00640 1 Processor -- start 2024-09-21T06:31:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.053+0000 7f710bc00640 1 -- start start 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 0x7f71040b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 0x7f7104146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f710406f670 con 0x7f7104073dd0 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f710406f4f0 con 0x7f710406e330 2024-09-21T06:31:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f710bc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f710406f7f0 con 0x7f7104074cb0 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.054+0000 7f7103e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54746/0 (socket says 172.21.15.37:54746) 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 -- 172.21.15.37:0/4172561588 learned_addr learned my addr 172.21.15.37:0/4172561588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7109200640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 0x7f71040b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7109c00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 0x7f7104146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 -- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 msgr2=0x7f7104146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 0x7f7104146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 -- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 msgr2=0x7f71040b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 0x7f71040b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 -- 172.21.15.37:0/4172561588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710414a800 con 0x7f7104073dd0 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7109200640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 0x7f71040b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7109c00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 0x7f7104146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.055+0000 7f7103e00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f70f002abb0 tx=0x7f70f0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.056+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0031d60 con 0x7f7104073dd0 2024-09-21T06:31:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.056+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f0033e10 con 0x7f7104073dd0 2024-09-21T06:31:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.056+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0005260 con 0x7f7104073dd0 2024-09-21T06:31:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.056+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7104146590 con 0x7f7104073dd0 2024-09-21T06:31:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.056+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710414a520 con 0x7f7104073dd0 2024-09-21T06:31:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.057+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710406ddf0 con 0x7f7104073dd0 2024-09-21T06:31:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.058+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70f0005400 con 0x7f7104073dd0 2024-09-21T06:31:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.059+0000 7f7100c00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 0x7f70e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.059+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f70f00bc060 con 0x7f7104073dd0 2024-09-21T06:31:19.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.060+0000 7f7109200640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 0x7f70e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.060+0000 7f7109200640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 0x7f70e407b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:19.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.064+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f70f003f030 con 0x7f7104073dd0 2024-09-21T06:31:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.306+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71040cfa70 con 0x7f7104073dd0 2024-09-21T06:31:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.306+0000 7f7100c00640 1 -- 172.21.15.37:0/4172561588 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7f70f00842e0 con 0x7f7104073dd0 2024-09-21T06:31:19.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:19.308 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.308+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 msgr2=0x7f70e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.308+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 0x7f70e407b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8009040 comp rx=0 tx=0).stop 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.308+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 msgr2=0x7f71041e8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.308+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f70f002abb0 tx=0x7f70f0033c20 comp rx=0 tx=0).stop 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 shutdown_connections 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e4078bc0 0x7f70e407b0d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104073dd0 0x7f71041e8920 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7104074cb0 0x7f7104146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 --2- 172.21.15.37:0/4172561588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710406e330 0x7f71040b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 >> 172.21.15.37:0/4172561588 conn(0x7f71040cca40 msgr2=0x7f71040ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:19.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 shutdown_connections 2024-09-21T06:31:19.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.309+0000 7f710bc00640 1 -- 172.21.15.37:0/4172561588 wait complete. 2024-09-21T06:31:19.338 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-21T06:31:19.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:31:19.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 -- 172.21.15.37:0/1846838011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f042406def0 msgr2=0x7f042406e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 --2- 172.21.15.37:0/1846838011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f042406def0 0x7f042406e300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c02aa80 comp rx=0 tx=0).stop 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 -- 172.21.15.37:0/1846838011 shutdown_connections 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 --2- 172.21.15.37:0/1846838011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f042406def0 0x7f042406e300 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 --2- 172.21.15.37:0/1846838011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424073dd0 0x7f042415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 -- 172.21.15.37:0/1846838011 >> 172.21.15.37:0/1846838011 conn(0x7f042406f940 msgr2=0x7f04240b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 -- 172.21.15.37:0/1846838011 shutdown_connections 2024-09-21T06:31:19.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.562+0000 7f0429200640 1 -- 172.21.15.37:0/1846838011 wait complete. 2024-09-21T06:31:19.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.564+0000 7f0429200640 1 Processor -- start 2024-09-21T06:31:19.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.564+0000 7f0429200640 1 -- start start 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406def0 0x7f04240afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424152530 0x7f042414b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04240cf5f0 con 0x7f0424073dd0 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f04240cf470 con 0x7f042406def0 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0429200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f04240cf770 con 0x7f0424152530 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0422a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0422a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54758/0 (socket says 172.21.15.37:54758) 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0422a00640 1 -- 172.21.15.37:0/1691435650 learned_addr learned my addr 172.21.15.37:0/1691435650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0423e00640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424152530 0x7f042414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.566+0000 7f0423400640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406def0 0x7f04240afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 -- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424152530 msgr2=0x7f042414b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424152530 0x7f042414b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 -- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406def0 msgr2=0x7f04240afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406def0 0x7f04240afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 -- 172.21.15.37:0/1691435650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04240b1850 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0422a00640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f041000c3c0 tx=0x7f041000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410017070 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04100035f0 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04240b1b30 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410003a40 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.567+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04240b0080 con 0x7f0424073dd0 2024-09-21T06:31:19.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.569+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04240b2050 con 0x7f0424073dd0 2024-09-21T06:31:19.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.570+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f041001a070 con 0x7f0424073dd0 2024-09-21T06:31:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.571+0000 7f041b400640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 0x7f040407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.571+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f041009a260 con 0x7f0424073dd0 2024-09-21T06:31:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.571+0000 7f0423400640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 0x7f040407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.572+0000 7f0423400640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 0x7f040407b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f041400bb00 tx=0x7f0414005e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.576+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04240b2050 con 0x7f0424073dd0 2024-09-21T06:31:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.846+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f04240cfc20 con 0x7f0424073dd0 2024-09-21T06:31:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.846+0000 7f041b400640 1 -- 172.21.15.37:0/1691435650 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1865 (secure 0 0 0) 0x7f042415c8f0 con 0x7f0424073dd0 2024-09-21T06:31:19.848 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:19.848 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":227,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":241004544,"bytes_avail":772853108736,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5500},{"filesystem_id":3,"rank":1,"name":"a","status":"up:active","gid":5506},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5574}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-09-21T06:31:19.141877+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:31:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 msgr2=0x7f040407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 0x7f040407b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f041400bb00 tx=0x7f0414005e30 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 msgr2=0x7f0424148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f041000c3c0 tx=0x7f041000c880 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 shutdown_connections 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0404078b90 0x7f040407b0a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0424073dd0 0x7f0424148490 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0424152530 0x7f042414b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 --2- 172.21.15.37:0/1691435650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406def0 0x7f04240afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 >> 172.21.15.37:0/1691435650 conn(0x7f042406f940 msgr2=0x7f04240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 shutdown_connections 2024-09-21T06:31:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:19.849+0000 7f0429200640 1 -- 172.21.15.37:0/1691435650 wait complete. 2024-09-21T06:31:19.874 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-21T06:31:19.874 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:31:19.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:31:19.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:31:19.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:31:19.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:31:20.080 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:31:20.080 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:31:20.080 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:20.080 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:20.116 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:20.116 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:20.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:31:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.458+0000 7f46ae200640 1 -- 172.21.15.37:0/2945599341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4690004000 msgr2=0x7f4690004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.458+0000 7f46ae200640 1 --2- 172.21.15.37:0/2945599341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4690004000 0x7f4690004800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4694001740 tx=0x7f4690005830 comp rx=0 tx=0).stop 2024-09-21T06:31:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.458+0000 7f46ad800640 1 -- 172.21.15.37:0/2945599341 reap_dead start 2024-09-21T06:31:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.459+0000 7f46ae200640 1 -- 172.21.15.37:0/2945599341 shutdown_connections 2024-09-21T06:31:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.459+0000 7f46ae200640 1 -- 172.21.15.37:0/2945599341 >> 172.21.15.37:0/2945599341 conn(0x7f46a80d05c0 msgr2=0x7f46a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:20.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.459+0000 7f46ae200640 1 -- 172.21.15.37:0/2945599341 shutdown_connections 2024-09-21T06:31:20.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.459+0000 7f46ae200640 1 -- 172.21.15.37:0/2945599341 wait complete. 2024-09-21T06:31:20.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.461+0000 7f46ae200640 1 Processor -- start 2024-09-21T06:31:20.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.461+0000 7f46ae200640 1 -- start start 2024-09-21T06:31:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4690004000 0x7f46a8074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a81ea130 0x7f46a81ea540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46a80b3820 con 0x7f46a81f87c0 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46a80707b0 con 0x7f46a81ea130 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.462+0000 7f46ae200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f46a80b39a0 con 0x7f4690004000 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54778/0 (socket says 172.21.15.37:54778) 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 -- 172.21.15.37:0/636959225 learned_addr learned my addr 172.21.15.37:0/636959225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ace00640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4690004000 0x7f46a8074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46a7e00640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a81ea130 0x7f46a81ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 -- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4690004000 msgr2=0x7f46a8074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4690004000 0x7f46a8074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 -- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a81ea130 msgr2=0x7f46a81ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a81ea130 0x7f46a81ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.463+0000 7f46ad800640 1 -- 172.21.15.37:0/636959225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a81e97e0 con 0x7f46a81f87c0 2024-09-21T06:31:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46ad800640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f46a8072b00 tx=0x7f46a0006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a000fab0 con 0x7f46a81f87c0 2024-09-21T06:31:20.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a000fc50 con 0x7f46a81f87c0 2024-09-21T06:31:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0011430 con 0x7f46a81f87c0 2024-09-21T06:31:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a81eaa80 con 0x7f46a81f87c0 2024-09-21T06:31:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.464+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a8074a90 con 0x7f46a81f87c0 2024-09-21T06:31:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.465+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a80b3f30 con 0x7f46a81f87c0 2024-09-21T06:31:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.466+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46a00115d0 con 0x7f46a81f87c0 2024-09-21T06:31:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.467+0000 7f46a4c00640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 0x7f468807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.467+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f46a009b760 con 0x7f46a81f87c0 2024-09-21T06:31:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.468+0000 7f46ace00640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 0x7f468807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.468+0000 7f46ace00640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 0x7f468807b0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f469c009c50 tx=0x7f469c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:20.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.472+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46a81eaa80 con 0x7f46a81f87c0 2024-09-21T06:31:20.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.679+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46a80cdde0 con 0x7f46a81f87c0 2024-09-21T06:31:20.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.680+0000 7f46a4c00640 1 -- 172.21.15.37:0/636959225 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7f46a00639e0 con 0x7f46a81f87c0 2024-09-21T06:31:20.682 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:31:20.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 msgr2=0x7f468807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:20.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 0x7f468807b0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f469c009c50 tx=0x7f469c005f90 comp rx=0 tx=0).stop 2024-09-21T06:31:20.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 msgr2=0x7f46a81f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f46a8072b00 tx=0x7f46a0006660 comp rx=0 tx=0).stop 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 shutdown_connections 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4688078bc0 0x7f468807b0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a81f87c0 0x7f46a81f8bd0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a81ea130 0x7f46a81ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 --2- 172.21.15.37:0/636959225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4690004000 0x7f46a8074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 >> 172.21.15.37:0/636959225 conn(0x7f46a80d05c0 msgr2=0x7f46a81e7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 shutdown_connections 2024-09-21T06:31:20.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:20.682+0000 7f46ae200640 1 -- 172.21.15.37:0/636959225 wait complete. 2024-09-21T06:31:20.706 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:31:20.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:20.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:31:20.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:31:20.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:20.773 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:31:20.809 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:31:20.809 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:31:20.809 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:20.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:31:20.850 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:31:20.850 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:20.851 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:20.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:31:20.886 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:31:20.886 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:31:20.886 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:20.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:31:20.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:21.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 -- 172.21.15.37:0/1770384577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c15cb20 msgr2=0x7f100c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 --2- 172.21.15.37:0/1770384577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c15cb20 0x7f100c0cddc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f100c0afb10 tx=0x7f0ffc02a560 comp rx=0 tx=0).stop 2024-09-21T06:31:21.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 -- 172.21.15.37:0/1770384577 shutdown_connections 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 --2- 172.21.15.37:0/1770384577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c0b4990 0x7f100c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 --2- 172.21.15.37:0/1770384577 >> v2:172.21.15.150:3300/0 conn(0x7f100c0cc8d0 0x7f100c0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 --2- 172.21.15.37:0/1770384577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c15cb20 0x7f100c0cddc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 -- 172.21.15.37:0/1770384577 >> 172.21.15.37:0/1770384577 conn(0x7f100c06f940 msgr2=0x7f100c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 -- 172.21.15.37:0/1770384577 shutdown_connections 2024-09-21T06:31:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.155+0000 7f100b200640 1 -- 172.21.15.37:0/1770384577 wait complete. 2024-09-21T06:31:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.157+0000 7f100b200640 1 Processor -- start 2024-09-21T06:31:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.157+0000 7f100b200640 1 -- start start 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 0x7f100c13a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c15cb20 0x7f100c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f100c06d3f0 con 0x7f100c0b4990 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f100c06d270 con 0x7f100c0cc8d0 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f100c06d570 con 0x7f100c15cb20 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54786/0 (socket says 172.21.15.37:54786) 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 -- 172.21.15.37:0/669385516 learned_addr learned my addr 172.21.15.37:0/669385516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1003e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 0x7f100c13a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f100a800640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c15cb20 0x7f100c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 -- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c15cb20 msgr2=0x7f100c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c15cb20 0x7f100c0c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 -- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 msgr2=0x7f100c13a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 0x7f100c13a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1009e00640 1 -- 172.21.15.37:0/669385516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c15b8d0 con 0x7f100c0b4990 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.159+0000 7f1003e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 0x7f100c13a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f1009e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc034040 tx=0x7f0ffc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc03e030 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ffc0047a0 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c15bbb0 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc035040 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.160+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c15c170 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.161+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c06da20 con 0x7f100c0b4990 2024-09-21T06:31:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.162+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ffc041030 con 0x7f100c0b4990 2024-09-21T06:31:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.163+0000 7f1002000640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 0x7f0fe807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.164+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f0ffc0bb640 con 0x7f100c0b4990 2024-09-21T06:31:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.164+0000 7f1003e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 0x7f0fe807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.165+0000 7f1003e00640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 0x7f0fe807b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f100c0c5620 tx=0x7f0ff4005f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.168+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ffc083270 con 0x7f100c0b4990 2024-09-21T06:31:21.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.407+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f100c161000 con 0x7f100c0b4990 2024-09-21T06:31:21.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.407+0000 7f1002000640 1 -- 172.21.15.37:0/669385516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7f0ffc087ca0 con 0x7f100c0b4990 2024-09-21T06:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:21.410 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 msgr2=0x7f0fe807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 0x7f0fe807b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f100c0c5620 tx=0x7f0ff4005f10 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 msgr2=0x7f100c0c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc034040 tx=0x7f0ffc0039e0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 shutdown_connections 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fe8078b90 0x7f0fe807b0a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100c15cb20 0x7f100c0c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f100c0cc8d0 0x7f100c13a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 --2- 172.21.15.37:0/669385516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f100c0b4990 0x7f100c0c4e40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.409+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 >> 172.21.15.37:0/669385516 conn(0x7f100c06f940 msgr2=0x7f100c160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:21.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.410+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 shutdown_connections 2024-09-21T06:31:21.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.410+0000 7f100b200640 1 -- 172.21.15.37:0/669385516 wait complete. 2024-09-21T06:31:21.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 -- 172.21.15.37:0/3447937682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 msgr2=0x7fc2380cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 --2- 172.21.15.37:0/3447937682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc2380cc020 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc238076390 tx=0x7fc22002a560 comp rx=0 tx=0).stop 2024-09-21T06:31:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 -- 172.21.15.37:0/3447937682 shutdown_connections 2024-09-21T06:31:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 --2- 172.21.15.37:0/3447937682 >> v2:172.21.15.150:3300/0 conn(0x7fc23806ddf0 0x7fc23806e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 --2- 172.21.15.37:0/3447937682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc2380cc020 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 --2- 172.21.15.37:0/3447937682 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc238074cb0 0x7fc238073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 -- 172.21.15.37:0/3447937682 >> 172.21.15.37:0/3447937682 conn(0x7fc2380cca40 msgr2=0x7fc2380cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:21.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 -- 172.21.15.37:0/3447937682 shutdown_connections 2024-09-21T06:31:21.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.661+0000 7fc23f200640 1 -- 172.21.15.37:0/3447937682 wait complete. 2024-09-21T06:31:21.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.664+0000 7fc23f200640 1 Processor -- start 2024-09-21T06:31:21.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.664+0000 7fc23f200640 1 -- start start 2024-09-21T06:31:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc23806ddf0 0x7fc238158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc238074cb0 0x7fc2380a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc23806f170 con 0x7fc2380cbc10 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc23806eff0 con 0x7fc238074cb0 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.665+0000 7fc23f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc23806f2f0 con 0x7fc23806ddf0 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc237e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc23806ddf0 0x7fc238158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54810/0 (socket says 172.21.15.37:54810) 2024-09-21T06:31:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc22fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc238074cb0 0x7fc2380a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 -- 172.21.15.37:0/3349973128 learned_addr learned my addr 172.21.15.37:0/3349973128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 -- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc23806ddf0 msgr2=0x7fc238158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc23806ddf0 0x7fc238158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 -- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc238074cb0 msgr2=0x7fc2380a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc238074cb0 0x7fc2380a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 -- 172.21.15.37:0/3349973128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2381f55b0 con 0x7fc2380cbc10 2024-09-21T06:31:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.666+0000 7fc23d200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d8d0 tx=0x7fc23000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.667+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230016070 con 0x7fc2380cbc10 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.667+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23000bce0 con 0x7fc2380cbc10 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.667+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2381f5890 con 0x7fc2380cbc10 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.667+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230003bd0 con 0x7fc2380cbc10 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.667+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2381f5e50 con 0x7fc2380cbc10 2024-09-21T06:31:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.668+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc238073dd0 con 0x7fc2380cbc10 2024-09-21T06:31:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.669+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc230010460 con 0x7fc2380cbc10 2024-09-21T06:31:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.671+0000 7fc234c00640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 0x7fc21407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.671+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc23009b500 con 0x7fc2380cbc10 2024-09-21T06:31:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.671+0000 7fc237e00640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 0x7fc21407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.672+0000 7fc237e00640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 0x7fc21407b0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc23815a730 tx=0x7fc22800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.675+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2300630a0 con 0x7fc2380cbc10 2024-09-21T06:31:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.912+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc23815f830 con 0x7fc2380cbc10 2024-09-21T06:31:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.912+0000 7fc234c00640 1 -- 172.21.15.37:0/3349973128 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7fc230067ad0 con 0x7fc2380cbc10 2024-09-21T06:31:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:21.915 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:21.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 msgr2=0x7fc21407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 0x7fc21407b0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc23815a730 tx=0x7fc22800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 msgr2=0x7fc23809c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d8d0 tx=0x7fc23000dd90 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 shutdown_connections 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc214078bc0 0x7fc21407b0d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2380cbc10 0x7fc23809c3e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc238074cb0 0x7fc2380a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 --2- 172.21.15.37:0/3349973128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc23806ddf0 0x7fc238158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 >> 172.21.15.37:0/3349973128 conn(0x7fc2380cca40 msgr2=0x7fc2380cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 shutdown_connections 2024-09-21T06:31:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:21.915+0000 7fc23f200640 1 -- 172.21.15.37:0/3349973128 wait complete. 2024-09-21T06:31:21.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 -- 172.21.15.37:0/1792179697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 msgr2=0x7fc1780ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 --2- 172.21.15.37:0/1792179697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1780ccce0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc170005fd0 tx=0x7fc17002aa10 comp rx=0 tx=0).stop 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 -- 172.21.15.37:0/1792179697 shutdown_connections 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 --2- 172.21.15.37:0/1792179697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1780ccce0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 --2- 172.21.15.37:0/1792179697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc17806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 -- 172.21.15.37:0/1792179697 >> 172.21.15.37:0/1792179697 conn(0x7fc17806f940 msgr2=0x7fc1780b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 -- 172.21.15.37:0/1792179697 shutdown_connections 2024-09-21T06:31:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.166+0000 7fc177e00640 1 -- 172.21.15.37:0/1792179697 wait complete. 2024-09-21T06:31:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.168+0000 7fc177e00640 1 Processor -- start 2024-09-21T06:31:22.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.169+0000 7fc177e00640 1 -- start start 2024-09-21T06:31:22.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc1780aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 0x7fc1781f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780b4c50 con 0x7fc1780cc8d0 2024-09-21T06:31:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780b4ad0 con 0x7fc17815dfe0 2024-09-21T06:31:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc177e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780b4dd0 con 0x7fc17806e160 2024-09-21T06:31:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc16fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc16fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54828/0 (socket says 172.21.15.37:54828) 2024-09-21T06:31:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.170+0000 7fc16fe00640 1 -- 172.21.15.37:0/1113613699 learned_addr learned my addr 172.21.15.37:0/1113613699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:22.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc177400640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 0x7fc1781f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 -- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 msgr2=0x7fc1780aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc176a00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc1780aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc1780aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 -- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 msgr2=0x7fc1781f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 0x7fc1781f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 -- 172.21.15.37:0/1113613699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc178145b50 con 0x7fc1780cc8d0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc177400640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 0x7fc1781f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc176a00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc1780aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16fe00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d650 tx=0x7fc16000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160017070 con 0x7fc1780cc8d0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc160009b10 con 0x7fc1780cc8d0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.171+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160010460 con 0x7fc1780cc8d0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.172+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1780a9a00 con 0x7fc1780cc8d0 2024-09-21T06:31:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.172+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1780a9fc0 con 0x7fc1780cc8d0 2024-09-21T06:31:22.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.173+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1780b52f0 con 0x7fc1780cc8d0 2024-09-21T06:31:22.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.174+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc16001a070 con 0x7fc1780cc8d0 2024-09-21T06:31:22.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.175+0000 7fc16ea00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 0x7fc15407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.176+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc16009b010 con 0x7fc1780cc8d0 2024-09-21T06:31:22.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.176+0000 7fc176a00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 0x7fc15407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.177+0000 7fc176a00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 0x7fc15407b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1680016a0 tx=0x7fc168009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.180+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc160063290 con 0x7fc1780cc8d0 2024-09-21T06:31:22.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.379+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1780cc0e0 con 0x7fc1780cc8d0 2024-09-21T06:31:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.381+0000 7fc16ea00640 1 -- 172.21.15.37:0/1113613699 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17003 (secure 0 0 0) 0x7fc160062b90 con 0x7fc1780cc8d0 2024-09-21T06:31:22.383 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:22.383 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":56,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:39.867959+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:30:28.767088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T06:30:29.779185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 msgr2=0x7fc15407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 0x7fc15407b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1680016a0 tx=0x7fc168009040 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 msgr2=0x7fc1781f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d650 tx=0x7fc16000db10 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 shutdown_connections 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc154078b90 0x7fc15407b0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1780cc8d0 0x7fc1781f47b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17815dfe0 0x7fc1781f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 --2- 172.21.15.37:0/1113613699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e160 0x7fc1780aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 >> 172.21.15.37:0/1113613699 conn(0x7fc17806f940 msgr2=0x7fc1780cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 shutdown_connections 2024-09-21T06:31:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.383+0000 7fc177e00640 1 -- 172.21.15.37:0/1113613699 wait complete. 2024-09-21T06:31:22.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:31:22.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 -- 172.21.15.37:0/2101721794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8654162d20 msgr2=0x7f865415e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 --2- 172.21.15.37:0/2101721794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8654162d20 0x7f865415e440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8648009830 tx=0x7f864802a530 comp rx=0 tx=0).stop 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 -- 172.21.15.37:0/2101721794 shutdown_connections 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 --2- 172.21.15.37:0/2101721794 >> v2:172.21.15.150:3300/0 conn(0x7f865406e020 0x7f8654162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 --2- 172.21.15.37:0/2101721794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8654162d20 0x7f865415e440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 -- 172.21.15.37:0/2101721794 >> 172.21.15.37:0/2101721794 conn(0x7f86540d0d40 msgr2=0x7f86540d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 -- 172.21.15.37:0/2101721794 shutdown_connections 2024-09-21T06:31:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.633+0000 7f865ae00640 1 -- 172.21.15.37:0/2101721794 wait complete. 2024-09-21T06:31:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.636+0000 7f865ae00640 1 Processor -- start 2024-09-21T06:31:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.636+0000 7f865ae00640 1 -- start start 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f865406e020 0x7f865414c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f865406e430 0x7f865414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8654162a50 con 0x7f8654162d20 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86541628d0 con 0x7f865406e020 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865ae00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f86540d3ab0 con 0x7f865406e430 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.637+0000 7f865a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54838/0 (socket says 172.21.15.37:54838) 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 -- 172.21.15.37:0/1639904478 learned_addr learned my addr 172.21.15.37:0/1639904478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f8659a00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f865406e020 0x7f865414c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f8659000640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f865406e430 0x7f865414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 -- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f865406e430 msgr2=0x7f865414c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f865406e430 0x7f865414c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 -- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f865406e020 msgr2=0x7f865414c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f865406e020 0x7f865414c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 -- 172.21.15.37:0/1639904478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865414b8b0 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.638+0000 7f865a400640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c6b0 tx=0x7f864c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.639+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c003b70 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.639+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f864c003d30 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.639+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c005330 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.639+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86540ac1f0 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.639+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865414be00 con 0x7f8654162d20 2024-09-21T06:31:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.640+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86540d3f60 con 0x7f8654162d20 2024-09-21T06:31:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.641+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f864c0054d0 con 0x7f8654162d20 2024-09-21T06:31:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.642+0000 7f8651600640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 0x7f863407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.642+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f864c09ab70 con 0x7f8654162d20 2024-09-21T06:31:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.643+0000 7f8659a00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 0x7f863407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:22.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.643+0000 7f8659a00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 0x7f863407b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8648034000 tx=0x7f8648004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.647+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f864c0627b0 con 0x7f8654162d20 2024-09-21T06:31:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.882+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86540abf80 con 0x7f8654162d20 2024-09-21T06:31:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.883+0000 7f8651600640 1 -- 172.21.15.37:0/1639904478 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7f864c0671e0 con 0x7f8654162d20 2024-09-21T06:31:22.885 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:22.885 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 msgr2=0x7f863407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 0x7f863407b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8648034000 tx=0x7f8648004730 comp rx=0 tx=0).stop 2024-09-21T06:31:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 msgr2=0x7f865414cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c6b0 tx=0x7f864c00cb70 comp rx=0 tx=0).stop 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 shutdown_connections 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8634078b90 0x7f863407b0a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8654162d20 0x7f865414cda0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f865406e430 0x7f865414c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 --2- 172.21.15.37:0/1639904478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f865406e020 0x7f865414c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 >> 172.21.15.37:0/1639904478 conn(0x7f86540d0d40 msgr2=0x7f8654161220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 shutdown_connections 2024-09-21T06:31:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:22.885+0000 7f865ae00640 1 -- 172.21.15.37:0/1639904478 wait complete. 2024-09-21T06:31:22.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 -- 172.21.15.37:0/2039402026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 msgr2=0x7fdd1806e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2039402026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd1806e300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdd00005fd0 tx=0x7fdd0002a5e0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 -- 172.21.15.37:0/2039402026 shutdown_connections 2024-09-21T06:31:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2039402026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd1806e300 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 -- 172.21.15.37:0/2039402026 >> 172.21.15.37:0/2039402026 conn(0x7fdd1806f940 msgr2=0x7fdd180b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 -- 172.21.15.37:0/2039402026 shutdown_connections 2024-09-21T06:31:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.134+0000 7fdd17e00640 1 -- 172.21.15.37:0/2039402026 wait complete. 2024-09-21T06:31:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.136+0000 7fdd17e00640 1 Processor -- start 2024-09-21T06:31:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.136+0000 7fdd17e00640 1 -- start start 2024-09-21T06:31:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 0x7fdd181f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd18155a00 0x7fdd181540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd180ceb70 con 0x7fdd1806def0 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd180cecf0 con 0x7fdd180ce050 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdd180cee70 con 0x7fdd18155a00 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54866/0 (socket says 172.21.15.37:54866) 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 -- 172.21.15.37:0/2974007824 learned_addr learned my addr 172.21.15.37:0/2974007824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd0fe00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 0x7fdd181f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd17400640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd18155a00 0x7fdd181540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 -- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd18155a00 msgr2=0x7fdd181540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd18155a00 0x7fdd181540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 -- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 msgr2=0x7fdd181f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 0x7fdd181f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd16a00640 1 -- 172.21.15.37:0/2974007824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd181f3790 con 0x7fdd1806def0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.138+0000 7fdd0fe00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 0x7fdd181f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd16a00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c400 tx=0x7fdd0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd1806def0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd080053c0 con 0x7fdd1806def0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd181f7ab0 con 0x7fdd1806def0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003850 con 0x7fdd1806def0 2024-09-21T06:31:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.139+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181f3d30 con 0x7fdd1806def0 2024-09-21T06:31:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.140+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1806e300 con 0x7fdd1806def0 2024-09-21T06:31:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.141+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd0801a070 con 0x7fdd1806def0 2024-09-21T06:31:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.142+0000 7fdd0ea00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 0x7fdcf407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.143+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fdd0809a2e0 con 0x7fdd1806def0 2024-09-21T06:31:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.143+0000 7fdd0fe00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 0x7fdcf407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.144+0000 7fdd0fe00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 0x7fdcf407b0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd00033000 tx=0x7fdd00004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.147+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdd1806e840 con 0x7fdd1806def0 2024-09-21T06:31:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.383+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd1806d6e0 con 0x7fdd1806def0 2024-09-21T06:31:23.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.384+0000 7fdd0ea00640 1 -- 172.21.15.37:0/2974007824 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5276 (secure 0 0 0) 0x7fdd08062560 con 0x7fdd1806def0 2024-09-21T06:31:23.386 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-21T06:31:05:228555+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:30:30.818966+0000","modified":"2024-09-21T06:31:04.902656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5574,"mds_1":5506,"mds_2":5500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5500":{"gid":5500,"name":"d","rank":2,"incarnation":37,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6835/1705702768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1705702768},{"type":"v1","addr":"172.21.15.150:6835","nonce":1705702768}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5506":{"gid":5506,"name":"a","rank":1,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1998484049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1998484049},{"type":"v1","addr":"172.21.15.37:6835","nonce":1998484049}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5574":{"gid":5574,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6834/3035654558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035654558},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035654558}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5574,"qdb_cluster":[5574,5506,5500]},"id":3}]} 2024-09-21T06:31:23.386 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-21T06:31:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 msgr2=0x7fdcf407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 0x7fdcf407b0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd00033000 tx=0x7fdd00004730 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 msgr2=0x7fdd18146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c400 tx=0x7fdd0800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 shutdown_connections 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdcf4078bc0 0x7fdcf407b0d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd18155a00 0x7fdd181540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd180ce050 0x7fdd181f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 --2- 172.21.15.37:0/2974007824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1806def0 0x7fdd18146470 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 >> 172.21.15.37:0/2974007824 conn(0x7fdd1806f940 msgr2=0x7fdd18072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 shutdown_connections 2024-09-21T06:31:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.386+0000 7fdd17e00640 1 -- 172.21.15.37:0/2974007824 wait complete. 2024-09-21T06:31:23.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:23.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 -- 172.21.15.37:0/817437057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931415cb20 msgr2=0x7f93140cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 --2- 172.21.15.37:0/817437057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931415cb20 0x7f93140cddc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f93140cdc70 tx=0x7f930402a560 comp rx=0 tx=0).stop 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 -- 172.21.15.37:0/817437057 shutdown_connections 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 --2- 172.21.15.37:0/817437057 >> v2:172.21.15.150:3300/0 conn(0x7f93140b4990 0x7f93140b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 --2- 172.21.15.37:0/817437057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 0x7f93140ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 --2- 172.21.15.37:0/817437057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931415cb20 0x7f93140cddc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 -- 172.21.15.37:0/817437057 >> 172.21.15.37:0/817437057 conn(0x7f931406f940 msgr2=0x7f93140b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 -- 172.21.15.37:0/817437057 shutdown_connections 2024-09-21T06:31:23.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.639+0000 7f9313e00640 1 -- 172.21.15.37:0/817437057 wait complete. 2024-09-21T06:31:23.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.641+0000 7f9313e00640 1 Processor -- start 2024-09-21T06:31:23.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.642+0000 7f9313e00640 1 -- start start 2024-09-21T06:31:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 0x7f931415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f931415cb20 0x7f9314160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f931406d3f0 con 0x7f93140b4990 2024-09-21T06:31:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f931406d270 con 0x7f931415cb20 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9313e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f931406d570 con 0x7f93140cc8d0 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9312a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9312a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54878/0 (socket says 172.21.15.37:54878) 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.643+0000 7f9312a00640 1 -- 172.21.15.37:0/2767143344 learned_addr learned my addr 172.21.15.37:0/2767143344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9313400640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f931415cb20 0x7f9314160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f930be00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 0x7f931415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 -- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 msgr2=0x7f931415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 0x7f931415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 -- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f931415cb20 msgr2=0x7f9314160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f931415cb20 0x7f9314160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 -- 172.21.15.37:0/2767143344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9314141420 con 0x7f93140b4990 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9312a00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9304000c00 tx=0x7f9304003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930403e030 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9304003e00 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304035040 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.644+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931415b430 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.645+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9314143c40 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.646+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931406db00 con 0x7f93140b4990 2024-09-21T06:31:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.647+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9304041030 con 0x7f93140b4990 2024-09-21T06:31:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.649+0000 7f930b400640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 0x7f92f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.649+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f93040bba60 con 0x7f93140b4990 2024-09-21T06:31:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.649+0000 7f930be00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 0x7f92f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.650+0000 7f930be00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 0x7f92f007b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0058c0 tx=0x7f92fc00c440 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:23.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.652+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f931415b430 con 0x7f93140b4990 2024-09-21T06:31:23.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.852+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f931415aee0 con 0x7f93140b4990 2024-09-21T06:31:23.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.854+0000 7f930b400640 1 -- 172.21.15.37:0/2767143344 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17003 (secure 0 0 0) 0x7f93040837a0 con 0x7f93140b4990 2024-09-21T06:31:23.856 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:23.856 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":56,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:30:39.867959+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:30:28.767088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T06:30:29.779185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:23.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 msgr2=0x7f92f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 0x7f92f007b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0058c0 tx=0x7f92fc00c440 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 msgr2=0x7f93140af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9304000c00 tx=0x7f9304003910 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 shutdown_connections 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f92f0078b90 0x7f92f007b0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f931415cb20 0x7f9314160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93140cc8d0 0x7f931415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 --2- 172.21.15.37:0/2767143344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93140b4990 0x7f93140af110 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 >> 172.21.15.37:0/2767143344 conn(0x7f931406f940 msgr2=0x7f93141534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 shutdown_connections 2024-09-21T06:31:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:23.856+0000 7f9313e00640 1 -- 172.21.15.37:0/2767143344 wait complete. 2024-09-21T06:31:23.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 -- 172.21.15.37:0/566382158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 msgr2=0x7f97b00cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 --2- 172.21.15.37:0/566382158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b00cf310 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f97b0074160 tx=0x7f97a802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 -- 172.21.15.37:0/566382158 shutdown_connections 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 --2- 172.21.15.37:0/566382158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b00cf310 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 --2- 172.21.15.37:0/566382158 >> v2:172.21.15.150:3300/0 conn(0x7f97b006ddf0 0x7f97b006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 -- 172.21.15.37:0/566382158 >> 172.21.15.37:0/566382158 conn(0x7f97b00b1b90 msgr2=0x7f97b00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 -- 172.21.15.37:0/566382158 shutdown_connections 2024-09-21T06:31:24.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.109+0000 7f97b6000640 1 -- 172.21.15.37:0/566382158 wait complete. 2024-09-21T06:31:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.112+0000 7f97b6000640 1 Processor -- start 2024-09-21T06:31:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.112+0000 7f97b6000640 1 -- start start 2024-09-21T06:31:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 0x7f97b01f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b0156a50 0x7f97b0159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97b00cf850 con 0x7f97b00cef00 2024-09-21T06:31:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0161060 con 0x7f97b006ddf0 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.113+0000 7f97b6000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f97b00cf9d0 con 0x7f97b0156a50 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59024/0 (socket says 172.21.15.37:59024) 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 -- 172.21.15.37:0/3060282310 learned_addr learned my addr 172.21.15.37:0/3060282310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97af400640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 0x7f97b01f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97afe00640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b0156a50 0x7f97b0159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 -- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b0156a50 msgr2=0x7f97b0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b0156a50 0x7f97b0159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 -- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 msgr2=0x7f97b01f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 0x7f97b01f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 -- 172.21.15.37:0/3060282310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b00b4f70 con 0x7f97b00cef00 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97af400640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 0x7f97b01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.114+0000 7f97a6a00640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f979800d700 tx=0x7f979800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.115+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798017070 con 0x7f97b00cef00 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.115+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9798009b10 con 0x7f97b00cef00 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.115+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798010460 con 0x7f97b00cef00 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.115+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b00b5250 con 0x7f97b00cef00 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.115+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b00b5810 con 0x7f97b00cef00 2024-09-21T06:31:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.116+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b006e200 con 0x7f97b00cef00 2024-09-21T06:31:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.117+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f979801a070 con 0x7f97b00cef00 2024-09-21T06:31:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.118+0000 7f97a7e00640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 0x7f978c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.118+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f979809b050 con 0x7f97b00cef00 2024-09-21T06:31:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.118+0000 7f97af400640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 0x7f978c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.119+0000 7f97af400640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 0x7f978c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97a00016a0 tx=0x7f97a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:24.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.122+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9798063370 con 0x7f97b00cef00 2024-09-21T06:31:24.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.364+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97b00c6a00 con 0x7f97b00cef00 2024-09-21T06:31:24.367 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:31:24.365+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:31:24.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.367+0000 7f97a7e00640 1 -- 172.21.15.37:0/3060282310 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f9798062c70 con 0x7f97b00cef00 2024-09-21T06:31:24.369 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.370+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 msgr2=0x7f978c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.370+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 0x7f978c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97a00016a0 tx=0x7f97a0009040 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.370+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 msgr2=0x7f97b0159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.370+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f979800d700 tx=0x7f979800dbc0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 shutdown_connections 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f978c078b90 0x7f978c07b0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b00cef00 0x7f97b0159100 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b0156a50 0x7f97b0159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 --2- 172.21.15.37:0/3060282310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b006ddf0 0x7f97b01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 >> 172.21.15.37:0/3060282310 conn(0x7f97b00b1b90 msgr2=0x7f97b00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 shutdown_connections 2024-09-21T06:31:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.371+0000 7f97b6000640 1 -- 172.21.15.37:0/3060282310 wait complete. 2024-09-21T06:31:24.386 INFO:tasks.ceph.mds.d.smithi150.stderr: -1208> 2024-09-21T06:31:00.792+0000 7f97e6c00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1) n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5617ead6e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.195451+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.386 INFO:tasks.ceph.mds.d.smithi150.stderr: -1193> 2024-09-21T06:31:00.793+0000 7f97e6c00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/AsLsXsFsx@2},l=5327 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5617ead6eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.392 INFO:tasks.ceph.mds.c.smithi037.stderr: -2788> 2024-09-21T06:31:00.792+0000 7f9349a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1) n(v3 rc2024-09-21T06:30:45.469447+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ece5b99600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.192451+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:30:45.469447+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.393 INFO:tasks.ceph.mds.c.smithi037.stderr: -2655> 2024-09-21T06:31:00.794+0000 7f9351c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1) n(v4 rc2024-09-21T06:30:45.469447+0000 5=0+5) (inest sync->mix g=1,2) caps={5327=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55ece5b99080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.193451+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:30:45.469447+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.397 INFO:tasks.ceph.mds.a.smithi037.stderr: -1424> 2024-09-21T06:31:00.792+0000 7f87e6a00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1) n(v2 rc2024-09-21T06:30:45.469447+0000 4=0+4) (inest lock) (ifile excl) caps={5327=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a1db340c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.469447+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:30:45.469447+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.398 INFO:tasks.ceph.mds.a.smithi037.stderr: -1274> 2024-09-21T06:31:00.795+0000 7f87eec00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1) n(v1 rc2024-09-21T06:30:45.469447+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5327=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a1db086580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:30:45.195451+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:30:45.194451+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:30:45.469447+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:31:24.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:31:24.504 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:31:24.516 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:31:24.527 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:31:24.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 -- 172.21.15.37:0/1363392774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed806d2b0 msgr2=0x7feed81621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 --2- 172.21.15.37:0/1363392774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed806d2b0 0x7feed81621d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feec8009830 tx=0x7feec802a4a0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 -- 172.21.15.37:0/1363392774 shutdown_connections 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 --2- 172.21.15.37:0/1363392774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed806d2b0 0x7feed81621d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 -- 172.21.15.37:0/1363392774 >> 172.21.15.37:0/1363392774 conn(0x7feed80d1960 msgr2=0x7feed80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 -- 172.21.15.37:0/1363392774 shutdown_connections 2024-09-21T06:31:24.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.644+0000 7feed7e00640 1 -- 172.21.15.37:0/1363392774 wait complete. 2024-09-21T06:31:24.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.646+0000 7feed7e00640 1 Processor -- start 2024-09-21T06:31:24.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.646+0000 7feed7e00640 1 -- start start 2024-09-21T06:31:24.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 0x7feed81f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed80cfa10 0x7feed81f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feed81555b0 con 0x7feed8162710 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feed80d3570 con 0x7feed80cfa10 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7feed8155730 con 0x7feed806d2b0 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feece000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed80cfa10 0x7feed81f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed6a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 0x7feed81f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59064/0 (socket says 172.21.15.37:59064) 2024-09-21T06:31:24.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.648+0000 7feed7400640 1 -- 172.21.15.37:0/1499320707 learned_addr learned my addr 172.21.15.37:0/1499320707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 -- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 msgr2=0x7feed81f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 0x7feed81f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 -- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed80cfa10 msgr2=0x7feed81f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed80cfa10 0x7feed81f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 -- 172.21.15.37:0/1499320707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed80c9110 con 0x7feed8162710 2024-09-21T06:31:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed6a00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 0x7feed81f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feed7400640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7feed000c760 tx=0x7feed000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0003da0 con 0x7feed8162710 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed000ce10 con 0x7feed8162710 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.649+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00052d0 con 0x7feed8162710 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.650+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed81f58b0 con 0x7feed8162710 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.650+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81f5e70 con 0x7feed8162710 2024-09-21T06:31:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.651+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed81f5620 con 0x7feed8162710 2024-09-21T06:31:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.652+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feed0005470 con 0x7feed8162710 2024-09-21T06:31:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.653+0000 7feecfe00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 0x7feeb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:24.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.654+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6388+0+0 (secure 0 0 0) 0x7feed009ac70 con 0x7feed8162710 2024-09-21T06:31:24.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.654+0000 7feed6a00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 0x7feeb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:24.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.655+0000 7feed6a00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 0x7feeb407b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.658+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feed8162b20 con 0x7feed8162710 2024-09-21T06:31:24.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:24.896+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feed81621d0 con 0x7feed8162710 2024-09-21T06:31:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.374+0000 7feecfe00640 1 -- 172.21.15.37:0/1499320707 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7feed00627a0 con 0x7feed8162710 2024-09-21T06:31:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 msgr2=0x7feeb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 0x7feeb407b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec8004730 comp rx=0 tx=0).stop 2024-09-21T06:31:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 msgr2=0x7feed81f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7feed000c760 tx=0x7feed000cc20 comp rx=0 tx=0).stop 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 shutdown_connections 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7feeb4078b90 0x7feeb407b0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed8162710 0x7feed81f4930 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feed80cfa10 0x7feed81f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 --2- 172.21.15.37:0/1499320707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feed806d2b0 0x7feed81f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 >> 172.21.15.37:0/1499320707 conn(0x7feed80d1960 msgr2=0x7feed80b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 shutdown_connections 2024-09-21T06:31:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.378+0000 7feed7e00640 1 -- 172.21.15.37:0/1499320707 wait complete. 2024-09-21T06:31:25.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 -- 172.21.15.37:0/1381876056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 msgr2=0x7f658c160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 --2- 172.21.15.37:0/1381876056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c160100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f658c0c70c0 tx=0x7f657c02a560 comp rx=0 tx=0).stop 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 -- 172.21.15.37:0/1381876056 shutdown_connections 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 --2- 172.21.15.37:0/1381876056 >> v2:172.21.15.150:3300/0 conn(0x7f658c06ef30 0x7f658c06f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 --2- 172.21.15.37:0/1381876056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 0x7f658c160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 --2- 172.21.15.37:0/1381876056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c160100 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.641+0000 7f6591600640 1 -- 172.21.15.37:0/1381876056 >> 172.21.15.37:0/1381876056 conn(0x7f658c0ccac0 msgr2=0x7f658c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:25.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.642+0000 7f6591600640 1 -- 172.21.15.37:0/1381876056 shutdown_connections 2024-09-21T06:31:25.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.642+0000 7f6591600640 1 -- 172.21.15.37:0/1381876056 wait complete. 2024-09-21T06:31:25.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.644+0000 7f6591600640 1 Processor -- start 2024-09-21T06:31:25.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.644+0000 7f6591600640 1 -- start start 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f658c06ef30 0x7f658c1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 0x7f658c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f658c06fc10 con 0x7f658c0ca690 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f658c06fa90 con 0x7f658c06ef30 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.645+0000 7f6591600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f658c06fd90 con 0x7f658c160640 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59088/0 (socket says 172.21.15.37:59088) 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 -- 172.21.15.37:0/1016772690 learned_addr learned my addr 172.21.15.37:0/1016772690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658aa00640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f658c06ef30 0x7f658c1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:25.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658b400640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 0x7f658c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:25.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 -- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 msgr2=0x7f658c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:25.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 0x7f658c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 -- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f658c06ef30 msgr2=0x7f658c1f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f658c06ef30 0x7f658c1f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 -- 172.21.15.37:0/1016772690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c0c68b0 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.646+0000 7f658a000640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f657800d6d0 tx=0x7f657800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.647+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578017070 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.647+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6578005330 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.647+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578009820 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.647+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c0c70c0 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.647+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c0c7680 con 0x7f658c0ca690 2024-09-21T06:31:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.648+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658c0c6620 con 0x7f658c0ca690 2024-09-21T06:31:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.649+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6578009bd0 con 0x7f658c0ca690 2024-09-21T06:31:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.650+0000 7f6582a00640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.650+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6388+0+0 (secure 0 0 0) 0x7f657809b2e0 con 0x7f658c0ca690 2024-09-21T06:31:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.651+0000 7f658aa00640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.652+0000 7f658aa00640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 0x7f656c07b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f657c033000 tx=0x7f657c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.655+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6578062d80 con 0x7f658c0ca690 2024-09-21T06:31:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:25.853+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f658c0738c0 con 0x7f658c0ca690 2024-09-21T06:31:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.376+0000 7f6582a00640 1 -- 172.21.15.37:0/1016772690 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f658c0738c0 con 0x7f658c0ca690 2024-09-21T06:31:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 msgr2=0x7f656c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 0x7f656c07b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f657c033000 tx=0x7f657c004730 comp rx=0 tx=0).stop 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 msgr2=0x7f658c158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f657800d6d0 tx=0x7f657800db90 comp rx=0 tx=0).stop 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 shutdown_connections 2024-09-21T06:31:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c0ca690 0x7f658c158080 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f658c160640 0x7f658c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 --2- 172.21.15.37:0/1016772690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f658c06ef30 0x7f658c1f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 >> 172.21.15.37:0/1016772690 conn(0x7f658c0ccac0 msgr2=0x7f658c0731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 shutdown_connections 2024-09-21T06:31:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.379+0000 7f6591600640 1 -- 172.21.15.37:0/1016772690 wait complete. 2024-09-21T06:31:26.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:31:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 -- 172.21.15.37:0/3744218327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c8c003a50 msgr2=0x7f5c8c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 --2- 172.21.15.37:0/3744218327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c8c003a50 0x7f5c8c003ec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009b20 tx=0x7f5c8c005830 comp rx=0 tx=0).stop 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5ca9c00640 1 -- 172.21.15.37:0/3744218327 reap_dead start 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 -- 172.21.15.37:0/3744218327 shutdown_connections 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 -- 172.21.15.37:0/3744218327 >> 172.21.15.37:0/3744218327 conn(0x7f5ca40d2280 msgr2=0x7f5ca40d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 -- 172.21.15.37:0/3744218327 shutdown_connections 2024-09-21T06:31:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.642+0000 7f5cabc00640 1 -- 172.21.15.37:0/3744218327 wait complete. 2024-09-21T06:31:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.644+0000 7f5cabc00640 1 Processor -- start 2024-09-21T06:31:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.645+0000 7f5cabc00640 1 -- start start 2024-09-21T06:31:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 0x7f5ca415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 0x7f5ca406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40b2d00 con 0x7f5ca4154f30 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40d0420 con 0x7f5ca406def0 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5cabc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca40b2e80 con 0x7f5c8c003a50 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59102/0 (socket says 172.21.15.37:59102) 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5ca3e00640 1 -- 172.21.15.37:0/1600954078 learned_addr learned my addr 172.21.15.37:0/1600954078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5ca9200640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 0x7f5ca415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.646+0000 7f5ca9c00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 0x7f5ca406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 -- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 msgr2=0x7f5ca415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 0x7f5ca415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 -- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 msgr2=0x7f5ca406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 0x7f5ca406e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 -- 172.21.15.37:0/1600954078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca41526f0 con 0x7f5ca4154f30 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca9c00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 0x7f5ca406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca9200640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 0x7f5ca415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca3e00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c90011ae0 tx=0x7f5c90009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9001e070 con 0x7f5ca4154f30 2024-09-21T06:31:26.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c90004480 con 0x7f5ca4154f30 2024-09-21T06:31:26.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca41529d0 con 0x7f5ca4154f30 2024-09-21T06:31:26.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.647+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000fc60 con 0x7f5ca4154f30 2024-09-21T06:31:26.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.648+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca415ecd0 con 0x7f5ca4154f30 2024-09-21T06:31:26.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.648+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca40b1fb0 con 0x7f5ca4154f30 2024-09-21T06:31:26.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.650+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c90021070 con 0x7f5ca4154f30 2024-09-21T06:31:26.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.651+0000 7f5ca0c00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 0x7f5c8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:26.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.651+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5950+0+0 (secure 0 0 0) 0x7f5c9009b950 con 0x7f5ca4154f30 2024-09-21T06:31:26.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.651+0000 7f5ca9200640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 0x7f5c8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:26.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.652+0000 7f5ca9200640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 0x7f5c8407b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c98035000 tx=0x7f5c9802d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.655+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c900635b0 con 0x7f5ca4154f30 2024-09-21T06:31:26.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:26.851+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ca4152ba0 con 0x7f5ca4154f30 2024-09-21T06:31:27.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.384+0000 7f5ca0c00640 1 -- 172.21.15.37:0/1600954078 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7f5c90067fe0 con 0x7f5ca4154f30 2024-09-21T06:31:27.386 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:31:27.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 msgr2=0x7f5c8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 0x7f5c8407b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c98035000 tx=0x7f5c9802d040 comp rx=0 tx=0).stop 2024-09-21T06:31:27.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 msgr2=0x7f5ca41516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c90011ae0 tx=0x7f5c90009e40 comp rx=0 tx=0).stop 2024-09-21T06:31:27.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 shutdown_connections 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5c84078b90 0x7f5c8407b0a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca4154f30 0x7f5ca41516f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ca406def0 0x7f5ca406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 --2- 172.21.15.37:0/1600954078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c8c003a50 0x7f5ca415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 >> 172.21.15.37:0/1600954078 conn(0x7f5ca40d2280 msgr2=0x7f5ca40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 shutdown_connections 2024-09-21T06:31:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.388+0000 7f5cabc00640 1 -- 172.21.15.37:0/1600954078 wait complete. 2024-09-21T06:31:27.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:31:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 -- 172.21.15.37:0/4182066084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 msgr2=0x7f07f0003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 --2- 172.21.15.37:0/4182066084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f07f0003ee0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009b50 tx=0x7f07f0005830 comp rx=0 tx=0).stop 2024-09-21T06:31:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0810c00640 1 -- 172.21.15.37:0/4182066084 reap_dead start 2024-09-21T06:31:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 -- 172.21.15.37:0/4182066084 shutdown_connections 2024-09-21T06:31:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 -- 172.21.15.37:0/4182066084 >> 172.21.15.37:0/4182066084 conn(0x7f080c0d41e0 msgr2=0x7f080c0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 -- 172.21.15.37:0/4182066084 shutdown_connections 2024-09-21T06:31:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.642+0000 7f0811600640 1 -- 172.21.15.37:0/4182066084 wait complete. 2024-09-21T06:31:27.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.644+0000 7f0811600640 1 Processor -- start 2024-09-21T06:31:27.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.644+0000 7f0811600640 1 -- start start 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f080c0ce2f0 0x7f080c1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f080c0ccce0 0x7f080c0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f080c15e7a0 con 0x7f07f0003a70 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f080c15e620 con 0x7f080c0ccce0 2024-09-21T06:31:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0811600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f080c15e920 con 0x7f080c0ce2f0 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59116/0 (socket says 172.21.15.37:59116) 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 -- 172.21.15.37:0/3237677487 learned_addr learned my addr 172.21.15.37:0/3237677487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0803400640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f080c0ce2f0 0x7f080c1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f0810c00640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f080c0ccce0 0x7f080c0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 -- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f080c0ce2f0 msgr2=0x7f080c1f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f080c0ce2f0 0x7f080c1f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 -- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f080c0ccce0 msgr2=0x7f080c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f080c0ccce0 0x7f080c0cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.646+0000 7f080be00640 1 -- 172.21.15.37:0/3237677487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0cb290 con 0x7f07f0003a70 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f080be00640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f07fc036040 tx=0x7f07fc033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:27.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc038040 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07fc02b7e0 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc004280 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1f5c90 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.647+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c0cb530 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.648+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c0d26a0 con 0x7f07f0003a70 2024-09-21T06:31:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.650+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07fc045020 con 0x7f07f0003a70 2024-09-21T06:31:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.650+0000 7f0808c00640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 0x7f07e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.651+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7f07fc0bb6a0 con 0x7f07f0003a70 2024-09-21T06:31:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.651+0000 7f0803400640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 0x7f07e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:27.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.652+0000 7f0803400640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 0x7f07e807b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07f4001860 tx=0x7f07f4000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:27.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.655+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f07fc02b9b0 con 0x7f07f0003a70 2024-09-21T06:31:27.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.856+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f080c0d4a00 con 0x7f07f0003a70 2024-09-21T06:31:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.860+0000 7f0808c00640 1 -- 172.21.15.37:0/3237677487 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f07fc083b00 con 0x7f07f0003a70 2024-09-21T06:31:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 msgr2=0x7f07e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 0x7f07e807b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07f4001860 tx=0x7f07f4000ed0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 msgr2=0x7f080c0cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f07fc036040 tx=0x7f07fc033670 comp rx=0 tx=0).stop 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 shutdown_connections 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f07e8078b90 0x7f07e807b0a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f080c0ccce0 0x7f080c0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f080c0ce2f0 0x7f080c1f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 --2- 172.21.15.37:0/3237677487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07f0003a70 0x7f080c0cbd30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 >> 172.21.15.37:0/3237677487 conn(0x7f080c0d41e0 msgr2=0x7f080c162620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 shutdown_connections 2024-09-21T06:31:27.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:27.863+0000 7f0811600640 1 -- 172.21.15.37:0/3237677487 wait complete. 2024-09-21T06:31:27.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-21T06:31:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.117+0000 7fd9b1e00640 1 -- 172.21.15.37:0/4151201900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 msgr2=0x7fd990003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.117+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/4151201900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd990003ec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009b20 tx=0x7fd990005830 comp rx=0 tx=0).stop 2024-09-21T06:31:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.118+0000 7fd9b1e00640 1 -- 172.21.15.37:0/4151201900 shutdown_connections 2024-09-21T06:31:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.118+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/4151201900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd990003ec0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.118+0000 7fd9b1e00640 1 -- 172.21.15.37:0/4151201900 >> 172.21.15.37:0/4151201900 conn(0x7fd9ac0cca40 msgr2=0x7fd9ac0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.118+0000 7fd9b1e00640 1 -- 172.21.15.37:0/4151201900 shutdown_connections 2024-09-21T06:31:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.118+0000 7fd9b1e00640 1 -- 172.21.15.37:0/4151201900 wait complete. 2024-09-21T06:31:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.120+0000 7fd9b1e00640 1 Processor -- start 2024-09-21T06:31:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.120+0000 7fd9b1e00640 1 -- start start 2024-09-21T06:31:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9ac0b0e30 0x7fd9ac0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ac0c9490 0x7fd9ac09b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac0cc1a0 con 0x7fd990003a50 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac0cc020 con 0x7fd9ac0c9490 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9b1e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac0cc320 con 0x7fd9ac0b0e30 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59132/0 (socket says 172.21.15.37:59132) 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 -- 172.21.15.37:0/2316856202 learned_addr learned my addr 172.21.15.37:0/2316856202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 -- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9ac0b0e30 msgr2=0x7fd9ac0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9ac0b0e30 0x7fd9ac0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 -- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ac0c9490 msgr2=0x7fd9ac09b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ac0c9490 0x7fd9ac09b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.122+0000 7fd9aaa00640 1 -- 172.21.15.37:0/2316856202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac0c98a0 con 0x7fd990003a50 2024-09-21T06:31:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9aaa00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009a70 tx=0x7fd99c033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c038040 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd99c033840 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c033c90 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac09e3c0 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.123+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac0c9ba0 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.124+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ac06e020 con 0x7fd990003a50 2024-09-21T06:31:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.125+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd99c043020 con 0x7fd990003a50 2024-09-21T06:31:28.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.127+0000 7fd9a3400640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 0x7fd98807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:28.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.127+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7fd99c0bae00 con 0x7fd990003a50 2024-09-21T06:31:28.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.127+0000 7fd9a3e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 0x7fd98807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.128+0000 7fd9a3e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 0x7fd98807b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd994001240 tx=0x7fd994000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:28.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.132+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd99c04a020 con 0x7fd990003a50 2024-09-21T06:31:28.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:28.411+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7fd9ac0cf4b0 con 0x7fd990003a50 2024-09-21T06:31:29.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.396+0000 7fd9a3400640 1 -- 172.21.15.37:0/2316856202 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v199) ==== 143+0+0 (secure 0 0 0) 0x7fd9ac0c98a0 con 0x7fd990003a50 2024-09-21T06:31:29.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 msgr2=0x7fd98807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 0x7fd98807b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd994001240 tx=0x7fd994000ed0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 msgr2=0x7fd9ac0b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009a70 tx=0x7fd99c033650 comp rx=0 tx=0).stop 2024-09-21T06:31:29.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 shutdown_connections 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd988078b90 0x7fd98807b0a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ac0c9490 0x7fd9ac09b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9ac0b0e30 0x7fd9ac0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 --2- 172.21.15.37:0/2316856202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd990003a50 0x7fd9ac0b05e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 >> 172.21.15.37:0/2316856202 conn(0x7fd9ac0cca40 msgr2=0x7fd9ac0ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 shutdown_connections 2024-09-21T06:31:29.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.400+0000 7fd9b1e00640 1 -- 172.21.15.37:0/2316856202 wait complete. 2024-09-21T06:31:29.427 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:31:29.427 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:31:29.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 -- 172.21.15.37:0/3554559297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 msgr2=0x7f80a80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 --2- 172.21.15.37:0/3554559297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a80afc30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8098009830 tx=0x7f809802a480 comp rx=0 tx=0).stop 2024-09-21T06:31:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 -- 172.21.15.37:0/3554559297 shutdown_connections 2024-09-21T06:31:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 --2- 172.21.15.37:0/3554559297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 0x7f80a80cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 --2- 172.21.15.37:0/3554559297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a80afc30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 -- 172.21.15.37:0/3554559297 >> 172.21.15.37:0/3554559297 conn(0x7f80a80d0a30 msgr2=0x7f80a80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 -- 172.21.15.37:0/3554559297 shutdown_connections 2024-09-21T06:31:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.654+0000 7f80af000640 1 -- 172.21.15.37:0/3554559297 wait complete. 2024-09-21T06:31:29.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.657+0000 7f80af000640 1 Processor -- start 2024-09-21T06:31:29.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.657+0000 7f80af000640 1 -- start start 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 0x7f80a814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80a806e160 con 0x7f80a8160030 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80a80d0190 con 0x7f80a8153410 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80af000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f80a806e2e0 con 0x7f80a80cf700 2024-09-21T06:31:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80ad000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80ad000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59144/0 (socket says 172.21.15.37:59144) 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.658+0000 7f80ad000640 1 -- 172.21.15.37:0/1774236458 learned_addr learned my addr 172.21.15.37:0/1774236458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f809fe00640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80a7e00640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 0x7f80a814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 -- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 msgr2=0x7f80a814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 0x7f80a814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 -- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 msgr2=0x7f80a8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 -- 172.21.15.37:0/1774236458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a814d930 con 0x7f80a8160030 2024-09-21T06:31:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.659+0000 7f80ad000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f80a000cae0 tx=0x7f80a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.660+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0016020 con 0x7f80a8160030 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.660+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80a000ad30 con 0x7f80a8160030 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.660+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0012680 con 0x7f80a8160030 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.660+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a8153db0 con 0x7f80a8160030 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.660+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a8154370 con 0x7f80a8160030 2024-09-21T06:31:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.661+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8160c30 con 0x7f80a8160030 2024-09-21T06:31:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.662+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80a0019020 con 0x7f80a8160030 2024-09-21T06:31:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.663+0000 7f80a4c00640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 0x7f808407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.663+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7f80a009b1e0 con 0x7f80a8160030 2024-09-21T06:31:29.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.663+0000 7f80a7e00640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 0x7f808407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:29.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.664+0000 7f80a7e00640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 0x7f808407b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8098034000 tx=0x7f8098004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:29.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.667+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80a0020020 con 0x7f80a8160030 2024-09-21T06:31:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.865+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80a8153b40 con 0x7f80a8160030 2024-09-21T06:31:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.866+0000 7f80a4c00640 1 -- 172.21.15.37:0/1774236458 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13579 (secure 0 0 0) 0x7f80a0063080 con 0x7f80a8160030 2024-09-21T06:31:29.868 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:29.869 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":59,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:31:27.378170+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6834/3035654558":"2024-09-22T06:31:24.366208+0000","172.21.15.150:6834/1705702768":"2024-09-22T06:31:24.366183+0000","172.21.15.150:6835/1705702768":"2024-09-22T06:31:24.366183+0000","172.21.15.37:6835/1998484049":"2024-09-22T06:31:24.366202+0000","172.21.15.37:6832/3035654558":"2024-09-22T06:31:24.366208+0000","172.21.15.37:6833/1998484049":"2024-09-22T06:31:24.366202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 msgr2=0x7f808407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 0x7f808407b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8098034000 tx=0x7f8098004730 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 msgr2=0x7f80a8150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f80a000cae0 tx=0x7f80a000a060 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 shutdown_connections 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8084078b90 0x7f808407b0a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a8160030 0x7f80a8150690 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80a8153410 0x7f80a8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 --2- 172.21.15.37:0/1774236458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80a80cf700 0x7f80a814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.868+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 >> 172.21.15.37:0/1774236458 conn(0x7f80a80d0a30 msgr2=0x7f80a8160440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.869+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 shutdown_connections 2024-09-21T06:31:29.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:29.869+0000 7f80af000640 1 -- 172.21.15.37:0/1774236458 wait complete. 2024-09-21T06:31:29.894 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:31:29.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 -- 172.21.15.37:0/920550423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec1627a0 msgr2=0x7f1cec0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 --2- 172.21.15.37:0/920550423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec1627a0 0x7f1cec0b2b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005fd0 tx=0x7f1ce402aa90 comp rx=0 tx=0).stop 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 -- 172.21.15.37:0/920550423 shutdown_connections 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 --2- 172.21.15.37:0/920550423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec1627a0 0x7f1cec0b2b70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 --2- 172.21.15.37:0/920550423 >> v2:172.21.15.150:3300/0 conn(0x7f1cec15c210 0x7f1cec06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 -- 172.21.15.37:0/920550423 >> 172.21.15.37:0/920550423 conn(0x7f1cec0d0c50 msgr2=0x7f1cec0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:30.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.122+0000 7f1cf2800640 1 -- 172.21.15.37:0/920550423 shutdown_connections 2024-09-21T06:31:30.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.123+0000 7f1cf2800640 1 -- 172.21.15.37:0/920550423 wait complete. 2024-09-21T06:31:30.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.125+0000 7f1cf2800640 1 Processor -- start 2024-09-21T06:31:30.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.125+0000 7f1cf2800640 1 -- start start 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec0ca6f0 0x7f1cec1e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cec1e9780 0x7f1cec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec0b3440 con 0x7f1cec0cf660 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec0b32c0 con 0x7f1cec1e9780 2024-09-21T06:31:30.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.126+0000 7f1cf2800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec0b35c0 con 0x7f1cec0ca6f0 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59150/0 (socket says 172.21.15.37:59150) 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 -- 172.21.15.37:0/3939214544 learned_addr learned my addr 172.21.15.37:0/3939214544 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf0a00640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec0ca6f0 0x7f1cec1e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1e00640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cec1e9780 0x7f1cec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 -- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec0ca6f0 msgr2=0x7f1cec1e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec0ca6f0 0x7f1cec1e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 -- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cec1e9780 msgr2=0x7f1cec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cec1e9780 0x7f1cec0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 -- 172.21.15.37:0/3939214544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1e9b90 con 0x7f1cec0cf660 2024-09-21T06:31:30.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.127+0000 7f1cf1400640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000ea20 tx=0x7f1ce000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:30.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.128+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000ce40 con 0x7f1cec0cf660 2024-09-21T06:31:30.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.128+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec0b5be0 con 0x7f1cec0cf660 2024-09-21T06:31:30.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.128+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ce0004020 con 0x7f1cec0cf660 2024-09-21T06:31:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.128+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004470 con 0x7f1cec0cf660 2024-09-21T06:31:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.128+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1f5140 con 0x7f1cec0cf660 2024-09-21T06:31:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.129+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec15c210 con 0x7f1cec0cf660 2024-09-21T06:31:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.130+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ce00035f0 con 0x7f1cec0cf660 2024-09-21T06:31:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.131+0000 7f1ce9600640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 0x7f1ccc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.131+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7f1ce009b4e0 con 0x7f1cec0cf660 2024-09-21T06:31:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.132+0000 7f1cf0a00640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 0x7f1ccc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.132+0000 7f1cf0a00640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 0x7f1ccc07b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8007fd0 tx=0x7f1cd8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:30.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.136+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1cec1e9b90 con 0x7f1cec0cf660 2024-09-21T06:31:30.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.332+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cec0c9d50 con 0x7f1cec0cf660 2024-09-21T06:31:30.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.332+0000 7f1ce9600640 1 -- 172.21.15.37:0/3939214544 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f1ce0063390 con 0x7f1cec0cf660 2024-09-21T06:31:30.334 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 msgr2=0x7f1ccc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 0x7f1ccc07b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8007fd0 tx=0x7f1cd8009040 comp rx=0 tx=0).stop 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 msgr2=0x7f1cec0b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000ea20 tx=0x7f1ce000eee0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 shutdown_connections 2024-09-21T06:31:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1ccc078b90 0x7f1ccc07b0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cec1e9780 0x7f1cec0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cec0ca6f0 0x7f1cec1e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 --2- 172.21.15.37:0/3939214544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cec0cf660 0x7f1cec0b56a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 >> 172.21.15.37:0/3939214544 conn(0x7f1cec0d0c50 msgr2=0x7f1cec161270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 shutdown_connections 2024-09-21T06:31:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.335+0000 7f1cf2800640 1 -- 172.21.15.37:0/3939214544 wait complete. 2024-09-21T06:31:30.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-21T06:31:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 -- 172.21.15.37:0/3487460457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 msgr2=0x7fb7a80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 --2- 172.21.15.37:0/3487460457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a80cf930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009830 tx=0x7fb79c02a4b0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 -- 172.21.15.37:0/3487460457 shutdown_connections 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 --2- 172.21.15.37:0/3487460457 >> v2:172.21.15.150:3300/0 conn(0x7fb7a80cefa0 0x7fb7a80ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 --2- 172.21.15.37:0/3487460457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a80cf930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 -- 172.21.15.37:0/3487460457 >> 172.21.15.37:0/3487460457 conn(0x7fb7a80cca40 msgr2=0x7fb7a80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 -- 172.21.15.37:0/3487460457 shutdown_connections 2024-09-21T06:31:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.590+0000 7fb7b0200640 1 -- 172.21.15.37:0/3487460457 wait complete. 2024-09-21T06:31:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.592+0000 7fb7b0200640 1 Processor -- start 2024-09-21T06:31:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.592+0000 7fb7b0200640 1 -- start start 2024-09-21T06:31:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 0x7fb7a809ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a81f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a806df70 con 0x7fb7a80cefa0 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a806ddf0 con 0x7fb7a80ce860 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7b0200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a806e0f0 con 0x7fb7a815b9f0 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7ace00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7ace00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59166/0 (socket says 172.21.15.37:59166) 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7ace00640 1 -- 172.21.15.37:0/4122831430 learned_addr learned my addr 172.21.15.37:0/4122831430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7ad800640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 0x7fb7a809ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.594+0000 7fb7ae200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a81f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 -- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 msgr2=0x7fb7a81f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a81f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 -- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 msgr2=0x7fb7a809ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 0x7fb7a809ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 -- 172.21.15.37:0/4122831430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a81f3640 con 0x7fb7a80cefa0 2024-09-21T06:31:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ad800640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 0x7fb7a809ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7ace00640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e9f0 tx=0x7fb79400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79400cd80 con 0x7fb7a80cefa0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb794004020 con 0x7fb7a80cefa0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794004450 con 0x7fb7a80cefa0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a8156bf0 con 0x7fb7a80cefa0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.595+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81f9440 con 0x7fb7a80cefa0 2024-09-21T06:31:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.596+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a806e5e0 con 0x7fb7a80cefa0 2024-09-21T06:31:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.598+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7940035f0 con 0x7fb7a80cefa0 2024-09-21T06:31:30.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.599+0000 7fb7a5600640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 0x7fb78807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:30.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.599+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7fb79409b690 con 0x7fb7a80cefa0 2024-09-21T06:31:30.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.599+0000 7fb7ad800640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 0x7fb78807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:30.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.600+0000 7fb7ad800640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 0x7fb78807b0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb79c034000 tx=0x7fb79c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.603+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7a81f3640 con 0x7fb7a80cefa0 2024-09-21T06:31:30.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:30.877+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7fb7a8075050 con 0x7fb7a80cefa0 2024-09-21T06:31:31.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.400+0000 7fb7a5600640 1 -- 172.21.15.37:0/4122831430 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v201) ==== 145+0+0 (secure 0 0 0) 0x7fb794063af0 con 0x7fb7a80cefa0 2024-09-21T06:31:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 msgr2=0x7fb78807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 0x7fb78807b0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb79c034000 tx=0x7fb79c004730 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 msgr2=0x7fb7a81f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e9f0 tx=0x7fb79400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 shutdown_connections 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb788078bc0 0x7fb78807b0d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a80cefa0 0x7fb7a81f9c40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a815b9f0 0x7fb7a81f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 --2- 172.21.15.37:0/4122831430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a80ce860 0x7fb7a809ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 >> 172.21.15.37:0/4122831430 conn(0x7fb7a80cca40 msgr2=0x7fb7a815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 shutdown_connections 2024-09-21T06:31:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.403+0000 7fb7b0200640 1 -- 172.21.15.37:0/4122831430 wait complete. 2024-09-21T06:31:31.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:31:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 -- 172.21.15.37:0/4245570152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0cddc0 msgr2=0x7fba5c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 --2- 172.21.15.37:0/4245570152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0cddc0 0x7fba5c0cc810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fba5c070fc0 tx=0x7fba5002a4d0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 -- 172.21.15.37:0/4245570152 shutdown_connections 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 --2- 172.21.15.37:0/4245570152 >> v2:172.21.15.150:3300/0 conn(0x7fba5c0b4ed0 0x7fba5c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 --2- 172.21.15.37:0/4245570152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 0x7fba5c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 --2- 172.21.15.37:0/4245570152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0cddc0 0x7fba5c0cc810 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 -- 172.21.15.37:0/4245570152 >> 172.21.15.37:0/4245570152 conn(0x7fba5c06f940 msgr2=0x7fba5c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 -- 172.21.15.37:0/4245570152 shutdown_connections 2024-09-21T06:31:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.657+0000 7fba62000640 1 -- 172.21.15.37:0/4245570152 wait complete. 2024-09-21T06:31:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.659+0000 7fba62000640 1 Processor -- start 2024-09-21T06:31:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.659+0000 7fba62000640 1 -- start start 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 0x7fba5c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 0x7fba5c159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c06e3e0 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c06e560 con 0x7fba5c0cddc0 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba62000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fba5c15d460 con 0x7fba5c0ccd50 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba60c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba60c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59176/0 (socket says 172.21.15.37:59176) 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba60c00640 1 -- 172.21.15.37:0/3251056252 learned_addr learned my addr 172.21.15.37:0/3251056252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba61600640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 0x7fba5c159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.661+0000 7fba5be00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 0x7fba5c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 -- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 msgr2=0x7fba5c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 0x7fba5c1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 -- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 msgr2=0x7fba5c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 0x7fba5c159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 -- 172.21.15.37:0/3251056252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c153c60 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba61600640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 0x7fba5c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba60c00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fba50034040 tx=0x7fba500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5003e030 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba50003db0 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50035040 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.662+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1f3e10 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.663+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1f43d0 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.664+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5c15d980 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.665+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba50041030 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.666+0000 7fba58c00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 0x7fba3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.666+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7fba500bb590 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.666+0000 7fba5be00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 0x7fba3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.667+0000 7fba5be00640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 0x7fba3c07b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba5c1f1140 tx=0x7fba48005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:31.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.670+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba500834d0 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.871+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fba5c0cdaa0 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.872+0000 7fba58c00640 1 -- 172.21.15.37:0/3251056252 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7fba50087f00 con 0x7fba5c0b4ed0 2024-09-21T06:31:31.874 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:31:31.874 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:31:31.875 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:31:31.876 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:31:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:31:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:31:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:31:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:31:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 msgr2=0x7fba3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 0x7fba3c07b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba5c1f1140 tx=0x7fba48005f90 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 msgr2=0x7fba5c1f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fba50034040 tx=0x7fba500038a0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 shutdown_connections 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fba3c078b90 0x7fba3c07b0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba5c0cddc0 0x7fba5c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba5c0ccd50 0x7fba5c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 --2- 172.21.15.37:0/3251056252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c0b4ed0 0x7fba5c1f35e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 >> 172.21.15.37:0/3251056252 conn(0x7fba5c06f940 msgr2=0x7fba5c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:31.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 shutdown_connections 2024-09-21T06:31:31.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:31.875+0000 7fba62000640 1 -- 172.21.15.37:0/3251056252 wait complete. 2024-09-21T06:31:31.899 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 -- 172.21.15.37:0/2858107948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00b4f90 msgr2=0x7fe5b006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 --2- 172.21.15.37:0/2858107948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00b4f90 0x7fe5b006d780 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe598005fd0 tx=0x7fe59802a520 comp rx=0 tx=0).stop 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 -- 172.21.15.37:0/2858107948 shutdown_connections 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 --2- 172.21.15.37:0/2858107948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b00cc8d0 0x7fe5b00ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 --2- 172.21.15.37:0/2858107948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00b4f90 0x7fe5b006d780 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 -- 172.21.15.37:0/2858107948 >> 172.21.15.37:0/2858107948 conn(0x7fe5b006f920 msgr2=0x7fe5b00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 -- 172.21.15.37:0/2858107948 shutdown_connections 2024-09-21T06:31:32.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.126+0000 7fe5afe00640 1 -- 172.21.15.37:0/2858107948 wait complete. 2024-09-21T06:31:32.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.129+0000 7fe5afe00640 1 Processor -- start 2024-09-21T06:31:32.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.129+0000 7fe5afe00640 1 -- start start 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5b00b4f90 0x7fe5b015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b01456e0 0x7fe5b01e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b00ce150 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b00cdfd0 con 0x7fe5b00b4f90 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5afe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b00ce2d0 con 0x7fe5b01456e0 2024-09-21T06:31:32.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59184/0 (socket says 172.21.15.37:59184) 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.130+0000 7fe5a7e00640 1 -- 172.21.15.37:0/3495003918 learned_addr learned my addr 172.21.15.37:0/3495003918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5af400640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b01456e0 0x7fe5b01e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5aea00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5b00b4f90 0x7fe5b015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 -- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b01456e0 msgr2=0x7fe5b01e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b01456e0 0x7fe5b01e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 -- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5b00b4f90 msgr2=0x7fe5b015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5b00b4f90 0x7fe5b015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 -- 172.21.15.37:0/3495003918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01f92a0 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a7e00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe598000c00 tx=0x7fe598003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59803e030 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe598003e00 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035040 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.131+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01f9580 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.132+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01f9b40 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.133+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b00ce7a0 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.134+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe598041030 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.135+0000 7fe5a6a00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 0x7fe58c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.135+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7fe5980bb530 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.135+0000 7fe5aea00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 0x7fe58c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.136+0000 7fe5aea00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 0x7fe58c07b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0006fd0 tx=0x7fe5a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.139+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe5b00cd220 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.336+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe5b00b0bd0 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.341+0000 7fe5a6a00640 1 -- 172.21.15.37:0/3495003918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fe598083990 con 0x7fe5b00cc8d0 2024-09-21T06:31:32.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 msgr2=0x7fe58c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 0x7fe58c07b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0006fd0 tx=0x7fe5a0008040 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 msgr2=0x7fe5b0140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe598000c00 tx=0x7fe598003910 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 shutdown_connections 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe58c078b90 0x7fe58c07b0a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00cc8d0 0x7fe5b0140fd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5b01456e0 0x7fe5b01e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 --2- 172.21.15.37:0/3495003918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5b00b4f90 0x7fe5b015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 >> 172.21.15.37:0/3495003918 conn(0x7fe5b006f920 msgr2=0x7fe5b015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 shutdown_connections 2024-09-21T06:31:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.343+0000 7fe5afe00640 1 -- 172.21.15.37:0/3495003918 wait complete. 2024-09-21T06:31:32.367 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:31:32.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:31:32.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:31:32.543 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:31:32.543 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:32.543 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:32.579 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:31:32.579 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:31:32.579 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:32.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:31:32.616 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:32.616 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:32.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:31:32.654 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:31:32.654 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:32.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:31:32.693 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:31:32.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 -- 172.21.15.37:0/3817388884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c06e330 msgr2=0x7ff66c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 --2- 172.21.15.37:0/3817388884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c06e330 0x7ff66c0cdb90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff664005fd0 tx=0x7ff66402aa80 comp rx=0 tx=0).stop 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 -- 172.21.15.37:0/3817388884 shutdown_connections 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 --2- 172.21.15.37:0/3817388884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c06e330 0x7ff66c0cdb90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 --2- 172.21.15.37:0/3817388884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 0x7ff66c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 -- 172.21.15.37:0/3817388884 >> 172.21.15.37:0/3817388884 conn(0x7ff66c0cca40 msgr2=0x7ff66c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 -- 172.21.15.37:0/3817388884 shutdown_connections 2024-09-21T06:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.917+0000 7ff672200640 1 -- 172.21.15.37:0/3817388884 wait complete. 2024-09-21T06:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.920+0000 7ff672200640 1 Processor -- start 2024-09-21T06:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.920+0000 7ff672200640 1 -- start start 2024-09-21T06:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff66c06e330 0x7ff66c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 0x7ff66c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c071000 con 0x7ff66c074d90 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c070e80 con 0x7ff66c06e330 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.921+0000 7ff672200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c0ce0d0 con 0x7ff66c070030 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59202/0 (socket says 172.21.15.37:59202) 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 -- 172.21.15.37:0/702011939 learned_addr learned my addr 172.21.15.37:0/702011939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66b400640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff66c06e330 0x7ff66c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66aa00640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 0x7ff66c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 -- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 msgr2=0x7ff66c154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 0x7ff66c154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 -- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff66c06e330 msgr2=0x7ff66c0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff66c06e330 0x7ff66c0c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.922+0000 7ff66be00640 1 -- 172.21.15.37:0/702011939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1553e0 con 0x7ff66c074d90 2024-09-21T06:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff66be00640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff664009d90 tx=0x7ff664003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664044070 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff664003e50 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c15af80 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640044b0 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.923+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c15b540 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.924+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c0ce5e0 con 0x7ff66c074d90 2024-09-21T06:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.925+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff664047070 con 0x7ff66c074d90 2024-09-21T06:31:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.926+0000 7ff663400640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 0x7ff64c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.926+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7ff6640bbb90 con 0x7ff66c074d90 2024-09-21T06:31:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.927+0000 7ff66b400640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 0x7ff64c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.927+0000 7ff66b400640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 0x7ff64c07b0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f60 tx=0x7ff65c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:32.931+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff66c1553e0 con 0x7ff66c074d90 2024-09-21T06:31:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.169+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff66c0b5f40 con 0x7ff66c074d90 2024-09-21T06:31:33.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.170+0000 7ff663400640 1 -- 172.21.15.37:0/702011939 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+3917 (secure 0 0 0) 0x7ff66c06e740 con 0x7ff66c074d90 2024-09-21T06:31:33.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:33.172 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":45,"btime":"2024-09-21T06:31:25:369071+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3220174481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3220174481},{"type":"v1","addr":"172.21.15.150:6837","nonce":3220174481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2884291606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2884291606},{"type":"v1","addr":"172.21.15.150:6835","nonce":2884291606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3079851475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3079851475},{"type":"v1","addr":"172.21.15.37:6833","nonce":3079851475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2069798882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2069798882},{"type":"v1","addr":"172.21.15.37:6835","nonce":2069798882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[]} 2024-09-21T06:31:33.172 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 45 2024-09-21T06:31:33.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 msgr2=0x7ff64c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 0x7ff64c07b0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f60 tx=0x7ff65c009040 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 msgr2=0x7ff66c0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff664009d90 tx=0x7ff664003ce0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 shutdown_connections 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff64c078bc0 0x7ff64c07b0d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c074d90 0x7ff66c0b53b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff66c070030 0x7ff66c154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 --2- 172.21.15.37:0/702011939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff66c06e330 0x7ff66c0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 >> 172.21.15.37:0/702011939 conn(0x7ff66c0cca40 msgr2=0x7ff66c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 shutdown_connections 2024-09-21T06:31:33.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.172+0000 7ff672200640 1 -- 172.21.15.37:0/702011939 wait complete. 2024-09-21T06:31:33.197 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:31:33.197 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:31:33.197 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:31:33.198 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:31:33.198 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:31:33.198 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:31:33.198 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:31:33.198 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:31:33.198 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:31:33.198 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:31:33.199 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:31:33.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:31:33.199 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:31:33.199 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:31:33.199 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:31:33.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:31:33.232 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:31:33.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:31:33.233 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:31:33.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:31:33.235 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:31:33.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:31:33.236 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:31:33.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:31:33.278 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:31:33.278 DEBUG:teuthology.parallel:result is None 2024-09-21T06:31:33.280 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:31:33.280 DEBUG:teuthology.parallel:result is None 2024-09-21T06:31:33.281 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:31:33.281 DEBUG:teuthology.parallel:result is None 2024-09-21T06:31:33.284 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:31:33.284 DEBUG:teuthology.parallel:result is None 2024-09-21T06:31:33.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:31:33.475 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:31:33.475 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:31:33.482 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:31:33.483 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:31:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 -- 172.21.15.37:0/1549975435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b7710 msgr2=0x7ff2e00ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1549975435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b7710 0x7ff2e00ced40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005fd0 tx=0x7ff2c802a670 comp rx=0 tx=0).stop 2024-09-21T06:31:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 -- 172.21.15.37:0/1549975435 shutdown_connections 2024-09-21T06:31:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1549975435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b7710 0x7ff2e00ced40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 -- 172.21.15.37:0/1549975435 >> 172.21.15.37:0/1549975435 conn(0x7ff2e00d1960 msgr2=0x7ff2e00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.522+0000 7ff2e5800640 1 -- 172.21.15.37:0/1549975435 shutdown_connections 2024-09-21T06:31:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.523+0000 7ff2e5800640 1 -- 172.21.15.37:0/1549975435 wait complete. 2024-09-21T06:31:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.525+0000 7ff2e5800640 1 Processor -- start 2024-09-21T06:31:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.525+0000 7ff2e5800640 1 -- start start 2024-09-21T06:31:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 0x7ff2e00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2e00b7710 0x7ff2e0154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0169a80 con 0x7ff2e00b36c0 2024-09-21T06:31:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0169900 con 0x7ff2e006e070 2024-09-21T06:31:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2e5800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e0169c00 con 0x7ff2e00b7710 2024-09-21T06:31:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.526+0000 7ff2d7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59270/0 (socket says 172.21.15.37:59270) 2024-09-21T06:31:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 -- 172.21.15.37:0/1338917303 learned_addr learned my addr 172.21.15.37:0/1338917303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2e4e00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2e00b7710 0x7ff2e0154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2dfe00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 0x7ff2e00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 -- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2e00b7710 msgr2=0x7ff2e0154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2e00b7710 0x7ff2e0154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 -- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 msgr2=0x7ff2e00b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 0x7ff2e00b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 -- 172.21.15.37:0/1338917303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e0152470 con 0x7ff2e00b36c0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2dfe00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 0x7ff2e00b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.527+0000 7ff2d7e00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8000c00 tx=0x7ff2c8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.528+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c803e030 con 0x7ff2e00b36c0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.528+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2c8003e60 con 0x7ff2e00b36c0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.528+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035040 con 0x7ff2e00b36c0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.528+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e0155bf0 con 0x7ff2e00b36c0 2024-09-21T06:31:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.528+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e0156690 con 0x7ff2e00b36c0 2024-09-21T06:31:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.529+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e0155960 con 0x7ff2e00b36c0 2024-09-21T06:31:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.530+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2c8041030 con 0x7ff2e00b36c0 2024-09-21T06:31:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.531+0000 7ff2dcc00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 0x7ff2bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.531+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5507+0+0 (secure 0 0 0) 0x7ff2c80bb580 con 0x7ff2e00b36c0 2024-09-21T06:31:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.532+0000 7ff2dfe00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 0x7ff2bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.532+0000 7ff2dfe00640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 0x7ff2bc07b0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0001b10 tx=0x7ff2d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.536+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2c8083310 con 0x7ff2e00b36c0 2024-09-21T06:31:33.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:33.748+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff2e01696e0 con 0x7ff2e00b36c0 2024-09-21T06:31:34.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.423+0000 7ff2dcc00640 1 -- 172.21.15.37:0/1338917303 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7ff2c8087d40 con 0x7ff2e00b36c0 2024-09-21T06:31:34.425 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:31:34.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 msgr2=0x7ff2bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 0x7ff2bc07b0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0001b10 tx=0x7ff2d0009040 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 msgr2=0x7ff2e0151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8000c00 tx=0x7ff2c8003970 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 shutdown_connections 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff2bc078bc0 0x7ff2bc07b0d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e00b36c0 0x7ff2e0151f30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2e00b7710 0x7ff2e0154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 --2- 172.21.15.37:0/1338917303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e006e070 0x7ff2e00b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 >> 172.21.15.37:0/1338917303 conn(0x7ff2e00d1960 msgr2=0x7ff2e015d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 shutdown_connections 2024-09-21T06:31:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.426+0000 7ff2e5800640 1 -- 172.21.15.37:0/1338917303 wait complete. 2024-09-21T06:31:34.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 -- 172.21.15.37:0/3802635817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 msgr2=0x7fc87c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 --2- 172.21.15.37:0/3802635817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c0cc020 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc87c076360 tx=0x7fc86802a560 comp rx=0 tx=0).stop 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 -- 172.21.15.37:0/3802635817 shutdown_connections 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 --2- 172.21.15.37:0/3802635817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c06ddf0 0x7fc87c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 --2- 172.21.15.37:0/3802635817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c0cc020 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 --2- 172.21.15.37:0/3802635817 >> v2:172.21.15.150:3300/0 conn(0x7fc87c074cb0 0x7fc87c073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 -- 172.21.15.37:0/3802635817 >> 172.21.15.37:0/3802635817 conn(0x7fc87c0cca40 msgr2=0x7fc87c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 -- 172.21.15.37:0/3802635817 shutdown_connections 2024-09-21T06:31:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.687+0000 7fc883400640 1 -- 172.21.15.37:0/3802635817 wait complete. 2024-09-21T06:31:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.689+0000 7fc883400640 1 Processor -- start 2024-09-21T06:31:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.690+0000 7fc883400640 1 -- start start 2024-09-21T06:31:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc87c06ddf0 0x7fc87c09f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c074cb0 0x7fc87c09d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c06f170 con 0x7fc87c0cbc10 2024-09-21T06:31:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c06eff0 con 0x7fc87c06ddf0 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc883400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c06f2f0 con 0x7fc87c074cb0 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc881400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc881400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34660/0 (socket says 172.21.15.37:34660) 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc881400640 1 -- 172.21.15.37:0/774336605 learned_addr learned my addr 172.21.15.37:0/774336605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc87be00640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c074cb0 0x7fc87c09d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.691+0000 7fc880a00640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc87c06ddf0 0x7fc87c09f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 -- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c074cb0 msgr2=0x7fc87c09d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c074cb0 0x7fc87c09d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 -- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc87c06ddf0 msgr2=0x7fc87c09f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc87c06ddf0 0x7fc87c09f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 -- 172.21.15.37:0/774336605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c09d620 con 0x7fc87c0cbc10 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc881400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc87400ba10 tx=0x7fc87400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874019070 con 0x7fc87c0cbc10 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc87400cc50 con 0x7fc87c0cbc10 2024-09-21T06:31:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.692+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c09e300 con 0x7fc87c0cbc10 2024-09-21T06:31:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.693+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740146a0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.693+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c09e8c0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.694+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c073dd0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.695+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc87401c070 con 0x7fc87c0cbc10 2024-09-21T06:31:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.696+0000 7fc878c00640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 0x7fc85c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.696+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fc87409a8d0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.697+0000 7fc880a00640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 0x7fc85c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.697+0000 7fc880a00640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 0x7fc85c07b0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc87c15a730 tx=0x7fc87000a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:34.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.700+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8740627e0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.975+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc87c1590c0 con 0x7fc87c0cbc10 2024-09-21T06:31:34.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.975+0000 7fc878c00640 1 -- 172.21.15.37:0/774336605 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7fc874067210 con 0x7fc87c0cbc10 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:31:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:31:34.991 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 msgr2=0x7fc85c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 0x7fc85c07b0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc87c15a730 tx=0x7fc87000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 msgr2=0x7fc87c09bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc87400ba10 tx=0x7fc87400bed0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 shutdown_connections 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc85c078b90 0x7fc85c07b0a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc87c0cbc10 0x7fc87c09bb60 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc87c074cb0 0x7fc87c09d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 --2- 172.21.15.37:0/774336605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc87c06ddf0 0x7fc87c09f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 >> 172.21.15.37:0/774336605 conn(0x7fc87c0cca40 msgr2=0x7fc87c0cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 shutdown_connections 2024-09-21T06:31:34.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:34.978+0000 7fc883400640 1 -- 172.21.15.37:0/774336605 wait complete. 2024-09-21T06:31:35.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 -- 172.21.15.37:0/2425851878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181415dfe0 msgr2=0x7f1814160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 --2- 172.21.15.37:0/2425851878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181415dfe0 0x7f1814160a40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1808005fd0 tx=0x7f180802a540 comp rx=0 tx=0).stop 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 -- 172.21.15.37:0/2425851878 shutdown_connections 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 --2- 172.21.15.37:0/2425851878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181406e160 0x7f181406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 --2- 172.21.15.37:0/2425851878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181415dfe0 0x7f1814160a40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 -- 172.21.15.37:0/2425851878 >> 172.21.15.37:0/2425851878 conn(0x7f181406f940 msgr2=0x7f18140b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 -- 172.21.15.37:0/2425851878 shutdown_connections 2024-09-21T06:31:35.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.228+0000 7f181a200640 1 -- 172.21.15.37:0/2425851878 wait complete. 2024-09-21T06:31:35.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.230+0000 7f181a200640 1 Processor -- start 2024-09-21T06:31:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.231+0000 7f181a200640 1 -- start start 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181406e160 0x7f18141f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 0x7f18140b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18140b4c60 con 0x7f18140cc8d0 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18140b4ae0 con 0x7f181406e160 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f181a200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f18140b4de0 con 0x7f181415dfe0 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f1813e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f1813e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34690/0 (socket says 172.21.15.37:34690) 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.232+0000 7f1813e00640 1 -- 172.21.15.37:0/1206795390 learned_addr learned my addr 172.21.15.37:0/1206795390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1818e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181406e160 0x7f18141f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1819800640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 0x7f18140b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 -- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 msgr2=0x7f18140b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 0x7f18140b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 -- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181406e160 msgr2=0x7f18141f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181406e160 0x7f18141f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 -- 172.21.15.37:0/1206795390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1814135e60 con 0x7f18140cc8d0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1819800640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 0x7f18140b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1813e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f180000d650 tx=0x7f180000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800017070 con 0x7f18140cc8d0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1800005310 con 0x7f18140cc8d0 2024-09-21T06:31:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141f9b00 con 0x7f18140cc8d0 2024-09-21T06:31:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.233+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800009820 con 0x7f18140cc8d0 2024-09-21T06:31:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.234+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141301a0 con 0x7f18140cc8d0 2024-09-21T06:31:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.235+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18141fa020 con 0x7f18140cc8d0 2024-09-21T06:31:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.236+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18000099c0 con 0x7f18140cc8d0 2024-09-21T06:31:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.237+0000 7f1810c00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 0x7f17f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.237+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f180009ac60 con 0x7f18140cc8d0 2024-09-21T06:31:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.237+0000 7f1818e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 0x7f17f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.238+0000 7f1818e00640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 0x7f17f407b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1808033000 tx=0x7f1808004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.242+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18141fa020 con 0x7f18140cc8d0 2024-09-21T06:31:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.446+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f181406dce0 con 0x7f18140cc8d0 2024-09-21T06:31:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.446+0000 7f1810c00640 1 -- 172.21.15.37:0/1206795390 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f181406dce0 con 0x7f18140cc8d0 2024-09-21T06:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 msgr2=0x7f17f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 0x7f17f407b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1808033000 tx=0x7f1808004730 comp rx=0 tx=0).stop 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 msgr2=0x7f1814135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f180000d650 tx=0x7f180000db10 comp rx=0 tx=0).stop 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 shutdown_connections 2024-09-21T06:31:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17f4078bc0 0x7f17f407b0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18140cc8d0 0x7f1814135920 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f181415dfe0 0x7f18140b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 --2- 172.21.15.37:0/1206795390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181406e160 0x7f18141f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 >> 172.21.15.37:0/1206795390 conn(0x7f181406f940 msgr2=0x7f18140ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 shutdown_connections 2024-09-21T06:31:35.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.449+0000 7f181a200640 1 -- 172.21.15.37:0/1206795390 wait complete. 2024-09-21T06:31:35.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:31:35.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:31:35.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 -- 172.21.15.37:0/763622519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f406f6f0 msgr2=0x7f51f4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 --2- 172.21.15.37:0/763622519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f406f6f0 0x7f51f4074c50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005fd0 tx=0x7f51e402a530 comp rx=0 tx=0).stop 2024-09-21T06:31:35.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 -- 172.21.15.37:0/763622519 shutdown_connections 2024-09-21T06:31:35.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 --2- 172.21.15.37:0/763622519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f415e9d0 0x7f51f4073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 --2- 172.21.15.37:0/763622519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f406f6f0 0x7f51f4074c50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 -- 172.21.15.37:0/763622519 >> 172.21.15.37:0/763622519 conn(0x7f51f40d1140 msgr2=0x7f51f40d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:35.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 -- 172.21.15.37:0/763622519 shutdown_connections 2024-09-21T06:31:35.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.705+0000 7f51f8a00640 1 -- 172.21.15.37:0/763622519 wait complete. 2024-09-21T06:31:35.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.707+0000 7f51f8a00640 1 Processor -- start 2024-09-21T06:31:35.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.707+0000 7f51f8a00640 1 -- start start 2024-09-21T06:31:35.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.708+0000 7f51f8a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f406f6f0 0x7f51f409b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f8a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f415e9d0 0x7f51f40cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f8a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51f406e600 con 0x7f51f4074490 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f8a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f51f40d04d0 con 0x7f51f415e9d0 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f8a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f51f406e780 con 0x7f51f406f6f0 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34710/0 (socket says 172.21.15.37:34710) 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 -- 172.21.15.37:0/3600789520 learned_addr learned my addr 172.21.15.37:0/3600789520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f1e00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f406f6f0 0x7f51f409b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51f2800640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f415e9d0 0x7f51f40cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 -- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f406f6f0 msgr2=0x7f51f409b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f406f6f0 0x7f51f409b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 -- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f415e9d0 msgr2=0x7f51f40cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f415e9d0 0x7f51f40cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:35.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.709+0000 7f51ebe00640 1 -- 172.21.15.37:0/3600789520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f409c1f0 con 0x7f51f4074490 2024-09-21T06:31:35.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51ebe00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c9a0 tx=0x7f51dc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:35.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc007b50 con 0x7f51f4074490 2024-09-21T06:31:35.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc007de0 con 0x7f51f4074490 2024-09-21T06:31:35.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f409b930 con 0x7f51f4074490 2024-09-21T06:31:35.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00f6b0 con 0x7f51f4074490 2024-09-21T06:31:35.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.710+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f409c490 con 0x7f51f4074490 2024-09-21T06:31:35.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.711+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f406eca0 con 0x7f51f4074490 2024-09-21T06:31:35.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.712+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51dc00f850 con 0x7f51f4074490 2024-09-21T06:31:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.713+0000 7f51ea000640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 0x7f51d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.713+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f51dc09b300 con 0x7f51f4074490 2024-09-21T06:31:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.714+0000 7f51f1e00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 0x7f51d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.715+0000 7f51f1e00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 0x7f51d007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f51e4033000 tx=0x7f51e4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.718+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51dc014030 con 0x7f51f4074490 2024-09-21T06:31:35.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:35.915+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f51f406a680 con 0x7f51f4074490 2024-09-21T06:31:36.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.447+0000 7f51ea000640 1 -- 172.21.15.37:0/3600789520 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7f51dc0638f0 con 0x7f51f4074490 2024-09-21T06:31:36.449 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:31:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 msgr2=0x7f51d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 0x7f51d007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f51e4033000 tx=0x7f51e4004730 comp rx=0 tx=0).stop 2024-09-21T06:31:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 msgr2=0x7f51f4159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c9a0 tx=0x7f51dc00ce60 comp rx=0 tx=0).stop 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 shutdown_connections 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f51d0078b90 0x7f51d007b0a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51f4074490 0x7f51f4159190 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f415e9d0 0x7f51f40cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 --2- 172.21.15.37:0/3600789520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51f406f6f0 0x7f51f409b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 >> 172.21.15.37:0/3600789520 conn(0x7f51f40d1140 msgr2=0x7f51f40d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 shutdown_connections 2024-09-21T06:31:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.450+0000 7f51f8a00640 1 -- 172.21.15.37:0/3600789520 wait complete. 2024-09-21T06:31:36.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 -- 172.21.15.37:0/3780784449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 msgr2=0x7fa344004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 --2- 172.21.15.37:0/3780784449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa344004080 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa350001800 tx=0x7fa344005830 comp rx=0 tx=0).stop 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 -- 172.21.15.37:0/3780784449 shutdown_connections 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 --2- 172.21.15.37:0/3780784449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa344004080 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 -- 172.21.15.37:0/3780784449 >> 172.21.15.37:0/3780784449 conn(0x7fa35c0cca40 msgr2=0x7fa35c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 -- 172.21.15.37:0/3780784449 shutdown_connections 2024-09-21T06:31:36.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.719+0000 7fa363600640 1 -- 172.21.15.37:0/3780784449 wait complete. 2024-09-21T06:31:36.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.721+0000 7fa363600640 1 Processor -- start 2024-09-21T06:31:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.722+0000 7fa363600640 1 -- start start 2024-09-21T06:31:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35c0b1090 0x7fa35c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 0x7fa35c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0cc0b0 con 0x7fa344003c30 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0cbf30 con 0x7fa35c0b1090 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa363600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c0cc230 con 0x7fa35c0b2e10 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa360c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa360c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34734/0 (socket says 172.21.15.37:34734) 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.723+0000 7fa360c00640 1 -- 172.21.15.37:0/3108222558 learned_addr learned my addr 172.21.15.37:0/3108222558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa35be00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35c0b1090 0x7fa35c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa361600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 0x7fa35c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 -- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 msgr2=0x7fa35c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 0x7fa35c070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 -- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35c0b1090 msgr2=0x7fa35c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35c0b1090 0x7fa35c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 -- 172.21.15.37:0/3108222558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c1e7840 con 0x7fa344003c30 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa361600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 0x7fa35c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa360c00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa350001800 tx=0x7fa350006170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.724+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350043070 con 0x7fa344003c30 2024-09-21T06:31:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.725+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa350003af0 con 0x7fa344003c30 2024-09-21T06:31:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.725+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1e7b20 con 0x7fa344003c30 2024-09-21T06:31:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.725+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350033480 con 0x7fa344003c30 2024-09-21T06:31:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.725+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c157000 con 0x7fa344003c30 2024-09-21T06:31:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.726+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c06dfb0 con 0x7fa344003c30 2024-09-21T06:31:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.727+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa350046070 con 0x7fa344003c30 2024-09-21T06:31:36.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.728+0000 7fa358c00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 0x7fa33c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.728+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5619+0+0 (secure 0 0 0) 0x7fa3500bb1b0 con 0x7fa344003c30 2024-09-21T06:31:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.729+0000 7fa35be00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 0x7fa33c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.729+0000 7fa35be00640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 0x7fa33c07b0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0c3560 tx=0x7fa348001110 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.732+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa350006260 con 0x7fa344003c30 2024-09-21T06:31:36.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:36.932+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa35c158ce0 con 0x7fa344003c30 2024-09-21T06:31:37.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.454+0000 7fa358c00640 1 -- 172.21.15.37:0/3108222558 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fa350083770 con 0x7fa344003c30 2024-09-21T06:31:37.456 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.457+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 msgr2=0x7fa33c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.457+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 0x7fa33c07b0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0c3560 tx=0x7fa348001110 comp rx=0 tx=0).stop 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.457+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 msgr2=0x7fa35c150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.457+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa350001800 tx=0x7fa350006170 comp rx=0 tx=0).stop 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 shutdown_connections 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa33c078bc0 0x7fa33c07b0d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa35c0b2e10 0x7fa35c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35c0b1090 0x7fa35c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 --2- 172.21.15.37:0/3108222558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa344003c30 0x7fa35c150e60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 >> 172.21.15.37:0/3108222558 conn(0x7fa35c0cca40 msgr2=0x7fa35c15b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 shutdown_connections 2024-09-21T06:31:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.458+0000 7fa363600640 1 -- 172.21.15.37:0/3108222558 wait complete. 2024-09-21T06:31:37.504 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:31:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 -- 172.21.15.37:0/3345422644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 msgr2=0x7f9d980b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 --2- 172.21.15.37:0/3345422644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d980b4f90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005fd0 tx=0x7f9d8802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 -- 172.21.15.37:0/3345422644 shutdown_connections 2024-09-21T06:31:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 --2- 172.21.15.37:0/3345422644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d980b4f90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 -- 172.21.15.37:0/3345422644 >> 172.21.15.37:0/3345422644 conn(0x7f9d9806f920 msgr2=0x7f9d980b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:37.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.733+0000 7f9d97e00640 1 -- 172.21.15.37:0/3345422644 shutdown_connections 2024-09-21T06:31:37.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.734+0000 7f9d97e00640 1 -- 172.21.15.37:0/3345422644 wait complete. 2024-09-21T06:31:37.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.736+0000 7f9d97e00640 1 Processor -- start 2024-09-21T06:31:37.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.736+0000 7f9d97e00640 1 -- start start 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 0x7f9d981502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 0x7f9d98150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9806d7d0 con 0x7f9d9806e370 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9806d950 con 0x7f9d980cf160 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.737+0000 7f9d97e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9806dad0 con 0x7f9d98156c60 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34746/0 (socket says 172.21.15.37:34746) 2024-09-21T06:31:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 -- 172.21.15.37:0/996647402 learned_addr learned my addr 172.21.15.37:0/996647402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d97400640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 0x7f9d98150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d8fe00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 0x7f9d981502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 -- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 msgr2=0x7f9d981502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 0x7f9d981502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 -- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 msgr2=0x7f9d98150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 0x7f9d98150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 -- 172.21.15.37:0/996647402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981574b0 con 0x7f9d9806e370 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d97400640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 0x7f9d98150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d8fe00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 0x7f9d981502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.738+0000 7f9d96a00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9d88034040 tx=0x7f9d880038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.739+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8803e030 con 0x7f9d9806e370 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.739+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d88003ca0 con 0x7f9d9806e370 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.739+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d98157790 con 0x7f9d9806e370 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.739+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88035040 con 0x7f9d9806e370 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.739+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d98157d50 con 0x7f9d9806e370 2024-09-21T06:31:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.740+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d980b4f90 con 0x7f9d9806e370 2024-09-21T06:31:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.742+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d88041030 con 0x7f9d9806e370 2024-09-21T06:31:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.742+0000 7f9d8ea00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 0x7f9d7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.743+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6001+0+0 (secure 0 0 0) 0x7f9d880bb800 con 0x7f9d9806e370 2024-09-21T06:31:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.743+0000 7f9d8fe00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 0x7f9d7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:37.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.744+0000 7f9d8fe00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 0x7f9d7407b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9d9814f230 tx=0x7f9d80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.746+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d88083c10 con 0x7f9d9806e370 2024-09-21T06:31:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:37.987+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9d98074c10 con 0x7f9d9806e370 2024-09-21T06:31:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.456+0000 7f9d8ea00640 1 -- 172.21.15.37:0/996647402 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v48) ==== 173+0+0 (secure 0 0 0) 0x7f9d88083510 con 0x7f9d9806e370 2024-09-21T06:31:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 9 and data pool 10 2024-09-21T06:31:38.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 msgr2=0x7f9d7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 0x7f9d7407b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9d9814f230 tx=0x7f9d80009040 comp rx=0 tx=0).stop 2024-09-21T06:31:38.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 msgr2=0x7f9d9814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9d88034040 tx=0x7f9d880038a0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 shutdown_connections 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d74078b90 0x7f9d7407b0a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d980cf160 0x7f9d98150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d98156c60 0x7f9d981502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.460+0000 7f9d97e00640 1 --2- 172.21.15.37:0/996647402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d9806e370 0x7f9d9814e890 secure :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9d88034040 tx=0x7f9d880038a0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.461+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 >> 172.21.15.37:0/996647402 conn(0x7f9d9806f920 msgr2=0x7f9d9815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.461+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 shutdown_connections 2024-09-21T06:31:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.461+0000 7f9d97e00640 1 -- 172.21.15.37:0/996647402 wait complete. 2024-09-21T06:31:38.511 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:38.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 -- 172.21.15.37:0/1103093002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 msgr2=0x7f43cc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 --2- 172.21.15.37:0/1103093002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc15e8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43c000a940 tx=0x7f43c002b880 comp rx=0 tx=0).stop 2024-09-21T06:31:38.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 -- 172.21.15.37:0/1103093002 shutdown_connections 2024-09-21T06:31:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 --2- 172.21.15.37:0/1103093002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc15e8a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 -- 172.21.15.37:0/1103093002 >> 172.21.15.37:0/1103093002 conn(0x7f43cc06f940 msgr2=0x7f43cc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 -- 172.21.15.37:0/1103093002 shutdown_connections 2024-09-21T06:31:38.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.737+0000 7f43d3000640 1 -- 172.21.15.37:0/1103093002 wait complete. 2024-09-21T06:31:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.739+0000 7f43d3000640 1 Processor -- start 2024-09-21T06:31:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.739+0000 7f43d3000640 1 -- start start 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43cc15ede0 0x7f43cc0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc074b90 con 0x7f43cc140da0 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc070f00 con 0x7f43cc15b820 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d3000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc0ce4d0 con 0x7f43cc15ede0 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34758/0 (socket says 172.21.15.37:34758) 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d2600640 1 -- 172.21.15.37:0/1451883183 learned_addr learned my addr 172.21.15.37:0/1451883183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d1c00640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d1200640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43cc15ede0 0x7f43cc0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d2600640 1 -- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43cc15ede0 msgr2=0x7f43cc0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.741+0000 7f43d2600640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43cc15ede0 0x7f43cc0ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d2600640 1 -- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 msgr2=0x7f43cc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d2600640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d2600640 1 -- 172.21.15.37:0/1451883183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc1f62f0 con 0x7f43cc140da0 2024-09-21T06:31:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d2600640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f43c400cbf0 tx=0x7f43c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4016020 con 0x7f43cc140da0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43c4012420 con 0x7f43cc140da0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4012870 con 0x7f43cc140da0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc152d10 con 0x7f43cc140da0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.742+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc1532d0 con 0x7f43cc140da0 2024-09-21T06:31:38.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.743+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc140ba0 con 0x7f43cc140da0 2024-09-21T06:31:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.745+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43c4019020 con 0x7f43cc140da0 2024-09-21T06:31:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.746+0000 7f43c9600640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 0x7f43ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.746+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f43c409b2b0 con 0x7f43cc140da0 2024-09-21T06:31:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.747+0000 7f43d1c00640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 0x7f43ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.748+0000 7f43d1c00640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 0x7f43ac07b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43c0035000 tx=0x7f43c0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.752+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f43c4020030 con 0x7f43cc140da0 2024-09-21T06:31:38.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.948+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43cc0ce920 con 0x7f43cc140da0 2024-09-21T06:31:38.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.949+0000 7f43c9600640 1 -- 172.21.15.37:0/1451883183 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17017 (secure 0 0 0) 0x7f43c4062e70 con 0x7f43cc140da0 2024-09-21T06:31:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:38.952 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":66,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:31:37.989032+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:31:35.929166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T06:31:36.946257+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 msgr2=0x7f43ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 0x7f43ac07b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43c0035000 tx=0x7f43c0004730 comp rx=0 tx=0).stop 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 msgr2=0x7f43cc0add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f43c400cbf0 tx=0x7f43c400a430 comp rx=0 tx=0).stop 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 shutdown_connections 2024-09-21T06:31:38.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f43ac078b90 0x7f43ac07b0a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43cc140da0 0x7f43cc0add90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43cc15ede0 0x7f43cc0ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 --2- 172.21.15.37:0/1451883183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43cc15b820 0x7f43cc0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 >> 172.21.15.37:0/1451883183 conn(0x7f43cc06f940 msgr2=0x7f43cc0cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 shutdown_connections 2024-09-21T06:31:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:38.952+0000 7f43d3000640 1 -- 172.21.15.37:0/1451883183 wait complete. 2024-09-21T06:31:38.977 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 -- 172.21.15.37:0/4273298290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 msgr2=0x7fb2f0145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 --2- 172.21.15.37:0/4273298290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f0145040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009860 tx=0x7fb2e402a5a0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 -- 172.21.15.37:0/4273298290 shutdown_connections 2024-09-21T06:31:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 --2- 172.21.15.37:0/4273298290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f0145040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 -- 172.21.15.37:0/4273298290 >> 172.21.15.37:0/4273298290 conn(0x7fb2f00d1020 msgr2=0x7fb2f00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 -- 172.21.15.37:0/4273298290 shutdown_connections 2024-09-21T06:31:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.203+0000 7fb2f6000640 1 -- 172.21.15.37:0/4273298290 wait complete. 2024-09-21T06:31:39.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.205+0000 7fb2f6000640 1 Processor -- start 2024-09-21T06:31:39.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.205+0000 7fb2f6000640 1 -- start start 2024-09-21T06:31:39.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f00ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 0x7fb2f00aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006dc90 con 0x7fb2f0145580 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006db10 con 0x7fb2f00ae920 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f6000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f006de10 con 0x7fb2f0160980 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34762/0 (socket says 172.21.15.37:34762) 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.207+0000 7fb2f4c00640 1 -- 172.21.15.37:0/3940399228 learned_addr learned my addr 172.21.15.37:0/3940399228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f5600640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 0x7fb2f00aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2efe00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f00ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 -- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 msgr2=0x7fb2f00ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f00ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 -- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 msgr2=0x7fb2f00aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 0x7fb2f00aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 -- 172.21.15.37:0/3940399228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f00af270 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f5600640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 0x7fb2f00aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2f4c00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009980 tx=0x7fb2e4033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.208+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4031b50 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.209+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f015fd10 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.209+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2e4031de0 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.209+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4033330 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.209+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f00b5580 con 0x7fb2f0145580 2024-09-21T06:31:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.210+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f0145040 con 0x7fb2f0145580 2024-09-21T06:31:39.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.212+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2e40334d0 con 0x7fb2f0145580 2024-09-21T06:31:39.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.213+0000 7fb2ecc00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.213+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fb2e40bc380 con 0x7fb2f0145580 2024-09-21T06:31:39.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.213+0000 7fb2efe00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.214+0000 7fb2efe00640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006ee60 tx=0x7fb2dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:39.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.216+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2e404a030 con 0x7fb2f0145580 2024-09-21T06:31:39.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.413+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2f006f450 con 0x7fb2f0145580 2024-09-21T06:31:39.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.414+0000 7fb2ecc00640 1 -- 172.21.15.37:0/3940399228 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17017 (secure 0 0 0) 0x7fb2e403f030 con 0x7fb2f0145580 2024-09-21T06:31:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:39.417 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":66,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:31:37.989032+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:31:35.929166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T06:31:36.946257+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.416+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 msgr2=0x7fb2d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.416+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006ee60 tx=0x7fb2dc009040 comp rx=0 tx=0).stop 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 msgr2=0x7fb2f01f4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009980 tx=0x7fb2e4033ba0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 shutdown_connections 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f00ae920 0x7fb2f00aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f0160980 0x7fb2f00ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 --2- 172.21.15.37:0/3940399228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0145580 0x7fb2f01f4860 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 >> 172.21.15.37:0/3940399228 conn(0x7fb2f00d1020 msgr2=0x7fb2f00b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 shutdown_connections 2024-09-21T06:31:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.417+0000 7fb2f6000640 1 -- 172.21.15.37:0/3940399228 wait complete. 2024-09-21T06:31:39.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 -- 172.21.15.37:0/1099787425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 msgr2=0x7ff1ec15ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 --2- 172.21.15.37:0/1099787425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec15ead0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009830 tx=0x7ff1e002a480 comp rx=0 tx=0).stop 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 -- 172.21.15.37:0/1099787425 shutdown_connections 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 --2- 172.21.15.37:0/1099787425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec162710 0x7ff1ec162b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 --2- 172.21.15.37:0/1099787425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec15ead0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 -- 172.21.15.37:0/1099787425 >> 172.21.15.37:0/1099787425 conn(0x7ff1ec0b46e0 msgr2=0x7ff1ec0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 -- 172.21.15.37:0/1099787425 shutdown_connections 2024-09-21T06:31:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.671+0000 7ff1f4000640 1 -- 172.21.15.37:0/1099787425 wait complete. 2024-09-21T06:31:39.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.674+0000 7ff1f4000640 1 Processor -- start 2024-09-21T06:31:39.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.674+0000 7ff1f4000640 1 -- start start 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec163060 0x7ff1ec1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0b7b90 con 0x7ff1ec162710 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0ce670 con 0x7ff1ec0b5bd0 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.675+0000 7ff1f4000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec0b7d10 con 0x7ff1ec163060 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34766/0 (socket says 172.21.15.37:34766) 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 -- 172.21.15.37:0/4211837833 learned_addr learned my addr 172.21.15.37:0/4211837833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f2000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec163060 0x7ff1ec1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f1600640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 -- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec163060 msgr2=0x7ff1ec1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec163060 0x7ff1ec1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 -- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 msgr2=0x7ff1ec159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 -- 172.21.15.37:0/4211837833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1f9cc0 con 0x7ff1ec162710 2024-09-21T06:31:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.676+0000 7ff1f0c00640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800e9f0 tx=0x7ff1d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.677+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800cd80 con 0x7ff1ec162710 2024-09-21T06:31:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.677+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1d8004020 con 0x7ff1ec162710 2024-09-21T06:31:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.677+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8004450 con 0x7ff1ec162710 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.677+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1f9420 con 0x7ff1ec162710 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.677+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1f99e0 con 0x7ff1ec162710 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.678+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec0b8230 con 0x7ff1ec162710 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.679+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1d80035f0 con 0x7ff1ec162710 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.680+0000 7ff1e9600640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.680+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff1d809b7b0 con 0x7ff1ec162710 2024-09-21T06:31:39.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.681+0000 7ff1f1600640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:39.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.681+0000 7ff1f1600640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034000 tx=0x7ff1e0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:39.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.684+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1d8063370 con 0x7ff1ec162710 2024-09-21T06:31:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:39.918+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff1ec0b55d0 con 0x7ff1ec162710 2024-09-21T06:31:40.433 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:31:40.431+0000 7f34ec400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:31:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.486+0000 7ff1e9600640 1 -- 172.21.15.37:0/4211837833 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) ==== 118+0+0 (secure 0 0 0) 0x7ff1d8067da0 con 0x7ff1ec162710 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.489+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 msgr2=0x7ff1cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.489+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034000 tx=0x7ff1e0004730 comp rx=0 tx=0).stop 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.489+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 msgr2=0x7ff1ec06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.489+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800e9f0 tx=0x7ff1d800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 shutdown_connections 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1ec162710 0x7ff1ec06f5c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1ec163060 0x7ff1ec1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 --2- 172.21.15.37:0/4211837833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec0b5bd0 0x7ff1ec159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 >> 172.21.15.37:0/4211837833 conn(0x7ff1ec0b46e0 msgr2=0x7ff1ec15d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 shutdown_connections 2024-09-21T06:31:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.490+0000 7ff1f4000640 1 -- 172.21.15.37:0/4211837833 wait complete. 2024-09-21T06:31:40.522 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:31:40.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:31:40.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 -- 172.21.15.37:0/3530356396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3874003fe0 msgr2=0x7f3874004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 --2- 172.21.15.37:0/3530356396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3874003fe0 0x7f3874004800 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3878009b10 tx=0x7f3874005830 comp rx=0 tx=0).stop 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3891e00640 1 -- 172.21.15.37:0/3530356396 reap_dead start 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 -- 172.21.15.37:0/3530356396 shutdown_connections 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 -- 172.21.15.37:0/3530356396 >> 172.21.15.37:0/3530356396 conn(0x7f388c09d830 msgr2=0x7f388c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 -- 172.21.15.37:0/3530356396 shutdown_connections 2024-09-21T06:31:40.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.756+0000 7f3893e00640 1 -- 172.21.15.37:0/3530356396 wait complete. 2024-09-21T06:31:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.758+0000 7f3893e00640 1 Processor -- start 2024-09-21T06:31:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.758+0000 7f3893e00640 1 -- start start 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874003fe0 0x7f388c1f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f388c06f410 0x7f388c14b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f388c0d0f70 con 0x7f388c1f5870 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f388c0d0df0 con 0x7f3874003fe0 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3893e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f388c0cf4f0 con 0x7f388c06f410 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3890a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3890a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34788/0 (socket says 172.21.15.37:34788) 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3890a00640 1 -- 172.21.15.37:0/86876517 learned_addr learned my addr 172.21.15.37:0/86876517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3891400640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874003fe0 0x7f388c1f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.760+0000 7f3891e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f388c06f410 0x7f388c14b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 -- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f388c06f410 msgr2=0x7f388c14b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f388c06f410 0x7f388c14b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 -- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874003fe0 msgr2=0x7f388c1f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874003fe0 0x7f388c1f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 -- 172.21.15.37:0/86876517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c06de30 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3890a00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3878009c30 tx=0x7f38780336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878038040 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3878041070 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.761+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c06e110 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.762+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878033c90 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.762+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c06e6d0 con 0x7f388c1f5870 2024-09-21T06:31:40.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.763+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c0ceb70 con 0x7f388c1f5870 2024-09-21T06:31:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.764+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3878046460 con 0x7f388c1f5870 2024-09-21T06:31:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.765+0000 7f3889600640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 0x7f386c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.765+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f38780bb7d0 con 0x7f388c1f5870 2024-09-21T06:31:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.765+0000 7f3891400640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 0x7f386c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.766+0000 7f3891400640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 0x7f386c07b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3880000c00 tx=0x7f3880006060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:40.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:40.769+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3878083460 con 0x7f388c1f5870 2024-09-21T06:31:41.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.003+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f388c158ce0 con 0x7f388c1f5870 2024-09-21T06:31:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.493+0000 7f3889600640 1 -- 172.21.15.37:0/86876517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f3878087e90 con 0x7f388c1f5870 2024-09-21T06:31:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 msgr2=0x7f386c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 0x7f386c07b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3880000c00 tx=0x7f3880006060 comp rx=0 tx=0).stop 2024-09-21T06:31:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 msgr2=0x7f388c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3878009c30 tx=0x7f38780336f0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 shutdown_connections 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f386c078b90 0x7f386c07b0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f388c1f5870 0x7f388c06eed0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f388c06f410 0x7f388c14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 --2- 172.21.15.37:0/86876517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874003fe0 0x7f388c1f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 >> 172.21.15.37:0/86876517 conn(0x7f388c09d830 msgr2=0x7f388c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 shutdown_connections 2024-09-21T06:31:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.496+0000 7f3893e00640 1 -- 172.21.15.37:0/86876517 wait complete. 2024-09-21T06:31:41.521 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:31:41.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.746+0000 7efdf9600640 1 -- 172.21.15.37:0/867343143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd8003a00 msgr2=0x7efdd8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.746+0000 7efdf9600640 1 --2- 172.21.15.37:0/867343143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd8003a00 0x7efdd8003e70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efdf415a730 tx=0x7efdd8005830 comp rx=0 tx=0).stop 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.746+0000 7efdf3400640 1 -- 172.21.15.37:0/867343143 reap_dead start 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.746+0000 7efdf9600640 1 -- 172.21.15.37:0/867343143 shutdown_connections 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.746+0000 7efdf9600640 1 -- 172.21.15.37:0/867343143 >> 172.21.15.37:0/867343143 conn(0x7efdf40cca40 msgr2=0x7efdf40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.747+0000 7efdf9600640 1 -- 172.21.15.37:0/867343143 shutdown_connections 2024-09-21T06:31:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.747+0000 7efdf9600640 1 -- 172.21.15.37:0/867343143 wait complete. 2024-09-21T06:31:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.749+0000 7efdf9600640 1 Processor -- start 2024-09-21T06:31:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.749+0000 7efdf9600640 1 -- start start 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdd8003a00 0x7efdf415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf40ce0d0 0x7efdf4157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efdf406ef60 con 0x7efdf41f5040 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efdf406ede0 con 0x7efdf40ce0d0 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.750+0000 7efdf9600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efdf406f0e0 con 0x7efdd8003a00 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34810/0 (socket says 172.21.15.37:34810) 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 -- 172.21.15.37:0/2667477846 learned_addr learned my addr 172.21.15.37:0/2667477846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2a00640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdd8003a00 0x7efdf415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf3400640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf40ce0d0 0x7efdf4157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 -- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdd8003a00 msgr2=0x7efdf415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdd8003a00 0x7efdf415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 -- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf40ce0d0 msgr2=0x7efdf4157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf40ce0d0 0x7efdf4157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.751+0000 7efdf2000640 1 -- 172.21.15.37:0/2667477846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf40ce4e0 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdf2000640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efde0011b10 tx=0x7efde0009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde001e070 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde000fe40 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00042b0 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf41f9320 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.752+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41f98e0 con 0x7efdf41f5040 2024-09-21T06:31:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.753+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf41f9090 con 0x7efdf41f5040 2024-09-21T06:31:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.755+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efde0021070 con 0x7efdf41f5040 2024-09-21T06:31:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.755+0000 7efdeaa00640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 0x7efdd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.756+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7efde009b8c0 con 0x7efdf41f5040 2024-09-21T06:31:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.756+0000 7efdf2a00640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 0x7efdd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.757+0000 7efdf2a00640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 0x7efdd407b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efdf415a730 tx=0x7efde4038040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.760+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efdf40ce4e0 con 0x7efdf41f5040 2024-09-21T06:31:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.996+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdf406ddf0 con 0x7efdf41f5040 2024-09-21T06:31:41.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.997+0000 7efdeaa00640 1 -- 172.21.15.37:0/2667477846 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5230 (secure 0 0 0) 0x7efde0063480 con 0x7efdf41f5040 2024-09-21T06:31:41.999 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:41.999 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"btime":"2024-09-21T06:31:41:488546+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:31:41.005513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":4}]} 2024-09-21T06:31:41.999 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-09-21T06:31:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 msgr2=0x7efdd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 0x7efdd407b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efdf415a730 tx=0x7efde4038040 comp rx=0 tx=0).stop 2024-09-21T06:31:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 msgr2=0x7efdf40cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efde0011b10 tx=0x7efde0009e00 comp rx=0 tx=0).stop 2024-09-21T06:31:42.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 shutdown_connections 2024-09-21T06:31:42.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efdd4078b90 0x7efdd407b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf41f5040 0x7efdf40cdb90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf40ce0d0 0x7efdf4157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 --2- 172.21.15.37:0/2667477846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdd8003a00 0x7efdf415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:41.999+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 >> 172.21.15.37:0/2667477846 conn(0x7efdf40cca40 msgr2=0x7efdf40ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.000+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 shutdown_connections 2024-09-21T06:31:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.000+0000 7efdf9600640 1 -- 172.21.15.37:0/2667477846 wait complete. 2024-09-21T06:31:42.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 -- 172.21.15.37:0/1513035024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb8073f20 msgr2=0x7f8eb815b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 --2- 172.21.15.37:0/1513035024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb8073f20 0x7f8eb815b2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009830 tx=0x7f8ea802a410 comp rx=0 tx=0).stop 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 -- 172.21.15.37:0/1513035024 shutdown_connections 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 --2- 172.21.15.37:0/1513035024 >> v2:172.21.15.150:3300/0 conn(0x7f8eb80cb0c0 0x7f8eb80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 --2- 172.21.15.37:0/1513035024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb8073f20 0x7f8eb815b2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.253+0000 7f8eb7200640 1 -- 172.21.15.37:0/1513035024 >> 172.21.15.37:0/1513035024 conn(0x7f8eb8073400 msgr2=0x7f8eb8073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.254+0000 7f8eb7200640 1 -- 172.21.15.37:0/1513035024 shutdown_connections 2024-09-21T06:31:42.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.254+0000 7f8eb7200640 1 -- 172.21.15.37:0/1513035024 wait complete. 2024-09-21T06:31:42.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.256+0000 7f8eb7200640 1 Processor -- start 2024-09-21T06:31:42.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.256+0000 7f8eb7200640 1 -- start start 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb806ddf0 0x7f8eb80ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb80cb0c0 0x7f8eb80b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80cbb90 con 0x7f8eb8073f20 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80cba10 con 0x7f8eb806ddf0 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.257+0000 7f8eb7200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb80cbd10 con 0x7f8eb80cb0c0 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb806ddf0 0x7f8eb80ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb80cb0c0 0x7f8eb80b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34818/0 (socket says 172.21.15.37:34818) 2024-09-21T06:31:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 -- 172.21.15.37:0/624228803 learned_addr learned my addr 172.21.15.37:0/624228803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 -- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb80cb0c0 msgr2=0x7f8eb80b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb80cb0c0 0x7f8eb80b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 -- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb806ddf0 msgr2=0x7f8eb80ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb806ddf0 0x7f8eb80ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.258+0000 7f8eb5400640 1 -- 172.21.15.37:0/624228803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb80c6ea0 con 0x7f8eb8073f20 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eb5400640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c500 tx=0x7f8ea400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4016020 con 0x7f8eb8073f20 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ea4012940 con 0x7f8eb8073f20 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb814d4c0 con 0x7f8eb8073f20 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4012d90 con 0x7f8eb8073f20 2024-09-21T06:31:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.259+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb80b0930 con 0x7f8eb8073f20 2024-09-21T06:31:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.260+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb80738c0 con 0x7f8eb8073f20 2024-09-21T06:31:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.261+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ea4019020 con 0x7f8eb8073f20 2024-09-21T06:31:42.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.263+0000 7f8eae000640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 0x7f8e9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.263+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8ea409af00 con 0x7f8eb8073f20 2024-09-21T06:31:42.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.263+0000 7f8eb5e00640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 0x7f8e9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.264+0000 7f8eb5e00640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 0x7f8e9807b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034000 tx=0x7f8ea8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:42.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.268+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ea4020020 con 0x7f8eb8073f20 2024-09-21T06:31:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.466+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8eb80b0440 con 0x7f8eb8073f20 2024-09-21T06:31:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.467+0000 7f8eae000640 1 -- 172.21.15.37:0/624228803 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17004 (secure 0 0 0) 0x7f8ea4062ad0 con 0x7f8eb8073f20 2024-09-21T06:31:42.470 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:42.470 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":67,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:31:39.469665+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:31:35.929166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T06:31:36.946257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:31:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 msgr2=0x7f8e9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 0x7f8e9807b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034000 tx=0x7f8ea8004730 comp rx=0 tx=0).stop 2024-09-21T06:31:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 msgr2=0x7f8eb814cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c500 tx=0x7f8ea400c9c0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 shutdown_connections 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e98078b90 0x7f8e9807b0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8073f20 0x7f8eb814cf80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb80cb0c0 0x7f8eb80b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 --2- 172.21.15.37:0/624228803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb806ddf0 0x7f8eb80ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 >> 172.21.15.37:0/624228803 conn(0x7f8eb8073400 msgr2=0x7f8eb80ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 shutdown_connections 2024-09-21T06:31:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.470+0000 7f8eb7200640 1 -- 172.21.15.37:0/624228803 wait complete. 2024-09-21T06:31:42.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:31:42.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 -- 172.21.15.37:0/3599319232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4004800 msgr2=0x7ff8b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 --2- 172.21.15.37:0/3599319232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4004800 0x7ff8b4004c10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800c310 tx=0x7ff8b4005830 comp rx=0 tx=0).stop 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 -- 172.21.15.37:0/3599319232 shutdown_connections 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 --2- 172.21.15.37:0/3599319232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4004800 0x7ff8b4004c10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 -- 172.21.15.37:0/3599319232 >> 172.21.15.37:0/3599319232 conn(0x7ff8d00cca40 msgr2=0x7ff8d00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 -- 172.21.15.37:0/3599319232 shutdown_connections 2024-09-21T06:31:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.721+0000 7ff8d5400640 1 -- 172.21.15.37:0/3599319232 wait complete. 2024-09-21T06:31:42.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.724+0000 7ff8d5400640 1 Processor -- start 2024-09-21T06:31:42.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.724+0000 7ff8d5400640 1 -- start start 2024-09-21T06:31:42.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b4004800 0x7ff8d01f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8d015c680 0x7ff8d01f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d015e7c0 con 0x7ff8d015be80 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d00cb960 con 0x7ff8b4004800 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8d5400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d015e940 con 0x7ff8d015c680 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8cf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8cf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34840/0 (socket says 172.21.15.37:34840) 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.725+0000 7ff8cf200640 1 -- 172.21.15.37:0/1479406425 learned_addr learned my addr 172.21.15.37:0/1479406425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8ce800640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b4004800 0x7ff8d01f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8c5e00640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8d015c680 0x7ff8d01f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 -- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8d015c680 msgr2=0x7ff8d01f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8d015c680 0x7ff8d01f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 -- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b4004800 msgr2=0x7ff8d01f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b4004800 0x7ff8d01f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 -- 172.21.15.37:0/1479406425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d01f8720 con 0x7ff8d015be80 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.726+0000 7ff8cf200640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0072d20 tx=0x7ff8c8010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.727+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800ee70 con 0x7ff8d015be80 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.727+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8c8015070 con 0x7ff8d015be80 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.727+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8003d00 con 0x7ff8d015be80 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.727+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d01f8a00 con 0x7ff8d015be80 2024-09-21T06:31:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.727+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d01f49f0 con 0x7ff8d015be80 2024-09-21T06:31:42.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.728+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d006ddf0 con 0x7ff8d015be80 2024-09-21T06:31:42.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.729+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8c8004030 con 0x7ff8d015be80 2024-09-21T06:31:42.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.730+0000 7ff8c7400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 0x7ff8ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:42.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.730+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff8c809ba10 con 0x7ff8d015be80 2024-09-21T06:31:42.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.731+0000 7ff8ce800640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 0x7ff8ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.732+0000 7ff8ce800640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 0x7ff8ac07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00036b0 tx=0x7ff8c0006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:42.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.735+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff8c8063660 con 0x7ff8d015be80 2024-09-21T06:31:42.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.932+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8d01f84b0 con 0x7ff8ac078b90 2024-09-21T06:31:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.938+0000 7ff8c7400640 1 -- 172.21.15.37:0/1479406425 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247250 (secure 0 0 0) 0x7ff8d01f84b0 con 0x7ff8ac078b90 2024-09-21T06:31:42.943 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:42.944 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:31:42.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 msgr2=0x7ff8ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 0x7ff8ac07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00036b0 tx=0x7ff8c0006b70 comp rx=0 tx=0).stop 2024-09-21T06:31:42.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 msgr2=0x7ff8d009a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:42.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0072d20 tx=0x7ff8c8010e90 comp rx=0 tx=0).stop 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 shutdown_connections 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff8ac078b90 0x7ff8ac07b0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d015be80 0x7ff8d009a9e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8d015c680 0x7ff8d01f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 --2- 172.21.15.37:0/1479406425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8b4004800 0x7ff8d01f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 >> 172.21.15.37:0/1479406425 conn(0x7ff8d00cca40 msgr2=0x7ff8d006f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 shutdown_connections 2024-09-21T06:31:42.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:42.942+0000 7ff8d5400640 1 -- 172.21.15.37:0/1479406425 wait complete. 2024-09-21T06:31:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-21T06:31:41.147535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240136,"kb_used_data":7236,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734584,"statfs":{"total":773094113280,"available":772848214016,"internally_reserved":0,"allocated":7409664,"data_stored":3957642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.697469"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.471166+0000","last_change":"2024-09-21T06:31:37.471166+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.471166+0000","last_undegraded":"2024-09-21T06:31:37.471166+0000","last_fullsized":"2024-09-21T06:31:37.471166+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484546+0000","last_change":"2024-09-21T06:31:38.481832+0000","last_active":"2024-09-21T06:31:39.484546+0000","last_peered":"2024-09-21T06:31:39.484546+0000","last_clean":"2024-09-21T06:31:39.484546+0000","last_became_active":"2024-09-21T06:31:38.481202+0000","last_became_peered":"2024-09-21T06:31:38.481202+0000","last_unstale":"2024-09-21T06:31:39.484546+0000","last_undegraded":"2024-09-21T06:31:39.484546+0000","last_fullsized":"2024-09-21T06:31:39.484546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:19.061826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465920+0000","last_change":"2024-09-21T06:31:37.465920+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465920+0000","last_undegraded":"2024-09-21T06:31:37.465920+0000","last_fullsized":"2024-09-21T06:31:37.465920+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.474460+0000","last_change":"2024-09-21T06:31:37.474460+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.474460+0000","last_undegraded":"2024-09-21T06:31:37.474460+0000","last_fullsized":"2024-09-21T06:31:37.474460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.464980+0000","last_change":"2024-09-21T06:31:37.464980+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.464980+0000","last_undegraded":"2024-09-21T06:31:37.464980+0000","last_fullsized":"2024-09-21T06:31:37.464980+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484571+0000","last_change":"2024-09-21T06:31:38.482703+0000","last_active":"2024-09-21T06:31:39.484571+0000","last_peered":"2024-09-21T06:31:39.484571+0000","last_clean":"2024-09-21T06:31:39.484571+0000","last_became_active":"2024-09-21T06:31:38.481272+0000","last_became_peered":"2024-09-21T06:31:38.481272+0000","last_unstale":"2024-09-21T06:31:39.484571+0000","last_undegraded":"2024-09-21T06:31:39.484571+0000","last_fullsized":"2024-09-21T06:31:39.484571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:45.840338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.472982+0000","last_change":"2024-09-21T06:31:37.472982+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.472982+0000","last_undegraded":"2024-09-21T06:31:37.472982+0000","last_fullsized":"2024-09-21T06:31:37.472982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466805+0000","last_change":"2024-09-21T06:31:37.466805+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466805+0000","last_undegraded":"2024-09-21T06:31:37.466805+0000","last_fullsized":"2024-09-21T06:31:37.466805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.472171+0000","last_change":"2024-09-21T06:31:37.472171+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.472171+0000","last_undegraded":"2024-09-21T06:31:37.472171+0000","last_fullsized":"2024-09-21T06:31:37.472171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468602+0000","last_change":"2024-09-21T06:31:37.468602+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468602+0000","last_undegraded":"2024-09-21T06:31:37.468602+0000","last_fullsized":"2024-09-21T06:31:37.468602+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.469453+0000","last_change":"2024-09-21T06:31:37.469453+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.469453+0000","last_undegraded":"2024-09-21T06:31:37.469453+0000","last_fullsized":"2024-09-21T06:31:37.469453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.473938+0000","last_change":"2024-09-21T06:31:37.473938+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.473938+0000","last_undegraded":"2024-09-21T06:31:37.473938+0000","last_fullsized":"2024-09-21T06:31:37.473938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.475000+0000","last_change":"2024-09-21T06:31:37.475000+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.475000+0000","last_undegraded":"2024-09-21T06:31:37.475000+0000","last_fullsized":"2024-09-21T06:31:37.475000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.475885+0000","last_change":"2024-09-21T06:31:37.475885+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.475885+0000","last_undegraded":"2024-09-21T06:31:37.475885+0000","last_fullsized":"2024-09-21T06:31:37.475885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467920+0000","last_change":"2024-09-21T06:31:37.467920+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467920+0000","last_undegraded":"2024-09-21T06:31:37.467920+0000","last_fullsized":"2024-09-21T06:31:37.467920+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.470951+0000","last_change":"2024-09-21T06:31:37.470951+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.470951+0000","last_undegraded":"2024-09-21T06:31:37.470951+0000","last_fullsized":"2024-09-21T06:31:37.470951+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.462024+0000","last_change":"2024-09-21T06:31:37.462024+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.462024+0000","last_undegraded":"2024-09-21T06:31:37.462024+0000","last_fullsized":"2024-09-21T06:31:37.462024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.472145+0000","last_change":"2024-09-21T06:31:37.472145+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.472145+0000","last_undegraded":"2024-09-21T06:31:37.472145+0000","last_fullsized":"2024-09-21T06:31:37.472145+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468899+0000","last_change":"2024-09-21T06:31:37.468899+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468899+0000","last_undegraded":"2024-09-21T06:31:37.468899+0000","last_fullsized":"2024-09-21T06:31:37.468899+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.463127+0000","last_change":"2024-09-21T06:31:37.463127+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.463127+0000","last_undegraded":"2024-09-21T06:31:37.463127+0000","last_fullsized":"2024-09-21T06:31:37.463127+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.474154+0000","last_change":"2024-09-21T06:31:37.474154+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.474154+0000","last_undegraded":"2024-09-21T06:31:37.474154+0000","last_fullsized":"2024-09-21T06:31:37.474154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484776+0000","last_change":"2024-09-21T06:31:38.481328+0000","last_active":"2024-09-21T06:31:39.484776+0000","last_peered":"2024-09-21T06:31:39.484776+0000","last_clean":"2024-09-21T06:31:39.484776+0000","last_became_active":"2024-09-21T06:31:38.480298+0000","last_became_peered":"2024-09-21T06:31:38.480298+0000","last_unstale":"2024-09-21T06:31:39.484776+0000","last_undegraded":"2024-09-21T06:31:39.484776+0000","last_fullsized":"2024-09-21T06:31:39.484776+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:34.869541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.469584+0000","last_change":"2024-09-21T06:31:37.469584+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.469584+0000","last_undegraded":"2024-09-21T06:31:37.469584+0000","last_fullsized":"2024-09-21T06:31:37.469584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466182+0000","last_change":"2024-09-21T06:31:37.466182+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466182+0000","last_undegraded":"2024-09-21T06:31:37.466182+0000","last_fullsized":"2024-09-21T06:31:37.466182+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.476094+0000","last_change":"2024-09-21T06:31:37.476094+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.476094+0000","last_undegraded":"2024-09-21T06:31:37.476094+0000","last_fullsized":"2024-09-21T06:31:37.476094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465139+0000","last_change":"2024-09-21T06:31:37.465139+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465139+0000","last_undegraded":"2024-09-21T06:31:37.465139+0000","last_fullsized":"2024-09-21T06:31:37.465139+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.474867+0000","last_change":"2024-09-21T06:31:37.474867+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.474867+0000","last_undegraded":"2024-09-21T06:31:37.474867+0000","last_fullsized":"2024-09-21T06:31:37.474867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.473443+0000","last_change":"2024-09-21T06:31:37.473443+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.473443+0000","last_undegraded":"2024-09-21T06:31:37.473443+0000","last_fullsized":"2024-09-21T06:31:37.473443+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467904+0000","last_change":"2024-09-21T06:31:37.467904+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467904+0000","last_undegraded":"2024-09-21T06:31:37.467904+0000","last_fullsized":"2024-09-21T06:31:37.467904+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466250+0000","last_change":"2024-09-21T06:31:37.466250+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466250+0000","last_undegraded":"2024-09-21T06:31:37.466250+0000","last_fullsized":"2024-09-21T06:31:37.466250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466935+0000","last_change":"2024-09-21T06:31:37.466935+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466935+0000","last_undegraded":"2024-09-21T06:31:37.466935+0000","last_fullsized":"2024-09-21T06:31:37.466935+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467526+0000","last_change":"2024-09-21T06:31:37.467526+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467526+0000","last_undegraded":"2024-09-21T06:31:37.467526+0000","last_fullsized":"2024-09-21T06:31:37.467526+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.472512+0000","last_change":"2024-09-21T06:31:37.472512+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.472512+0000","last_undegraded":"2024-09-21T06:31:37.472512+0000","last_fullsized":"2024-09-21T06:31:37.472512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.489961+0000","last_change":"2024-09-21T06:31:37.484380+0000","last_active":"2024-09-21T06:31:37.489961+0000","last_peered":"2024-09-21T06:31:37.489961+0000","last_clean":"2024-09-21T06:31:37.489961+0000","last_became_active":"2024-09-21T06:31:37.484046+0000","last_became_peered":"2024-09-21T06:31:37.484046+0000","last_unstale":"2024-09-21T06:31:37.489961+0000","last_undegraded":"2024-09-21T06:31:37.489961+0000","last_fullsized":"2024-09-21T06:31:37.489961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:11.592873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465165+0000","last_change":"2024-09-21T06:31:37.465165+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465165+0000","last_undegraded":"2024-09-21T06:31:37.465165+0000","last_fullsized":"2024-09-21T06:31:37.465165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.488949+0000","last_change":"2024-09-21T06:31:37.482254+0000","last_active":"2024-09-21T06:31:37.488949+0000","last_peered":"2024-09-21T06:31:37.488949+0000","last_clean":"2024-09-21T06:31:37.488949+0000","last_became_active":"2024-09-21T06:31:37.481797+0000","last_became_peered":"2024-09-21T06:31:37.481797+0000","last_unstale":"2024-09-21T06:31:37.488949+0000","last_undegraded":"2024-09-21T06:31:37.488949+0000","last_fullsized":"2024-09-21T06:31:37.488949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:35.393119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484906+0000","last_change":"2024-09-21T06:31:38.481536+0000","last_active":"2024-09-21T06:31:39.484906+0000","last_peered":"2024-09-21T06:31:39.484906+0000","last_clean":"2024-09-21T06:31:39.484906+0000","last_became_active":"2024-09-21T06:31:38.480779+0000","last_became_peered":"2024-09-21T06:31:38.480779+0000","last_unstale":"2024-09-21T06:31:39.484906+0000","last_undegraded":"2024-09-21T06:31:39.484906+0000","last_fullsized":"2024-09-21T06:31:39.484906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:10.890474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.489669+0000","last_change":"2024-09-21T06:31:37.483834+0000","last_active":"2024-09-21T06:31:37.489669+0000","last_peered":"2024-09-21T06:31:37.489669+0000","last_clean":"2024-09-21T06:31:37.489669+0000","last_became_active":"2024-09-21T06:31:37.483475+0000","last_became_peered":"2024-09-21T06:31:37.483475+0000","last_unstale":"2024-09-21T06:31:37.489669+0000","last_undegraded":"2024-09-21T06:31:37.489669+0000","last_fullsized":"2024-09-21T06:31:37.489669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:13.289768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.471577+0000","last_change":"2024-09-21T06:31:37.471577+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.471577+0000","last_undegraded":"2024-09-21T06:31:37.471577+0000","last_fullsized":"2024-09-21T06:31:37.471577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484473+0000","last_change":"2024-09-21T06:31:37.481111+0000","last_active":"2024-09-21T06:31:39.484473+0000","last_peered":"2024-09-21T06:31:39.484473+0000","last_clean":"2024-09-21T06:31:39.484473+0000","last_became_active":"2024-09-21T06:31:37.480723+0000","last_became_peered":"2024-09-21T06:31:37.480723+0000","last_unstale":"2024-09-21T06:31:39.484473+0000","last_undegraded":"2024-09-21T06:31:39.484473+0000","last_fullsized":"2024-09-21T06:31:39.484473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:52.252258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.463652+0000","last_change":"2024-09-21T06:31:37.463652+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.463652+0000","last_undegraded":"2024-09-21T06:31:37.463652+0000","last_fullsized":"2024-09-21T06:31:37.463652+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.478931+0000","last_change":"2024-09-21T06:31:37.474902+0000","last_active":"2024-09-21T06:31:37.478931+0000","last_peered":"2024-09-21T06:31:37.478931+0000","last_clean":"2024-09-21T06:31:37.478931+0000","last_became_active":"2024-09-21T06:31:37.473795+0000","last_became_peered":"2024-09-21T06:31:37.473795+0000","last_unstale":"2024-09-21T06:31:37.478931+0000","last_undegraded":"2024-09-21T06:31:37.478931+0000","last_fullsized":"2024-09-21T06:31:37.478931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:20.029778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468075+0000","last_change":"2024-09-21T06:31:37.468075+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468075+0000","last_undegraded":"2024-09-21T06:31:37.468075+0000","last_fullsized":"2024-09-21T06:31:37.468075+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484852+0000","last_change":"2024-09-21T06:31:37.476147+0000","last_active":"2024-09-21T06:31:39.484852+0000","last_peered":"2024-09-21T06:31:39.484852+0000","last_clean":"2024-09-21T06:31:39.484852+0000","last_became_active":"2024-09-21T06:31:37.475721+0000","last_became_peered":"2024-09-21T06:31:37.475721+0000","last_unstale":"2024-09-21T06:31:39.484852+0000","last_undegraded":"2024-09-21T06:31:39.484852+0000","last_fullsized":"2024-09-21T06:31:39.484852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:10.564512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467387+0000","last_change":"2024-09-21T06:31:37.467387+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467387+0000","last_undegraded":"2024-09-21T06:31:37.467387+0000","last_fullsized":"2024-09-21T06:31:37.467387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.490251+0000","last_change":"2024-09-21T06:31:37.487702+0000","last_active":"2024-09-21T06:31:37.490251+0000","last_peered":"2024-09-21T06:31:37.490251+0000","last_clean":"2024-09-21T06:31:37.490251+0000","last_became_active":"2024-09-21T06:31:37.487201+0000","last_became_peered":"2024-09-21T06:31:37.487201+0000","last_unstale":"2024-09-21T06:31:37.490251+0000","last_undegraded":"2024-09-21T06:31:37.490251+0000","last_fullsized":"2024-09-21T06:31:37.490251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:34.680067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465237+0000","last_change":"2024-09-21T06:31:37.465237+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465237+0000","last_undegraded":"2024-09-21T06:31:37.465237+0000","last_fullsized":"2024-09-21T06:31:37.465237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.487337+0000","last_change":"2024-09-21T06:31:37.485399+0000","last_active":"2024-09-21T06:31:37.487337+0000","last_peered":"2024-09-21T06:31:37.487337+0000","last_clean":"2024-09-21T06:31:37.487337+0000","last_became_active":"2024-09-21T06:31:37.485049+0000","last_became_peered":"2024-09-21T06:31:37.485049+0000","last_unstale":"2024-09-21T06:31:37.487337+0000","last_undegraded":"2024-09-21T06:31:37.487337+0000","last_fullsized":"2024-09-21T06:31:37.487337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:53.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.469689+0000","last_change":"2024-09-21T06:31:37.469689+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.469689+0000","last_undegraded":"2024-09-21T06:31:37.469689+0000","last_fullsized":"2024-09-21T06:31:37.469689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.483244+0000","last_change":"2024-09-21T06:31:37.480126+0000","last_active":"2024-09-21T06:31:37.483244+0000","last_peered":"2024-09-21T06:31:37.483244+0000","last_clean":"2024-09-21T06:31:37.483244+0000","last_became_active":"2024-09-21T06:31:37.479811+0000","last_became_peered":"2024-09-21T06:31:37.479811+0000","last_unstale":"2024-09-21T06:31:37.483244+0000","last_undegraded":"2024-09-21T06:31:37.483244+0000","last_fullsized":"2024-09-21T06:31:37.483244+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:52.076981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465742+0000","last_change":"2024-09-21T06:31:37.465742+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465742+0000","last_undegraded":"2024-09-21T06:31:37.465742+0000","last_fullsized":"2024-09-21T06:31:37.465742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.479932+0000","last_change":"2024-09-21T06:31:37.473471+0000","last_active":"2024-09-21T06:31:37.479932+0000","last_peered":"2024-09-21T06:31:37.479932+0000","last_clean":"2024-09-21T06:31:37.479932+0000","last_became_active":"2024-09-21T06:31:37.473163+0000","last_became_peered":"2024-09-21T06:31:37.473163+0000","last_unstale":"2024-09-21T06:31:37.479932+0000","last_undegraded":"2024-09-21T06:31:37.479932+0000","last_fullsized":"2024-09-21T06:31:37.479932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:00.822797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.469133+0000","last_change":"2024-09-21T06:31:37.469133+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.469133+0000","last_undegraded":"2024-09-21T06:31:37.469133+0000","last_fullsized":"2024-09-21T06:31:37.469133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.485971+0000","last_change":"2024-09-21T06:31:37.479615+0000","last_active":"2024-09-21T06:31:37.485971+0000","last_peered":"2024-09-21T06:31:37.485971+0000","last_clean":"2024-09-21T06:31:37.485971+0000","last_became_active":"2024-09-21T06:31:37.479274+0000","last_became_peered":"2024-09-21T06:31:37.479274+0000","last_unstale":"2024-09-21T06:31:37.485971+0000","last_undegraded":"2024-09-21T06:31:37.485971+0000","last_fullsized":"2024-09-21T06:31:37.485971+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:14.418097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468581+0000","last_change":"2024-09-21T06:31:37.468581+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468581+0000","last_undegraded":"2024-09-21T06:31:37.468581+0000","last_fullsized":"2024-09-21T06:31:37.468581+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":21,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484586+0000","last_change":"2024-09-21T06:31:37.485790+0000","last_active":"2024-09-21T06:31:39.484586+0000","last_peered":"2024-09-21T06:31:39.484586+0000","last_clean":"2024-09-21T06:31:39.484586+0000","last_became_active":"2024-09-21T06:31:37.485458+0000","last_became_peered":"2024-09-21T06:31:37.485458+0000","last_unstale":"2024-09-21T06:31:39.484586+0000","last_undegraded":"2024-09-21T06:31:39.484586+0000","last_fullsized":"2024-09-21T06:31:39.484586+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:57.211471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.464487+0000","last_change":"2024-09-21T06:31:37.464487+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.464487+0000","last_undegraded":"2024-09-21T06:31:37.464487+0000","last_fullsized":"2024-09-21T06:31:37.464487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.483709+0000","last_change":"2024-09-21T06:31:37.481783+0000","last_active":"2024-09-21T06:31:37.483709+0000","last_peered":"2024-09-21T06:31:37.483709+0000","last_clean":"2024-09-21T06:31:37.483709+0000","last_became_active":"2024-09-21T06:31:37.480964+0000","last_became_peered":"2024-09-21T06:31:37.480964+0000","last_unstale":"2024-09-21T06:31:37.483709+0000","last_undegraded":"2024-09-21T06:31:37.483709+0000","last_fullsized":"2024-09-21T06:31:37.483709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:52.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467377+0000","last_change":"2024-09-21T06:31:37.467377+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467377+0000","last_undegraded":"2024-09-21T06:31:37.467377+0000","last_fullsized":"2024-09-21T06:31:37.467377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.485870+0000","last_change":"2024-09-21T06:31:37.480933+0000","last_active":"2024-09-21T06:31:37.485870+0000","last_peered":"2024-09-21T06:31:37.485870+0000","last_clean":"2024-09-21T06:31:37.485870+0000","last_became_active":"2024-09-21T06:31:37.480357+0000","last_became_peered":"2024-09-21T06:31:37.480357+0000","last_unstale":"2024-09-21T06:31:37.485870+0000","last_undegraded":"2024-09-21T06:31:37.485870+0000","last_fullsized":"2024-09-21T06:31:37.485870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:52.076981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.470520+0000","last_change":"2024-09-21T06:31:37.470520+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.470520+0000","last_undegraded":"2024-09-21T06:31:37.470520+0000","last_fullsized":"2024-09-21T06:31:37.470520+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.485089+0000","last_change":"2024-09-21T06:31:37.477963+0000","last_active":"2024-09-21T06:31:37.485089+0000","last_peered":"2024-09-21T06:31:37.485089+0000","last_clean":"2024-09-21T06:31:37.485089+0000","last_became_active":"2024-09-21T06:31:37.477279+0000","last_became_peered":"2024-09-21T06:31:37.477279+0000","last_unstale":"2024-09-21T06:31:37.485089+0000","last_undegraded":"2024-09-21T06:31:37.485089+0000","last_fullsized":"2024-09-21T06:31:37.485089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:44.847529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466981+0000","last_change":"2024-09-21T06:31:37.466981+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466981+0000","last_undegraded":"2024-09-21T06:31:37.466981+0000","last_fullsized":"2024-09-21T06:31:37.466981+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.489327+0000","last_change":"2024-09-21T06:31:37.484364+0000","last_active":"2024-09-21T06:31:37.489327+0000","last_peered":"2024-09-21T06:31:37.489327+0000","last_clean":"2024-09-21T06:31:37.489327+0000","last_became_active":"2024-09-21T06:31:37.483142+0000","last_became_peered":"2024-09-21T06:31:37.483142+0000","last_unstale":"2024-09-21T06:31:37.489327+0000","last_undegraded":"2024-09-21T06:31:37.489327+0000","last_fullsized":"2024-09-21T06:31:37.489327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:54.749049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.470149+0000","last_change":"2024-09-21T06:31:37.470149+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.470149+0000","last_undegraded":"2024-09-21T06:31:37.470149+0000","last_fullsized":"2024-09-21T06:31:37.470149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.483273+0000","last_change":"2024-09-21T06:31:37.480329+0000","last_active":"2024-09-21T06:31:37.483273+0000","last_peered":"2024-09-21T06:31:37.483273+0000","last_clean":"2024-09-21T06:31:37.483273+0000","last_became_active":"2024-09-21T06:31:37.479978+0000","last_became_peered":"2024-09-21T06:31:37.479978+0000","last_unstale":"2024-09-21T06:31:37.483273+0000","last_undegraded":"2024-09-21T06:31:37.483273+0000","last_fullsized":"2024-09-21T06:31:37.483273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:54.749049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468309+0000","last_change":"2024-09-21T06:31:37.468309+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468309+0000","last_undegraded":"2024-09-21T06:31:37.468309+0000","last_fullsized":"2024-09-21T06:31:37.468309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.484707+0000","last_change":"2024-09-21T06:31:37.483409+0000","last_active":"2024-09-21T06:31:37.484707+0000","last_peered":"2024-09-21T06:31:37.484707+0000","last_clean":"2024-09-21T06:31:37.484707+0000","last_became_active":"2024-09-21T06:31:37.482937+0000","last_became_peered":"2024-09-21T06:31:37.482937+0000","last_unstale":"2024-09-21T06:31:37.484707+0000","last_undegraded":"2024-09-21T06:31:37.484707+0000","last_fullsized":"2024-09-21T06:31:37.484707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:04.005866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.468247+0000","last_change":"2024-09-21T06:31:37.468247+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.468247+0000","last_undegraded":"2024-09-21T06:31:37.468247+0000","last_fullsized":"2024-09-21T06:31:37.468247+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.491325+0000","last_change":"2024-09-21T06:31:37.488790+0000","last_active":"2024-09-21T06:31:37.491325+0000","last_peered":"2024-09-21T06:31:37.491325+0000","last_clean":"2024-09-21T06:31:37.491325+0000","last_became_active":"2024-09-21T06:31:37.486018+0000","last_became_peered":"2024-09-21T06:31:37.486018+0000","last_unstale":"2024-09-21T06:31:37.491325+0000","last_undegraded":"2024-09-21T06:31:37.491325+0000","last_fullsized":"2024-09-21T06:31:37.491325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:27.357677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465686+0000","last_change":"2024-09-21T06:31:37.465686+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465686+0000","last_undegraded":"2024-09-21T06:31:37.465686+0000","last_fullsized":"2024-09-21T06:31:37.465686+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.491148+0000","last_change":"2024-09-21T06:31:37.489892+0000","last_active":"2024-09-21T06:31:37.491148+0000","last_peered":"2024-09-21T06:31:37.491148+0000","last_clean":"2024-09-21T06:31:37.491148+0000","last_became_active":"2024-09-21T06:31:37.489477+0000","last_became_peered":"2024-09-21T06:31:37.489477+0000","last_unstale":"2024-09-21T06:31:37.491148+0000","last_undegraded":"2024-09-21T06:31:37.491148+0000","last_fullsized":"2024-09-21T06:31:37.491148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:55.589790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484209+0000","last_change":"2024-09-21T06:31:38.478388+0000","last_active":"2024-09-21T06:31:39.484209+0000","last_peered":"2024-09-21T06:31:39.484209+0000","last_clean":"2024-09-21T06:31:39.484209+0000","last_became_active":"2024-09-21T06:31:38.478068+0000","last_became_peered":"2024-09-21T06:31:38.478068+0000","last_unstale":"2024-09-21T06:31:39.484209+0000","last_undegraded":"2024-09-21T06:31:39.484209+0000","last_fullsized":"2024-09-21T06:31:39.484209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:56.126853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484563+0000","last_change":"2024-09-21T06:31:37.479881+0000","last_active":"2024-09-21T06:31:39.484563+0000","last_peered":"2024-09-21T06:31:39.484563+0000","last_clean":"2024-09-21T06:31:39.484563+0000","last_became_active":"2024-09-21T06:31:37.479532+0000","last_became_peered":"2024-09-21T06:31:37.479532+0000","last_unstale":"2024-09-21T06:31:39.484563+0000","last_undegraded":"2024-09-21T06:31:39.484563+0000","last_fullsized":"2024-09-21T06:31:39.484563+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:12.733895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.463728+0000","last_change":"2024-09-21T06:31:37.463728+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.463728+0000","last_undegraded":"2024-09-21T06:31:37.463728+0000","last_fullsized":"2024-09-21T06:31:37.463728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.491861+0000","last_change":"2024-09-21T06:31:37.490781+0000","last_active":"2024-09-21T06:31:37.491861+0000","last_peered":"2024-09-21T06:31:37.491861+0000","last_clean":"2024-09-21T06:31:37.491861+0000","last_became_active":"2024-09-21T06:31:37.489848+0000","last_became_peered":"2024-09-21T06:31:37.489848+0000","last_unstale":"2024-09-21T06:31:37.491861+0000","last_undegraded":"2024-09-21T06:31:37.491861+0000","last_fullsized":"2024-09-21T06:31:37.491861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:55.589790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.465730+0000","last_change":"2024-09-21T06:31:37.465730+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.465730+0000","last_undegraded":"2024-09-21T06:31:37.465730+0000","last_fullsized":"2024-09-21T06:31:37.465730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483934+0000","last_change":"2024-09-21T06:31:37.482866+0000","last_active":"2024-09-21T06:31:39.483934+0000","last_peered":"2024-09-21T06:31:39.483934+0000","last_clean":"2024-09-21T06:31:39.483934+0000","last_became_active":"2024-09-21T06:31:37.482450+0000","last_became_peered":"2024-09-21T06:31:37.482450+0000","last_unstale":"2024-09-21T06:31:39.483934+0000","last_undegraded":"2024-09-21T06:31:39.483934+0000","last_fullsized":"2024-09-21T06:31:39.483934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:35.577652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.460286+0000","last_change":"2024-09-21T06:31:37.460286+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.460286+0000","last_undegraded":"2024-09-21T06:31:37.460286+0000","last_fullsized":"2024-09-21T06:31:37.460286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.489345+0000","last_change":"2024-09-21T06:31:37.487752+0000","last_active":"2024-09-21T06:31:37.489345+0000","last_peered":"2024-09-21T06:31:37.489345+0000","last_clean":"2024-09-21T06:31:37.489345+0000","last_became_active":"2024-09-21T06:31:37.487280+0000","last_became_peered":"2024-09-21T06:31:37.487280+0000","last_unstale":"2024-09-21T06:31:37.489345+0000","last_undegraded":"2024-09-21T06:31:37.489345+0000","last_fullsized":"2024-09-21T06:31:37.489345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:48.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.464172+0000","last_change":"2024-09-21T06:31:37.464172+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.464172+0000","last_undegraded":"2024-09-21T06:31:37.464172+0000","last_fullsized":"2024-09-21T06:31:37.464172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.483620+0000","last_change":"2024-09-21T06:31:37.480490+0000","last_active":"2024-09-21T06:31:37.483620+0000","last_peered":"2024-09-21T06:31:37.483620+0000","last_clean":"2024-09-21T06:31:37.483620+0000","last_became_active":"2024-09-21T06:31:37.479181+0000","last_became_peered":"2024-09-21T06:31:37.479181+0000","last_unstale":"2024-09-21T06:31:37.483620+0000","last_undegraded":"2024-09-21T06:31:37.483620+0000","last_fullsized":"2024-09-21T06:31:37.483620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:55.589790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.464455+0000","last_change":"2024-09-21T06:31:37.464455+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.464455+0000","last_undegraded":"2024-09-21T06:31:37.464455+0000","last_fullsized":"2024-09-21T06:31:37.464455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484239+0000","last_change":"2024-09-21T06:31:37.478225+0000","last_active":"2024-09-21T06:31:39.484239+0000","last_peered":"2024-09-21T06:31:39.484239+0000","last_clean":"2024-09-21T06:31:39.484239+0000","last_became_active":"2024-09-21T06:31:37.477900+0000","last_became_peered":"2024-09-21T06:31:37.477900+0000","last_unstale":"2024-09-21T06:31:39.484239+0000","last_undegraded":"2024-09-21T06:31:39.484239+0000","last_fullsized":"2024-09-21T06:31:39.484239+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:18.545420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.464056+0000","last_change":"2024-09-21T06:31:37.464056+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.464056+0000","last_undegraded":"2024-09-21T06:31:37.464056+0000","last_fullsized":"2024-09-21T06:31:37.464056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.489779+0000","last_change":"2024-09-21T06:31:37.487862+0000","last_active":"2024-09-21T06:31:37.489779+0000","last_peered":"2024-09-21T06:31:37.489779+0000","last_clean":"2024-09-21T06:31:37.489779+0000","last_became_active":"2024-09-21T06:31:37.487545+0000","last_became_peered":"2024-09-21T06:31:37.487545+0000","last_unstale":"2024-09-21T06:31:37.489779+0000","last_undegraded":"2024-09-21T06:31:37.489779+0000","last_fullsized":"2024-09-21T06:31:37.489779+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:54.749049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.467047+0000","last_change":"2024-09-21T06:31:37.467047+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.467047+0000","last_undegraded":"2024-09-21T06:31:37.467047+0000","last_fullsized":"2024-09-21T06:31:37.467047+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.478905+0000","last_change":"2024-09-21T06:31:37.471427+0000","last_active":"2024-09-21T06:31:37.478905+0000","last_peered":"2024-09-21T06:31:37.478905+0000","last_clean":"2024-09-21T06:31:37.478905+0000","last_became_active":"2024-09-21T06:31:37.470772+0000","last_became_peered":"2024-09-21T06:31:37.470772+0000","last_unstale":"2024-09-21T06:31:37.478905+0000","last_undegraded":"2024-09-21T06:31:37.478905+0000","last_fullsized":"2024-09-21T06:31:37.478905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:31.123522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.463149+0000","last_change":"2024-09-21T06:31:37.463149+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.463149+0000","last_undegraded":"2024-09-21T06:31:37.463149+0000","last_fullsized":"2024-09-21T06:31:37.463149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.488276+0000","last_change":"2024-09-21T06:31:37.477853+0000","last_active":"2024-09-21T06:31:37.488276+0000","last_peered":"2024-09-21T06:31:37.488276+0000","last_clean":"2024-09-21T06:31:37.488276+0000","last_became_active":"2024-09-21T06:31:37.477546+0000","last_became_peered":"2024-09-21T06:31:37.477546+0000","last_unstale":"2024-09-21T06:31:37.488276+0000","last_undegraded":"2024-09-21T06:31:37.488276+0000","last_fullsized":"2024-09-21T06:31:37.488276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:51.745128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484825+0000","last_change":"2024-09-21T06:31:38.492957+0000","last_active":"2024-09-21T06:31:39.484825+0000","last_peered":"2024-09-21T06:31:39.484825+0000","last_clean":"2024-09-21T06:31:39.484825+0000","last_became_active":"2024-09-21T06:31:38.492025+0000","last_became_peered":"2024-09-21T06:31:38.492025+0000","last_unstale":"2024-09-21T06:31:39.484825+0000","last_undegraded":"2024-09-21T06:31:39.484825+0000","last_fullsized":"2024-09-21T06:31:39.484825+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:16.306533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.488337+0000","last_change":"2024-09-21T06:31:37.485763+0000","last_active":"2024-09-21T06:31:37.488337+0000","last_peered":"2024-09-21T06:31:37.488337+0000","last_clean":"2024-09-21T06:31:37.488337+0000","last_became_active":"2024-09-21T06:31:37.485087+0000","last_became_peered":"2024-09-21T06:31:37.485087+0000","last_unstale":"2024-09-21T06:31:37.488337+0000","last_undegraded":"2024-09-21T06:31:37.488337+0000","last_fullsized":"2024-09-21T06:31:37.488337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:52.076981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.460797+0000","last_change":"2024-09-21T06:31:37.460797+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.460797+0000","last_undegraded":"2024-09-21T06:31:37.460797+0000","last_fullsized":"2024-09-21T06:31:37.460797+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484054+0000","last_change":"2024-09-21T06:31:37.478552+0000","last_active":"2024-09-21T06:31:39.484054+0000","last_peered":"2024-09-21T06:31:39.484054+0000","last_clean":"2024-09-21T06:31:39.484054+0000","last_became_active":"2024-09-21T06:31:37.476760+0000","last_became_peered":"2024-09-21T06:31:37.476760+0000","last_unstale":"2024-09-21T06:31:39.484054+0000","last_undegraded":"2024-09-21T06:31:39.484054+0000","last_fullsized":"2024-09-21T06:31:39.484054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:59.379517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T06:31:37.466316+0000","last_change":"2024-09-21T06:31:37.466316+0000","last_active":"2024-09-21T06:31:37.448823+0000","last_peered":"2024-09-21T06:31:37.448823+0000","last_clean":"2024-09-21T06:31:37.448823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:31:37.466316+0000","last_undegraded":"2024-09-21T06:31:37.466316+0000","last_fullsized":"2024-09-21T06:31:37.466316+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484185+0000","last_change":"2024-09-21T06:31:37.486831+0000","last_active":"2024-09-21T06:31:39.484185+0000","last_peered":"2024-09-21T06:31:39.484185+0000","last_clean":"2024-09-21T06:31:39.484185+0000","last_became_active":"2024-09-21T06:31:37.486531+0000","last_became_peered":"2024-09-21T06:31:37.486531+0000","last_unstale":"2024-09-21T06:31:39.484185+0000","last_undegraded":"2024-09-21T06:31:39.484185+0000","last_fullsized":"2024-09-21T06:31:39.484185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:18.361705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T06:31:37.473553+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:31:37.473553+0000","last_peered":"2024-09-21T06:31:37.473553+0000","last_clean":"2024-09-21T06:31:37.473553+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:31:37.473553+0000","last_undegraded":"2024-09-21T06:31:37.473553+0000","last_fullsized":"2024-09-21T06:31:37.473553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":348502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":348502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":765952,"data_stored":341835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":348502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":778240,"data_stored":348502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":932203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":341835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1404928,"data_stored":947761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:31:42.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:31:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 -- 172.21.15.37:0/555993641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac06e500 msgr2=0x7f9eac0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/555993641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac06e500 0x7f9eac0cdb90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005fd0 tx=0x7f9ea402bc70 comp rx=0 tx=0).stop 2024-09-21T06:31:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 -- 172.21.15.37:0/555993641 shutdown_connections 2024-09-21T06:31:43.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/555993641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac06e500 0x7f9eac0cdb90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 -- 172.21.15.37:0/555993641 >> 172.21.15.37:0/555993641 conn(0x7f9eac0cca40 msgr2=0x7f9eac0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:43.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 -- 172.21.15.37:0/555993641 shutdown_connections 2024-09-21T06:31:43.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.235+0000 7f9eb1e00640 1 -- 172.21.15.37:0/555993641 wait complete. 2024-09-21T06:31:43.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.237+0000 7f9eb1e00640 1 Processor -- start 2024-09-21T06:31:43.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.237+0000 7f9eb1e00640 1 -- start start 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 0x7f9eac1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 0x7f9eac1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac15ace0 con 0x7f9eac070030 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac071090 con 0x7f9eac09e380 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eb1e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0ce0d0 con 0x7f9eac074d90 2024-09-21T06:31:43.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34852/0 (socket says 172.21.15.37:34852) 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 -- 172.21.15.37:0/2735601623 learned_addr learned my addr 172.21.15.37:0/2735601623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eab400640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 0x7f9eac1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 -- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 msgr2=0x7f9eac1f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaa000640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 0x7f9eac1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 0x7f9eac1f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 -- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 msgr2=0x7f9eac1f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 0x7f9eac1f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaaa00640 1 -- 172.21.15.37:0/2735601623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac0a1080 con 0x7f9eac070030 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eab400640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 0x7f9eac1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.239+0000 7f9eaa000640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 0x7f9eac1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9eaaa00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c400 tx=0x7f9e9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c011790 con 0x7f9eac070030 2024-09-21T06:31:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e9c017070 con 0x7f9eac070030 2024-09-21T06:31:43.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac0b5570 con 0x7f9eac070030 2024-09-21T06:31:43.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c011cc0 con 0x7f9eac070030 2024-09-21T06:31:43.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.240+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac0b5b30 con 0x7f9eac070030 2024-09-21T06:31:43.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.241+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac0ce5f0 con 0x7f9eac070030 2024-09-21T06:31:43.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.242+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e9c01c460 con 0x7f9eac070030 2024-09-21T06:31:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.243+0000 7f9ea3400640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.244+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f9e9c09ad20 con 0x7f9eac070030 2024-09-21T06:31:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.244+0000 7f9eaa000640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.245+0000 7f9eaa000640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd8d0 tx=0x7f9e98009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:43.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.248+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9eac0a1080 con 0x7f9eac070030 2024-09-21T06:31:43.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.441+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9eac0b5f40 con 0x7f9e8c078b90 2024-09-21T06:31:43.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.446+0000 7f9ea3400640 1 -- 172.21.15.37:0/2735601623 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252013 (secure 0 0 0) 0x7f9eac0b5f40 con 0x7f9e8c078b90 2024-09-21T06:31:43.451 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:31:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 msgr2=0x7f9e8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd8d0 tx=0x7f9e98009040 comp rx=0 tx=0).stop 2024-09-21T06:31:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 msgr2=0x7f9eac158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c400 tx=0x7f9e9c00c8c0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 shutdown_connections 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9eac09e380 0x7f9eac1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9eac074d90 0x7f9eac1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 --2- 172.21.15.37:0/2735601623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9eac070030 0x7f9eac158020 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 >> 172.21.15.37:0/2735601623 conn(0x7f9eac0cca40 msgr2=0x7f9eac06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 shutdown_connections 2024-09-21T06:31:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.450+0000 7f9eb1e00640 1 -- 172.21.15.37:0/2735601623 wait complete. 2024-09-21T06:31:43.478 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-21T06:31:43.148236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244564,"kb_used_data":7484,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730156,"statfs":{"total":773094113280,"available":772843679744,"internally_reserved":0,"allocated":7663616,"data_stored":4077752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001773"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956946+0000","last_change":"2024-09-21T06:31:38.499606+0000","last_active":"2024-09-21T06:31:39.956946+0000","last_peered":"2024-09-21T06:31:39.956946+0000","last_clean":"2024-09-21T06:31:39.956946+0000","last_became_active":"2024-09-21T06:31:38.499326+0000","last_became_peered":"2024-09-21T06:31:38.499326+0000","last_unstale":"2024-09-21T06:31:39.956946+0000","last_undegraded":"2024-09-21T06:31:39.956946+0000","last_fullsized":"2024-09-21T06:31:39.956946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:14.523452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484546+0000","last_change":"2024-09-21T06:31:38.481832+0000","last_active":"2024-09-21T06:31:39.484546+0000","last_peered":"2024-09-21T06:31:39.484546+0000","last_clean":"2024-09-21T06:31:39.484546+0000","last_became_active":"2024-09-21T06:31:38.481202+0000","last_became_peered":"2024-09-21T06:31:38.481202+0000","last_unstale":"2024-09-21T06:31:39.484546+0000","last_undegraded":"2024-09-21T06:31:39.484546+0000","last_fullsized":"2024-09-21T06:31:39.484546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:19.061826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957129+0000","last_change":"2024-09-21T06:31:38.490911+0000","last_active":"2024-09-21T06:31:39.957129+0000","last_peered":"2024-09-21T06:31:39.957129+0000","last_clean":"2024-09-21T06:31:39.957129+0000","last_became_active":"2024-09-21T06:31:38.489980+0000","last_became_peered":"2024-09-21T06:31:38.489980+0000","last_unstale":"2024-09-21T06:31:39.957129+0000","last_undegraded":"2024-09-21T06:31:39.957129+0000","last_fullsized":"2024-09-21T06:31:39.957129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:47.968953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568411+0000","last_change":"2024-09-21T06:31:38.494404+0000","last_active":"2024-09-21T06:31:39.568411+0000","last_peered":"2024-09-21T06:31:39.568411+0000","last_clean":"2024-09-21T06:31:39.568411+0000","last_became_active":"2024-09-21T06:31:38.492368+0000","last_became_peered":"2024-09-21T06:31:38.492368+0000","last_unstale":"2024-09-21T06:31:39.568411+0000","last_undegraded":"2024-09-21T06:31:39.568411+0000","last_fullsized":"2024-09-21T06:31:39.568411+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:11.962663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956528+0000","last_change":"2024-09-21T06:31:38.505820+0000","last_active":"2024-09-21T06:31:39.956528+0000","last_peered":"2024-09-21T06:31:39.956528+0000","last_clean":"2024-09-21T06:31:39.956528+0000","last_became_active":"2024-09-21T06:31:38.505473+0000","last_became_peered":"2024-09-21T06:31:38.505473+0000","last_unstale":"2024-09-21T06:31:39.956528+0000","last_undegraded":"2024-09-21T06:31:39.956528+0000","last_fullsized":"2024-09-21T06:31:39.956528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:41.968528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484571+0000","last_change":"2024-09-21T06:31:38.482703+0000","last_active":"2024-09-21T06:31:39.484571+0000","last_peered":"2024-09-21T06:31:39.484571+0000","last_clean":"2024-09-21T06:31:39.484571+0000","last_became_active":"2024-09-21T06:31:38.481272+0000","last_became_peered":"2024-09-21T06:31:38.481272+0000","last_unstale":"2024-09-21T06:31:39.484571+0000","last_undegraded":"2024-09-21T06:31:39.484571+0000","last_fullsized":"2024-09-21T06:31:39.484571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:45.840338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956683+0000","last_change":"2024-09-21T06:31:38.487556+0000","last_active":"2024-09-21T06:31:39.956683+0000","last_peered":"2024-09-21T06:31:39.956683+0000","last_clean":"2024-09-21T06:31:39.956683+0000","last_became_active":"2024-09-21T06:31:38.486529+0000","last_became_peered":"2024-09-21T06:31:38.486529+0000","last_unstale":"2024-09-21T06:31:39.956683+0000","last_undegraded":"2024-09-21T06:31:39.956683+0000","last_fullsized":"2024-09-21T06:31:39.956683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:40.908557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956261+0000","last_change":"2024-09-21T06:31:38.495793+0000","last_active":"2024-09-21T06:31:39.956261+0000","last_peered":"2024-09-21T06:31:39.956261+0000","last_clean":"2024-09-21T06:31:39.956261+0000","last_became_active":"2024-09-21T06:31:38.495488+0000","last_became_peered":"2024-09-21T06:31:38.495488+0000","last_unstale":"2024-09-21T06:31:39.956261+0000","last_undegraded":"2024-09-21T06:31:39.956261+0000","last_fullsized":"2024-09-21T06:31:39.956261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:13.479999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568380+0000","last_change":"2024-09-21T06:31:38.501954+0000","last_active":"2024-09-21T06:31:39.568380+0000","last_peered":"2024-09-21T06:31:39.568380+0000","last_clean":"2024-09-21T06:31:39.568380+0000","last_became_active":"2024-09-21T06:31:38.499268+0000","last_became_peered":"2024-09-21T06:31:38.499268+0000","last_unstale":"2024-09-21T06:31:39.568380+0000","last_undegraded":"2024-09-21T06:31:39.568380+0000","last_fullsized":"2024-09-21T06:31:39.568380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:54.974335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.480053+0000","last_change":"2024-09-21T06:31:38.495251+0000","last_active":"2024-09-21T06:31:39.480053+0000","last_peered":"2024-09-21T06:31:39.480053+0000","last_clean":"2024-09-21T06:31:39.480053+0000","last_became_active":"2024-09-21T06:31:38.494744+0000","last_became_peered":"2024-09-21T06:31:38.494744+0000","last_unstale":"2024-09-21T06:31:39.480053+0000","last_undegraded":"2024-09-21T06:31:39.480053+0000","last_fullsized":"2024-09-21T06:31:39.480053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:52.235633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568069+0000","last_change":"2024-09-21T06:31:38.500652+0000","last_active":"2024-09-21T06:31:39.568069+0000","last_peered":"2024-09-21T06:31:39.568069+0000","last_clean":"2024-09-21T06:31:39.568069+0000","last_became_active":"2024-09-21T06:31:38.499029+0000","last_became_peered":"2024-09-21T06:31:38.499029+0000","last_unstale":"2024-09-21T06:31:39.568069+0000","last_undegraded":"2024-09-21T06:31:39.568069+0000","last_fullsized":"2024-09-21T06:31:39.568069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:22.825185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568320+0000","last_change":"2024-09-21T06:31:38.501285+0000","last_active":"2024-09-21T06:31:39.568320+0000","last_peered":"2024-09-21T06:31:39.568320+0000","last_clean":"2024-09-21T06:31:39.568320+0000","last_became_active":"2024-09-21T06:31:38.499143+0000","last_became_peered":"2024-09-21T06:31:38.499143+0000","last_unstale":"2024-09-21T06:31:39.568320+0000","last_undegraded":"2024-09-21T06:31:39.568320+0000","last_fullsized":"2024-09-21T06:31:39.568320+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:32.019342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956639+0000","last_change":"2024-09-21T06:31:38.507698+0000","last_active":"2024-09-21T06:31:39.956639+0000","last_peered":"2024-09-21T06:31:39.956639+0000","last_clean":"2024-09-21T06:31:39.956639+0000","last_became_active":"2024-09-21T06:31:38.505975+0000","last_became_peered":"2024-09-21T06:31:38.505975+0000","last_unstale":"2024-09-21T06:31:39.956639+0000","last_undegraded":"2024-09-21T06:31:39.956639+0000","last_fullsized":"2024-09-21T06:31:39.956639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:59.718158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956604+0000","last_change":"2024-09-21T06:31:38.481618+0000","last_active":"2024-09-21T06:31:39.956604+0000","last_peered":"2024-09-21T06:31:39.956604+0000","last_clean":"2024-09-21T06:31:39.956604+0000","last_became_active":"2024-09-21T06:31:38.481268+0000","last_became_peered":"2024-09-21T06:31:38.481268+0000","last_unstale":"2024-09-21T06:31:39.956604+0000","last_undegraded":"2024-09-21T06:31:39.956604+0000","last_fullsized":"2024-09-21T06:31:39.956604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:19.886556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479590+0000","last_change":"2024-09-21T06:31:38.483971+0000","last_active":"2024-09-21T06:31:39.479590+0000","last_peered":"2024-09-21T06:31:39.479590+0000","last_clean":"2024-09-21T06:31:39.479590+0000","last_became_active":"2024-09-21T06:31:38.482822+0000","last_became_peered":"2024-09-21T06:31:38.482822+0000","last_unstale":"2024-09-21T06:31:39.479590+0000","last_undegraded":"2024-09-21T06:31:39.479590+0000","last_fullsized":"2024-09-21T06:31:39.479590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:16.010719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956755+0000","last_change":"2024-09-21T06:31:38.484295+0000","last_active":"2024-09-21T06:31:39.956755+0000","last_peered":"2024-09-21T06:31:39.956755+0000","last_clean":"2024-09-21T06:31:39.956755+0000","last_became_active":"2024-09-21T06:31:38.483503+0000","last_became_peered":"2024-09-21T06:31:38.483503+0000","last_unstale":"2024-09-21T06:31:39.956755+0000","last_undegraded":"2024-09-21T06:31:39.956755+0000","last_fullsized":"2024-09-21T06:31:39.956755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:33.047124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483128+0000","last_change":"2024-09-21T06:31:38.484282+0000","last_active":"2024-09-21T06:31:39.483128+0000","last_peered":"2024-09-21T06:31:39.483128+0000","last_clean":"2024-09-21T06:31:39.483128+0000","last_became_active":"2024-09-21T06:31:38.483959+0000","last_became_peered":"2024-09-21T06:31:38.483959+0000","last_unstale":"2024-09-21T06:31:39.483128+0000","last_undegraded":"2024-09-21T06:31:39.483128+0000","last_fullsized":"2024-09-21T06:31:39.483128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:32.321890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568299+0000","last_change":"2024-09-21T06:31:38.501187+0000","last_active":"2024-09-21T06:31:39.568299+0000","last_peered":"2024-09-21T06:31:39.568299+0000","last_clean":"2024-09-21T06:31:39.568299+0000","last_became_active":"2024-09-21T06:31:38.500821+0000","last_became_peered":"2024-09-21T06:31:38.500821+0000","last_unstale":"2024-09-21T06:31:39.568299+0000","last_undegraded":"2024-09-21T06:31:39.568299+0000","last_fullsized":"2024-09-21T06:31:39.568299+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:26.785119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568170+0000","last_change":"2024-09-21T06:31:38.500518+0000","last_active":"2024-09-21T06:31:39.568170+0000","last_peered":"2024-09-21T06:31:39.568170+0000","last_clean":"2024-09-21T06:31:39.568170+0000","last_became_active":"2024-09-21T06:31:38.498660+0000","last_became_peered":"2024-09-21T06:31:38.498660+0000","last_unstale":"2024-09-21T06:31:39.568170+0000","last_undegraded":"2024-09-21T06:31:39.568170+0000","last_fullsized":"2024-09-21T06:31:39.568170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:46.042438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483291+0000","last_change":"2024-09-21T06:31:38.494817+0000","last_active":"2024-09-21T06:31:39.483291+0000","last_peered":"2024-09-21T06:31:39.483291+0000","last_clean":"2024-09-21T06:31:39.483291+0000","last_became_active":"2024-09-21T06:31:38.494469+0000","last_became_peered":"2024-09-21T06:31:38.494469+0000","last_unstale":"2024-09-21T06:31:39.483291+0000","last_undegraded":"2024-09-21T06:31:39.483291+0000","last_fullsized":"2024-09-21T06:31:39.483291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:24.878159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956844+0000","last_change":"2024-09-21T06:31:38.475083+0000","last_active":"2024-09-21T06:31:39.956844+0000","last_peered":"2024-09-21T06:31:39.956844+0000","last_clean":"2024-09-21T06:31:39.956844+0000","last_became_active":"2024-09-21T06:31:38.474792+0000","last_became_peered":"2024-09-21T06:31:38.474792+0000","last_unstale":"2024-09-21T06:31:39.956844+0000","last_undegraded":"2024-09-21T06:31:39.956844+0000","last_fullsized":"2024-09-21T06:31:39.956844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:42.052641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484776+0000","last_change":"2024-09-21T06:31:38.481328+0000","last_active":"2024-09-21T06:31:39.484776+0000","last_peered":"2024-09-21T06:31:39.484776+0000","last_clean":"2024-09-21T06:31:39.484776+0000","last_became_active":"2024-09-21T06:31:38.480298+0000","last_became_peered":"2024-09-21T06:31:38.480298+0000","last_unstale":"2024-09-21T06:31:39.484776+0000","last_undegraded":"2024-09-21T06:31:39.484776+0000","last_fullsized":"2024-09-21T06:31:39.484776+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:34.869541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957237+0000","last_change":"2024-09-21T06:31:38.493422+0000","last_active":"2024-09-21T06:31:39.957237+0000","last_peered":"2024-09-21T06:31:39.957237+0000","last_clean":"2024-09-21T06:31:39.957237+0000","last_became_active":"2024-09-21T06:31:38.493101+0000","last_became_peered":"2024-09-21T06:31:38.493101+0000","last_unstale":"2024-09-21T06:31:39.957237+0000","last_undegraded":"2024-09-21T06:31:39.957237+0000","last_fullsized":"2024-09-21T06:31:39.957237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:38.219567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956689+0000","last_change":"2024-09-21T06:31:38.482679+0000","last_active":"2024-09-21T06:31:39.956689+0000","last_peered":"2024-09-21T06:31:39.956689+0000","last_clean":"2024-09-21T06:31:39.956689+0000","last_became_active":"2024-09-21T06:31:38.482359+0000","last_became_peered":"2024-09-21T06:31:38.482359+0000","last_unstale":"2024-09-21T06:31:39.956689+0000","last_undegraded":"2024-09-21T06:31:39.956689+0000","last_fullsized":"2024-09-21T06:31:39.956689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:44.776221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957453+0000","last_change":"2024-09-21T06:31:38.476211+0000","last_active":"2024-09-21T06:31:39.957453+0000","last_peered":"2024-09-21T06:31:39.957453+0000","last_clean":"2024-09-21T06:31:39.957453+0000","last_became_active":"2024-09-21T06:31:38.474433+0000","last_became_peered":"2024-09-21T06:31:38.474433+0000","last_unstale":"2024-09-21T06:31:39.957453+0000","last_undegraded":"2024-09-21T06:31:39.957453+0000","last_fullsized":"2024-09-21T06:31:39.957453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:19.696194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957045+0000","last_change":"2024-09-21T06:31:38.492432+0000","last_active":"2024-09-21T06:31:39.957045+0000","last_peered":"2024-09-21T06:31:39.957045+0000","last_clean":"2024-09-21T06:31:39.957045+0000","last_became_active":"2024-09-21T06:31:38.492043+0000","last_became_peered":"2024-09-21T06:31:38.492043+0000","last_unstale":"2024-09-21T06:31:39.957045+0000","last_undegraded":"2024-09-21T06:31:39.957045+0000","last_fullsized":"2024-09-21T06:31:39.957045+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:46.990385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956557+0000","last_change":"2024-09-21T06:31:38.489237+0000","last_active":"2024-09-21T06:31:39.956557+0000","last_peered":"2024-09-21T06:31:39.956557+0000","last_clean":"2024-09-21T06:31:39.956557+0000","last_became_active":"2024-09-21T06:31:38.488899+0000","last_became_peered":"2024-09-21T06:31:38.488899+0000","last_unstale":"2024-09-21T06:31:39.956557+0000","last_undegraded":"2024-09-21T06:31:39.956557+0000","last_fullsized":"2024-09-21T06:31:39.956557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:23.957131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956627+0000","last_change":"2024-09-21T06:31:38.495261+0000","last_active":"2024-09-21T06:31:39.956627+0000","last_peered":"2024-09-21T06:31:39.956627+0000","last_clean":"2024-09-21T06:31:39.956627+0000","last_became_active":"2024-09-21T06:31:38.494962+0000","last_became_peered":"2024-09-21T06:31:38.494962+0000","last_unstale":"2024-09-21T06:31:39.956627+0000","last_undegraded":"2024-09-21T06:31:39.956627+0000","last_fullsized":"2024-09-21T06:31:39.956627+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:22.160183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568783+0000","last_change":"2024-09-21T06:31:38.504286+0000","last_active":"2024-09-21T06:31:39.568783+0000","last_peered":"2024-09-21T06:31:39.568783+0000","last_clean":"2024-09-21T06:31:39.568783+0000","last_became_active":"2024-09-21T06:31:38.503952+0000","last_became_peered":"2024-09-21T06:31:38.503952+0000","last_unstale":"2024-09-21T06:31:39.568783+0000","last_undegraded":"2024-09-21T06:31:39.568783+0000","last_fullsized":"2024-09-21T06:31:39.568783+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:40.876595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.567809+0000","last_change":"2024-09-21T06:31:38.492548+0000","last_active":"2024-09-21T06:31:39.567809+0000","last_peered":"2024-09-21T06:31:39.567809+0000","last_clean":"2024-09-21T06:31:39.567809+0000","last_became_active":"2024-09-21T06:31:38.492181+0000","last_became_peered":"2024-09-21T06:31:38.492181+0000","last_unstale":"2024-09-21T06:31:39.567809+0000","last_undegraded":"2024-09-21T06:31:39.567809+0000","last_fullsized":"2024-09-21T06:31:39.567809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:42.204279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956564+0000","last_change":"2024-09-21T06:31:38.488449+0000","last_active":"2024-09-21T06:31:39.956564+0000","last_peered":"2024-09-21T06:31:39.956564+0000","last_clean":"2024-09-21T06:31:39.956564+0000","last_became_active":"2024-09-21T06:31:38.485767+0000","last_became_peered":"2024-09-21T06:31:38.485767+0000","last_unstale":"2024-09-21T06:31:39.956564+0000","last_undegraded":"2024-09-21T06:31:39.956564+0000","last_fullsized":"2024-09-21T06:31:39.956564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:12.934593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956437+0000","last_change":"2024-09-21T06:31:38.491545+0000","last_active":"2024-09-21T06:31:39.956437+0000","last_peered":"2024-09-21T06:31:39.956437+0000","last_clean":"2024-09-21T06:31:39.956437+0000","last_became_active":"2024-09-21T06:31:38.491147+0000","last_became_peered":"2024-09-21T06:31:38.491147+0000","last_unstale":"2024-09-21T06:31:39.956437+0000","last_undegraded":"2024-09-21T06:31:39.956437+0000","last_fullsized":"2024-09-21T06:31:39.956437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:47.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568672+0000","last_change":"2024-09-21T06:31:38.496850+0000","last_active":"2024-09-21T06:31:39.568672+0000","last_peered":"2024-09-21T06:31:39.568672+0000","last_clean":"2024-09-21T06:31:39.568672+0000","last_became_active":"2024-09-21T06:31:38.494860+0000","last_became_peered":"2024-09-21T06:31:38.494860+0000","last_unstale":"2024-09-21T06:31:39.568672+0000","last_undegraded":"2024-09-21T06:31:39.568672+0000","last_fullsized":"2024-09-21T06:31:39.568672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:34.964909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956945+0000","last_change":"2024-09-21T06:31:37.484380+0000","last_active":"2024-09-21T06:31:39.956945+0000","last_peered":"2024-09-21T06:31:39.956945+0000","last_clean":"2024-09-21T06:31:39.956945+0000","last_became_active":"2024-09-21T06:31:37.484046+0000","last_became_peered":"2024-09-21T06:31:37.484046+0000","last_unstale":"2024-09-21T06:31:39.956945+0000","last_undegraded":"2024-09-21T06:31:39.956945+0000","last_fullsized":"2024-09-21T06:31:39.956945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:58.574576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957184+0000","last_change":"2024-09-21T06:31:38.503196+0000","last_active":"2024-09-21T06:31:39.957184+0000","last_peered":"2024-09-21T06:31:39.957184+0000","last_clean":"2024-09-21T06:31:39.957184+0000","last_became_active":"2024-09-21T06:31:38.502899+0000","last_became_peered":"2024-09-21T06:31:38.502899+0000","last_unstale":"2024-09-21T06:31:39.957184+0000","last_undegraded":"2024-09-21T06:31:39.957184+0000","last_fullsized":"2024-09-21T06:31:39.957184+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:14.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956560+0000","last_change":"2024-09-21T06:31:37.482254+0000","last_active":"2024-09-21T06:31:39.956560+0000","last_peered":"2024-09-21T06:31:39.956560+0000","last_clean":"2024-09-21T06:31:39.956560+0000","last_became_active":"2024-09-21T06:31:37.481797+0000","last_became_peered":"2024-09-21T06:31:37.481797+0000","last_unstale":"2024-09-21T06:31:39.956560+0000","last_undegraded":"2024-09-21T06:31:39.956560+0000","last_fullsized":"2024-09-21T06:31:39.956560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:21.687638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484906+0000","last_change":"2024-09-21T06:31:38.481536+0000","last_active":"2024-09-21T06:31:39.484906+0000","last_peered":"2024-09-21T06:31:39.484906+0000","last_clean":"2024-09-21T06:31:39.484906+0000","last_became_active":"2024-09-21T06:31:38.480779+0000","last_became_peered":"2024-09-21T06:31:38.480779+0000","last_unstale":"2024-09-21T06:31:39.484906+0000","last_undegraded":"2024-09-21T06:31:39.484906+0000","last_fullsized":"2024-09-21T06:31:39.484906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:10.890474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956988+0000","last_change":"2024-09-21T06:31:37.483834+0000","last_active":"2024-09-21T06:31:39.956988+0000","last_peered":"2024-09-21T06:31:39.956988+0000","last_clean":"2024-09-21T06:31:39.956988+0000","last_became_active":"2024-09-21T06:31:37.483475+0000","last_became_peered":"2024-09-21T06:31:37.483475+0000","last_unstale":"2024-09-21T06:31:39.956988+0000","last_undegraded":"2024-09-21T06:31:39.956988+0000","last_fullsized":"2024-09-21T06:31:39.956988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:10.229100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568225+0000","last_change":"2024-09-21T06:31:38.500677+0000","last_active":"2024-09-21T06:31:39.568225+0000","last_peered":"2024-09-21T06:31:39.568225+0000","last_clean":"2024-09-21T06:31:39.568225+0000","last_became_active":"2024-09-21T06:31:38.498630+0000","last_became_peered":"2024-09-21T06:31:38.498630+0000","last_unstale":"2024-09-21T06:31:39.568225+0000","last_undegraded":"2024-09-21T06:31:39.568225+0000","last_fullsized":"2024-09-21T06:31:39.568225+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.543818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484473+0000","last_change":"2024-09-21T06:31:37.481111+0000","last_active":"2024-09-21T06:31:39.484473+0000","last_peered":"2024-09-21T06:31:39.484473+0000","last_clean":"2024-09-21T06:31:39.484473+0000","last_became_active":"2024-09-21T06:31:37.480723+0000","last_became_peered":"2024-09-21T06:31:37.480723+0000","last_unstale":"2024-09-21T06:31:39.484473+0000","last_undegraded":"2024-09-21T06:31:39.484473+0000","last_fullsized":"2024-09-21T06:31:39.484473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:52.252258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.482372+0000","last_change":"2024-09-21T06:31:38.479881+0000","last_active":"2024-09-21T06:31:39.482372+0000","last_peered":"2024-09-21T06:31:39.482372+0000","last_clean":"2024-09-21T06:31:39.482372+0000","last_became_active":"2024-09-21T06:31:38.479536+0000","last_became_peered":"2024-09-21T06:31:38.479536+0000","last_unstale":"2024-09-21T06:31:39.482372+0000","last_undegraded":"2024-09-21T06:31:39.482372+0000","last_fullsized":"2024-09-21T06:31:39.482372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.874049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483283+0000","last_change":"2024-09-21T06:31:37.474902+0000","last_active":"2024-09-21T06:31:39.483283+0000","last_peered":"2024-09-21T06:31:39.483283+0000","last_clean":"2024-09-21T06:31:39.483283+0000","last_became_active":"2024-09-21T06:31:37.473795+0000","last_became_peered":"2024-09-21T06:31:37.473795+0000","last_unstale":"2024-09-21T06:31:39.483283+0000","last_undegraded":"2024-09-21T06:31:39.483283+0000","last_fullsized":"2024-09-21T06:31:39.483283+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:19.326328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957096+0000","last_change":"2024-09-21T06:31:38.495027+0000","last_active":"2024-09-21T06:31:39.957096+0000","last_peered":"2024-09-21T06:31:39.957096+0000","last_clean":"2024-09-21T06:31:39.957096+0000","last_became_active":"2024-09-21T06:31:38.494603+0000","last_became_peered":"2024-09-21T06:31:38.494603+0000","last_unstale":"2024-09-21T06:31:39.957096+0000","last_undegraded":"2024-09-21T06:31:39.957096+0000","last_fullsized":"2024-09-21T06:31:39.957096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:55.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484852+0000","last_change":"2024-09-21T06:31:37.476147+0000","last_active":"2024-09-21T06:31:39.484852+0000","last_peered":"2024-09-21T06:31:39.484852+0000","last_clean":"2024-09-21T06:31:39.484852+0000","last_became_active":"2024-09-21T06:31:37.475721+0000","last_became_peered":"2024-09-21T06:31:37.475721+0000","last_unstale":"2024-09-21T06:31:39.484852+0000","last_undegraded":"2024-09-21T06:31:39.484852+0000","last_fullsized":"2024-09-21T06:31:39.484852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:10.564512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.480102+0000","last_change":"2024-09-21T06:31:38.495516+0000","last_active":"2024-09-21T06:31:39.480102+0000","last_peered":"2024-09-21T06:31:39.480102+0000","last_clean":"2024-09-21T06:31:39.480102+0000","last_became_active":"2024-09-21T06:31:38.495017+0000","last_became_peered":"2024-09-21T06:31:38.495017+0000","last_unstale":"2024-09-21T06:31:39.480102+0000","last_undegraded":"2024-09-21T06:31:39.480102+0000","last_fullsized":"2024-09-21T06:31:39.480102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:47.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956890+0000","last_change":"2024-09-21T06:31:37.487702+0000","last_active":"2024-09-21T06:31:39.956890+0000","last_peered":"2024-09-21T06:31:39.956890+0000","last_clean":"2024-09-21T06:31:39.956890+0000","last_became_active":"2024-09-21T06:31:37.487201+0000","last_became_peered":"2024-09-21T06:31:37.487201+0000","last_unstale":"2024-09-21T06:31:39.956890+0000","last_undegraded":"2024-09-21T06:31:39.956890+0000","last_fullsized":"2024-09-21T06:31:39.956890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:09.288673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.482808+0000","last_change":"2024-09-21T06:31:38.504684+0000","last_active":"2024-09-21T06:31:39.482808+0000","last_peered":"2024-09-21T06:31:39.482808+0000","last_clean":"2024-09-21T06:31:39.482808+0000","last_became_active":"2024-09-21T06:31:38.504383+0000","last_became_peered":"2024-09-21T06:31:38.504383+0000","last_unstale":"2024-09-21T06:31:39.482808+0000","last_undegraded":"2024-09-21T06:31:39.482808+0000","last_fullsized":"2024-09-21T06:31:39.482808+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:34.414928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.567913+0000","last_change":"2024-09-21T06:31:37.485399+0000","last_active":"2024-09-21T06:31:39.567913+0000","last_peered":"2024-09-21T06:31:39.567913+0000","last_clean":"2024-09-21T06:31:39.567913+0000","last_became_active":"2024-09-21T06:31:37.485049+0000","last_became_peered":"2024-09-21T06:31:37.485049+0000","last_unstale":"2024-09-21T06:31:39.567913+0000","last_undegraded":"2024-09-21T06:31:39.567913+0000","last_fullsized":"2024-09-21T06:31:39.567913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:53.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956716+0000","last_change":"2024-09-21T06:31:38.487072+0000","last_active":"2024-09-21T06:31:39.956716+0000","last_peered":"2024-09-21T06:31:39.956716+0000","last_clean":"2024-09-21T06:31:39.956716+0000","last_became_active":"2024-09-21T06:31:38.486627+0000","last_became_peered":"2024-09-21T06:31:38.486627+0000","last_unstale":"2024-09-21T06:31:39.956716+0000","last_undegraded":"2024-09-21T06:31:39.956716+0000","last_fullsized":"2024-09-21T06:31:39.956716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:39.025682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568436+0000","last_change":"2024-09-21T06:31:37.480126+0000","last_active":"2024-09-21T06:31:39.568436+0000","last_peered":"2024-09-21T06:31:39.568436+0000","last_clean":"2024-09-21T06:31:39.568436+0000","last_became_active":"2024-09-21T06:31:37.479811+0000","last_became_peered":"2024-09-21T06:31:37.479811+0000","last_unstale":"2024-09-21T06:31:39.568436+0000","last_undegraded":"2024-09-21T06:31:39.568436+0000","last_fullsized":"2024-09-21T06:31:39.568436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:48.959159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.482545+0000","last_change":"2024-09-21T06:31:38.475405+0000","last_active":"2024-09-21T06:31:39.482545+0000","last_peered":"2024-09-21T06:31:39.482545+0000","last_clean":"2024-09-21T06:31:39.482545+0000","last_became_active":"2024-09-21T06:31:38.475041+0000","last_became_peered":"2024-09-21T06:31:38.475041+0000","last_unstale":"2024-09-21T06:31:39.482545+0000","last_undegraded":"2024-09-21T06:31:39.482545+0000","last_fullsized":"2024-09-21T06:31:39.482545+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:05.109325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957260+0000","last_change":"2024-09-21T06:31:37.473471+0000","last_active":"2024-09-21T06:31:39.957260+0000","last_peered":"2024-09-21T06:31:39.957260+0000","last_clean":"2024-09-21T06:31:39.957260+0000","last_became_active":"2024-09-21T06:31:37.473163+0000","last_became_peered":"2024-09-21T06:31:37.473163+0000","last_unstale":"2024-09-21T06:31:39.957260+0000","last_undegraded":"2024-09-21T06:31:39.957260+0000","last_fullsized":"2024-09-21T06:31:39.957260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.985457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956948+0000","last_change":"2024-09-21T06:31:38.505671+0000","last_active":"2024-09-21T06:31:39.956948+0000","last_peered":"2024-09-21T06:31:39.956948+0000","last_clean":"2024-09-21T06:31:39.956948+0000","last_became_active":"2024-09-21T06:31:38.503231+0000","last_became_peered":"2024-09-21T06:31:38.503231+0000","last_unstale":"2024-09-21T06:31:39.956948+0000","last_undegraded":"2024-09-21T06:31:39.956948+0000","last_fullsized":"2024-09-21T06:31:39.956948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:53.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957315+0000","last_change":"2024-09-21T06:31:37.479615+0000","last_active":"2024-09-21T06:31:39.957315+0000","last_peered":"2024-09-21T06:31:39.957315+0000","last_clean":"2024-09-21T06:31:39.957315+0000","last_became_active":"2024-09-21T06:31:37.479274+0000","last_became_peered":"2024-09-21T06:31:37.479274+0000","last_unstale":"2024-09-21T06:31:39.957315+0000","last_undegraded":"2024-09-21T06:31:39.957315+0000","last_fullsized":"2024-09-21T06:31:39.957315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:27.796590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957207+0000","last_change":"2024-09-21T06:31:38.479347+0000","last_active":"2024-09-21T06:31:39.957207+0000","last_peered":"2024-09-21T06:31:39.957207+0000","last_clean":"2024-09-21T06:31:39.957207+0000","last_became_active":"2024-09-21T06:31:38.479039+0000","last_became_peered":"2024-09-21T06:31:38.479039+0000","last_unstale":"2024-09-21T06:31:39.957207+0000","last_undegraded":"2024-09-21T06:31:39.957207+0000","last_fullsized":"2024-09-21T06:31:39.957207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:00.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":21,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484586+0000","last_change":"2024-09-21T06:31:37.485790+0000","last_active":"2024-09-21T06:31:39.484586+0000","last_peered":"2024-09-21T06:31:39.484586+0000","last_clean":"2024-09-21T06:31:39.484586+0000","last_became_active":"2024-09-21T06:31:37.485458+0000","last_became_peered":"2024-09-21T06:31:37.485458+0000","last_unstale":"2024-09-21T06:31:39.484586+0000","last_undegraded":"2024-09-21T06:31:39.484586+0000","last_fullsized":"2024-09-21T06:31:39.484586+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:57.211471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479548+0000","last_change":"2024-09-21T06:31:38.493308+0000","last_active":"2024-09-21T06:31:39.479548+0000","last_peered":"2024-09-21T06:31:39.479548+0000","last_clean":"2024-09-21T06:31:39.479548+0000","last_became_active":"2024-09-21T06:31:38.493006+0000","last_became_peered":"2024-09-21T06:31:38.493006+0000","last_unstale":"2024-09-21T06:31:39.479548+0000","last_undegraded":"2024-09-21T06:31:39.479548+0000","last_fullsized":"2024-09-21T06:31:39.479548+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:09.427469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568615+0000","last_change":"2024-09-21T06:31:37.481783+0000","last_active":"2024-09-21T06:31:39.568615+0000","last_peered":"2024-09-21T06:31:39.568615+0000","last_clean":"2024-09-21T06:31:39.568615+0000","last_became_active":"2024-09-21T06:31:37.480964+0000","last_became_peered":"2024-09-21T06:31:37.480964+0000","last_unstale":"2024-09-21T06:31:39.568615+0000","last_undegraded":"2024-09-21T06:31:39.568615+0000","last_fullsized":"2024-09-21T06:31:39.568615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:28.076578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479778+0000","last_change":"2024-09-21T06:31:38.487708+0000","last_active":"2024-09-21T06:31:39.479778+0000","last_peered":"2024-09-21T06:31:39.479778+0000","last_clean":"2024-09-21T06:31:39.479778+0000","last_became_active":"2024-09-21T06:31:38.487401+0000","last_became_peered":"2024-09-21T06:31:38.487401+0000","last_unstale":"2024-09-21T06:31:39.479778+0000","last_undegraded":"2024-09-21T06:31:39.479778+0000","last_fullsized":"2024-09-21T06:31:39.479778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:57.685079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479759+0000","last_change":"2024-09-21T06:31:37.480933+0000","last_active":"2024-09-21T06:31:39.479759+0000","last_peered":"2024-09-21T06:31:39.479759+0000","last_clean":"2024-09-21T06:31:39.479759+0000","last_became_active":"2024-09-21T06:31:37.480357+0000","last_became_peered":"2024-09-21T06:31:37.480357+0000","last_unstale":"2024-09-21T06:31:39.479759+0000","last_undegraded":"2024-09-21T06:31:39.479759+0000","last_fullsized":"2024-09-21T06:31:39.479759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:52.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479825+0000","last_change":"2024-09-21T06:31:38.482798+0000","last_active":"2024-09-21T06:31:39.479825+0000","last_peered":"2024-09-21T06:31:39.479825+0000","last_clean":"2024-09-21T06:31:39.479825+0000","last_became_active":"2024-09-21T06:31:38.482349+0000","last_became_peered":"2024-09-21T06:31:38.482349+0000","last_unstale":"2024-09-21T06:31:39.479825+0000","last_undegraded":"2024-09-21T06:31:39.479825+0000","last_fullsized":"2024-09-21T06:31:39.479825+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:04.395609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956722+0000","last_change":"2024-09-21T06:31:37.477963+0000","last_active":"2024-09-21T06:31:39.956722+0000","last_peered":"2024-09-21T06:31:39.956722+0000","last_clean":"2024-09-21T06:31:39.956722+0000","last_became_active":"2024-09-21T06:31:37.477279+0000","last_became_peered":"2024-09-21T06:31:37.477279+0000","last_unstale":"2024-09-21T06:31:39.956722+0000","last_undegraded":"2024-09-21T06:31:39.956722+0000","last_fullsized":"2024-09-21T06:31:39.956722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:18.671948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956485+0000","last_change":"2024-09-21T06:31:38.489868+0000","last_active":"2024-09-21T06:31:39.956485+0000","last_peered":"2024-09-21T06:31:39.956485+0000","last_clean":"2024-09-21T06:31:39.956485+0000","last_became_active":"2024-09-21T06:31:38.488005+0000","last_became_peered":"2024-09-21T06:31:38.488005+0000","last_unstale":"2024-09-21T06:31:39.956485+0000","last_undegraded":"2024-09-21T06:31:39.956485+0000","last_fullsized":"2024-09-21T06:31:39.956485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:29.080878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":23,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479472+0000","last_change":"2024-09-21T06:31:37.484364+0000","last_active":"2024-09-21T06:31:39.479472+0000","last_peered":"2024-09-21T06:31:39.479472+0000","last_clean":"2024-09-21T06:31:39.479472+0000","last_became_active":"2024-09-21T06:31:37.483142+0000","last_became_peered":"2024-09-21T06:31:37.483142+0000","last_unstale":"2024-09-21T06:31:39.479472+0000","last_undegraded":"2024-09-21T06:31:39.479472+0000","last_fullsized":"2024-09-21T06:31:39.479472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:04.005866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568666+0000","last_change":"2024-09-21T06:31:38.497868+0000","last_active":"2024-09-21T06:31:39.568666+0000","last_peered":"2024-09-21T06:31:39.568666+0000","last_clean":"2024-09-21T06:31:39.568666+0000","last_became_active":"2024-09-21T06:31:38.497507+0000","last_became_peered":"2024-09-21T06:31:38.497507+0000","last_unstale":"2024-09-21T06:31:39.568666+0000","last_undegraded":"2024-09-21T06:31:39.568666+0000","last_fullsized":"2024-09-21T06:31:39.568666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:51.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568571+0000","last_change":"2024-09-21T06:31:37.480329+0000","last_active":"2024-09-21T06:31:39.568571+0000","last_peered":"2024-09-21T06:31:39.568571+0000","last_clean":"2024-09-21T06:31:39.568571+0000","last_became_active":"2024-09-21T06:31:37.479978+0000","last_became_peered":"2024-09-21T06:31:37.479978+0000","last_unstale":"2024-09-21T06:31:39.568571+0000","last_undegraded":"2024-09-21T06:31:39.568571+0000","last_fullsized":"2024-09-21T06:31:39.568571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:43.285916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956898+0000","last_change":"2024-09-21T06:31:38.492730+0000","last_active":"2024-09-21T06:31:39.956898+0000","last_peered":"2024-09-21T06:31:39.956898+0000","last_clean":"2024-09-21T06:31:39.956898+0000","last_became_active":"2024-09-21T06:31:38.488944+0000","last_became_peered":"2024-09-21T06:31:38.488944+0000","last_unstale":"2024-09-21T06:31:39.956898+0000","last_undegraded":"2024-09-21T06:31:39.956898+0000","last_fullsized":"2024-09-21T06:31:39.956898+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.091083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568141+0000","last_change":"2024-09-21T06:31:37.483409+0000","last_active":"2024-09-21T06:31:39.568141+0000","last_peered":"2024-09-21T06:31:39.568141+0000","last_clean":"2024-09-21T06:31:39.568141+0000","last_became_active":"2024-09-21T06:31:37.482937+0000","last_became_peered":"2024-09-21T06:31:37.482937+0000","last_unstale":"2024-09-21T06:31:39.568141+0000","last_undegraded":"2024-09-21T06:31:39.568141+0000","last_fullsized":"2024-09-21T06:31:39.568141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:25.557582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957263+0000","last_change":"2024-09-21T06:31:38.488482+0000","last_active":"2024-09-21T06:31:39.957263+0000","last_peered":"2024-09-21T06:31:39.957263+0000","last_clean":"2024-09-21T06:31:39.957263+0000","last_became_active":"2024-09-21T06:31:38.488120+0000","last_became_peered":"2024-09-21T06:31:38.488120+0000","last_unstale":"2024-09-21T06:31:39.957263+0000","last_undegraded":"2024-09-21T06:31:39.957263+0000","last_fullsized":"2024-09-21T06:31:39.957263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:35.731851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957361+0000","last_change":"2024-09-21T06:31:37.488790+0000","last_active":"2024-09-21T06:31:39.957361+0000","last_peered":"2024-09-21T06:31:39.957361+0000","last_clean":"2024-09-21T06:31:39.957361+0000","last_became_active":"2024-09-21T06:31:37.486018+0000","last_became_peered":"2024-09-21T06:31:37.486018+0000","last_unstale":"2024-09-21T06:31:39.957361+0000","last_undegraded":"2024-09-21T06:31:39.957361+0000","last_fullsized":"2024-09-21T06:31:39.957361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:58.003727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957476+0000","last_change":"2024-09-21T06:31:38.490674+0000","last_active":"2024-09-21T06:31:39.957476+0000","last_peered":"2024-09-21T06:31:39.957476+0000","last_clean":"2024-09-21T06:31:39.957476+0000","last_became_active":"2024-09-21T06:31:38.490385+0000","last_became_peered":"2024-09-21T06:31:38.490385+0000","last_unstale":"2024-09-21T06:31:39.957476+0000","last_undegraded":"2024-09-21T06:31:39.957476+0000","last_fullsized":"2024-09-21T06:31:39.957476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.037177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957342+0000","last_change":"2024-09-21T06:31:37.489892+0000","last_active":"2024-09-21T06:31:39.957342+0000","last_peered":"2024-09-21T06:31:39.957342+0000","last_clean":"2024-09-21T06:31:39.957342+0000","last_became_active":"2024-09-21T06:31:37.489477+0000","last_became_peered":"2024-09-21T06:31:37.489477+0000","last_unstale":"2024-09-21T06:31:39.957342+0000","last_undegraded":"2024-09-21T06:31:39.957342+0000","last_fullsized":"2024-09-21T06:31:39.957342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:04.005866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484209+0000","last_change":"2024-09-21T06:31:38.478388+0000","last_active":"2024-09-21T06:31:39.484209+0000","last_peered":"2024-09-21T06:31:39.484209+0000","last_clean":"2024-09-21T06:31:39.484209+0000","last_became_active":"2024-09-21T06:31:38.478068+0000","last_became_peered":"2024-09-21T06:31:38.478068+0000","last_unstale":"2024-09-21T06:31:39.484209+0000","last_undegraded":"2024-09-21T06:31:39.484209+0000","last_fullsized":"2024-09-21T06:31:39.484209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:56.126853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484563+0000","last_change":"2024-09-21T06:31:37.479881+0000","last_active":"2024-09-21T06:31:39.484563+0000","last_peered":"2024-09-21T06:31:39.484563+0000","last_clean":"2024-09-21T06:31:39.484563+0000","last_became_active":"2024-09-21T06:31:37.479532+0000","last_became_peered":"2024-09-21T06:31:37.479532+0000","last_unstale":"2024-09-21T06:31:39.484563+0000","last_undegraded":"2024-09-21T06:31:39.484563+0000","last_fullsized":"2024-09-21T06:31:39.484563+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:12.733895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956260+0000","last_change":"2024-09-21T06:31:38.475938+0000","last_active":"2024-09-21T06:31:39.956260+0000","last_peered":"2024-09-21T06:31:39.956260+0000","last_clean":"2024-09-21T06:31:39.956260+0000","last_became_active":"2024-09-21T06:31:38.475185+0000","last_became_peered":"2024-09-21T06:31:38.475185+0000","last_unstale":"2024-09-21T06:31:39.956260+0000","last_undegraded":"2024-09-21T06:31:39.956260+0000","last_fullsized":"2024-09-21T06:31:39.956260+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:38.219567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.480119+0000","last_change":"2024-09-21T06:31:37.490781+0000","last_active":"2024-09-21T06:31:39.480119+0000","last_peered":"2024-09-21T06:31:39.480119+0000","last_clean":"2024-09-21T06:31:39.480119+0000","last_became_active":"2024-09-21T06:31:37.489848+0000","last_became_peered":"2024-09-21T06:31:37.489848+0000","last_unstale":"2024-09-21T06:31:39.480119+0000","last_undegraded":"2024-09-21T06:31:39.480119+0000","last_fullsized":"2024-09-21T06:31:39.480119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:53.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.567912+0000","last_change":"2024-09-21T06:31:38.492468+0000","last_active":"2024-09-21T06:31:39.567912+0000","last_peered":"2024-09-21T06:31:39.567912+0000","last_clean":"2024-09-21T06:31:39.567912+0000","last_became_active":"2024-09-21T06:31:38.491282+0000","last_became_peered":"2024-09-21T06:31:38.491282+0000","last_unstale":"2024-09-21T06:31:39.567912+0000","last_undegraded":"2024-09-21T06:31:39.567912+0000","last_fullsized":"2024-09-21T06:31:39.567912+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:47.024535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483934+0000","last_change":"2024-09-21T06:31:37.482866+0000","last_active":"2024-09-21T06:31:39.483934+0000","last_peered":"2024-09-21T06:31:39.483934+0000","last_clean":"2024-09-21T06:31:39.483934+0000","last_became_active":"2024-09-21T06:31:37.482450+0000","last_became_peered":"2024-09-21T06:31:37.482450+0000","last_unstale":"2024-09-21T06:31:39.483934+0000","last_undegraded":"2024-09-21T06:31:39.483934+0000","last_fullsized":"2024-09-21T06:31:39.483934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:35.577652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483484+0000","last_change":"2024-09-21T06:31:38.475605+0000","last_active":"2024-09-21T06:31:39.483484+0000","last_peered":"2024-09-21T06:31:39.483484+0000","last_clean":"2024-09-21T06:31:39.483484+0000","last_became_active":"2024-09-21T06:31:38.472819+0000","last_became_peered":"2024-09-21T06:31:38.472819+0000","last_unstale":"2024-09-21T06:31:39.483484+0000","last_undegraded":"2024-09-21T06:31:39.483484+0000","last_fullsized":"2024-09-21T06:31:39.483484+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:07.560927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568596+0000","last_change":"2024-09-21T06:31:37.487752+0000","last_active":"2024-09-21T06:31:39.568596+0000","last_peered":"2024-09-21T06:31:39.568596+0000","last_clean":"2024-09-21T06:31:39.568596+0000","last_became_active":"2024-09-21T06:31:37.487280+0000","last_became_peered":"2024-09-21T06:31:37.487280+0000","last_unstale":"2024-09-21T06:31:39.568596+0000","last_undegraded":"2024-09-21T06:31:39.568596+0000","last_fullsized":"2024-09-21T06:31:39.568596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:58.762266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.482810+0000","last_change":"2024-09-21T06:31:38.482257+0000","last_active":"2024-09-21T06:31:39.482810+0000","last_peered":"2024-09-21T06:31:39.482810+0000","last_clean":"2024-09-21T06:31:39.482810+0000","last_became_active":"2024-09-21T06:31:38.481827+0000","last_became_peered":"2024-09-21T06:31:38.481827+0000","last_unstale":"2024-09-21T06:31:39.482810+0000","last_undegraded":"2024-09-21T06:31:39.482810+0000","last_fullsized":"2024-09-21T06:31:39.482810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:17.984513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.568443+0000","last_change":"2024-09-21T06:31:37.480490+0000","last_active":"2024-09-21T06:31:39.568443+0000","last_peered":"2024-09-21T06:31:39.568443+0000","last_clean":"2024-09-21T06:31:39.568443+0000","last_became_active":"2024-09-21T06:31:37.479181+0000","last_became_peered":"2024-09-21T06:31:37.479181+0000","last_unstale":"2024-09-21T06:31:39.568443+0000","last_undegraded":"2024-09-21T06:31:39.568443+0000","last_fullsized":"2024-09-21T06:31:39.568443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:12.748387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957480+0000","last_change":"2024-09-21T06:31:38.487055+0000","last_active":"2024-09-21T06:31:39.957480+0000","last_peered":"2024-09-21T06:31:39.957480+0000","last_clean":"2024-09-21T06:31:39.957480+0000","last_became_active":"2024-09-21T06:31:38.486347+0000","last_became_peered":"2024-09-21T06:31:38.486347+0000","last_unstale":"2024-09-21T06:31:39.957480+0000","last_undegraded":"2024-09-21T06:31:39.957480+0000","last_fullsized":"2024-09-21T06:31:39.957480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:38.329289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484239+0000","last_change":"2024-09-21T06:31:37.478225+0000","last_active":"2024-09-21T06:31:39.484239+0000","last_peered":"2024-09-21T06:31:39.484239+0000","last_clean":"2024-09-21T06:31:39.484239+0000","last_became_active":"2024-09-21T06:31:37.477900+0000","last_became_peered":"2024-09-21T06:31:37.477900+0000","last_unstale":"2024-09-21T06:31:39.484239+0000","last_undegraded":"2024-09-21T06:31:39.484239+0000","last_fullsized":"2024-09-21T06:31:39.484239+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:18.545420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957605+0000","last_change":"2024-09-21T06:31:38.490507+0000","last_active":"2024-09-21T06:31:39.957605+0000","last_peered":"2024-09-21T06:31:39.957605+0000","last_clean":"2024-09-21T06:31:39.957605+0000","last_became_active":"2024-09-21T06:31:38.490120+0000","last_became_peered":"2024-09-21T06:31:38.490120+0000","last_unstale":"2024-09-21T06:31:39.957605+0000","last_undegraded":"2024-09-21T06:31:39.957605+0000","last_fullsized":"2024-09-21T06:31:39.957605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:33.969447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957398+0000","last_change":"2024-09-21T06:31:37.487862+0000","last_active":"2024-09-21T06:31:39.957398+0000","last_peered":"2024-09-21T06:31:39.957398+0000","last_clean":"2024-09-21T06:31:39.957398+0000","last_became_active":"2024-09-21T06:31:37.487545+0000","last_became_peered":"2024-09-21T06:31:37.487545+0000","last_unstale":"2024-09-21T06:31:39.957398+0000","last_undegraded":"2024-09-21T06:31:39.957398+0000","last_fullsized":"2024-09-21T06:31:39.957398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:53.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.567843+0000","last_change":"2024-09-21T06:31:38.492329+0000","last_active":"2024-09-21T06:31:39.567843+0000","last_peered":"2024-09-21T06:31:39.567843+0000","last_clean":"2024-09-21T06:31:39.567843+0000","last_became_active":"2024-09-21T06:31:38.489596+0000","last_became_peered":"2024-09-21T06:31:38.489596+0000","last_unstale":"2024-09-21T06:31:39.567843+0000","last_undegraded":"2024-09-21T06:31:39.567843+0000","last_fullsized":"2024-09-21T06:31:39.567843+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:51.634149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956807+0000","last_change":"2024-09-21T06:31:37.471427+0000","last_active":"2024-09-21T06:31:39.956807+0000","last_peered":"2024-09-21T06:31:39.956807+0000","last_clean":"2024-09-21T06:31:39.956807+0000","last_became_active":"2024-09-21T06:31:37.470772+0000","last_became_peered":"2024-09-21T06:31:37.470772+0000","last_unstale":"2024-09-21T06:31:39.956807+0000","last_undegraded":"2024-09-21T06:31:39.956807+0000","last_fullsized":"2024-09-21T06:31:39.956807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:01.558450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956260+0000","last_change":"2024-09-21T06:31:38.494323+0000","last_active":"2024-09-21T06:31:39.956260+0000","last_peered":"2024-09-21T06:31:39.956260+0000","last_clean":"2024-09-21T06:31:39.956260+0000","last_became_active":"2024-09-21T06:31:38.493979+0000","last_became_peered":"2024-09-21T06:31:38.493979+0000","last_unstale":"2024-09-21T06:31:39.956260+0000","last_undegraded":"2024-09-21T06:31:39.956260+0000","last_fullsized":"2024-09-21T06:31:39.956260+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:43.065600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.956474+0000","last_change":"2024-09-21T06:31:37.477853+0000","last_active":"2024-09-21T06:31:39.956474+0000","last_peered":"2024-09-21T06:31:39.956474+0000","last_clean":"2024-09-21T06:31:39.956474+0000","last_became_active":"2024-09-21T06:31:37.477546+0000","last_became_peered":"2024-09-21T06:31:37.477546+0000","last_unstale":"2024-09-21T06:31:39.956474+0000","last_undegraded":"2024-09-21T06:31:39.956474+0000","last_fullsized":"2024-09-21T06:31:39.956474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:47.382301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484825+0000","last_change":"2024-09-21T06:31:38.492957+0000","last_active":"2024-09-21T06:31:39.484825+0000","last_peered":"2024-09-21T06:31:39.484825+0000","last_clean":"2024-09-21T06:31:39.484825+0000","last_became_active":"2024-09-21T06:31:38.492025+0000","last_became_peered":"2024-09-21T06:31:38.492025+0000","last_unstale":"2024-09-21T06:31:39.484825+0000","last_undegraded":"2024-09-21T06:31:39.484825+0000","last_fullsized":"2024-09-21T06:31:39.484825+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:16.306533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.957314+0000","last_change":"2024-09-21T06:31:37.485763+0000","last_active":"2024-09-21T06:31:39.957314+0000","last_peered":"2024-09-21T06:31:39.957314+0000","last_clean":"2024-09-21T06:31:39.957314+0000","last_became_active":"2024-09-21T06:31:37.485087+0000","last_became_peered":"2024-09-21T06:31:37.485087+0000","last_unstale":"2024-09-21T06:31:39.957314+0000","last_undegraded":"2024-09-21T06:31:39.957314+0000","last_fullsized":"2024-09-21T06:31:39.957314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:52.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.483077+0000","last_change":"2024-09-21T06:31:38.494706+0000","last_active":"2024-09-21T06:31:39.483077+0000","last_peered":"2024-09-21T06:31:39.483077+0000","last_clean":"2024-09-21T06:31:39.483077+0000","last_became_active":"2024-09-21T06:31:38.494380+0000","last_became_peered":"2024-09-21T06:31:38.494380+0000","last_unstale":"2024-09-21T06:31:39.483077+0000","last_undegraded":"2024-09-21T06:31:39.483077+0000","last_fullsized":"2024-09-21T06:31:39.483077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:06.450161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484054+0000","last_change":"2024-09-21T06:31:37.478552+0000","last_active":"2024-09-21T06:31:39.484054+0000","last_peered":"2024-09-21T06:31:39.484054+0000","last_clean":"2024-09-21T06:31:39.484054+0000","last_became_active":"2024-09-21T06:31:37.476760+0000","last_became_peered":"2024-09-21T06:31:37.476760+0000","last_unstale":"2024-09-21T06:31:39.484054+0000","last_undegraded":"2024-09-21T06:31:39.484054+0000","last_fullsized":"2024-09-21T06:31:39.484054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:59.379517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479848+0000","last_change":"2024-09-21T06:31:38.484889+0000","last_active":"2024-09-21T06:31:39.479848+0000","last_peered":"2024-09-21T06:31:39.479848+0000","last_clean":"2024-09-21T06:31:39.479848+0000","last_became_active":"2024-09-21T06:31:38.484570+0000","last_became_peered":"2024-09-21T06:31:38.484570+0000","last_unstale":"2024-09-21T06:31:39.479848+0000","last_undegraded":"2024-09-21T06:31:39.479848+0000","last_fullsized":"2024-09-21T06:31:39.479848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:37.448823+0000","last_clean_scrub_stamp":"2024-09-21T06:31:37.448823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:12.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.484185+0000","last_change":"2024-09-21T06:31:37.486831+0000","last_active":"2024-09-21T06:31:39.484185+0000","last_peered":"2024-09-21T06:31:39.484185+0000","last_clean":"2024-09-21T06:31:39.484185+0000","last_became_active":"2024-09-21T06:31:37.486531+0000","last_became_peered":"2024-09-21T06:31:37.486531+0000","last_unstale":"2024-09-21T06:31:39.484185+0000","last_undegraded":"2024-09-21T06:31:39.484185+0000","last_fullsized":"2024-09-21T06:31:39.484185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:31:36.444523+0000","last_clean_scrub_stamp":"2024-09-21T06:31:36.444523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:18.361705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":194,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T06:31:39.479606+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:31:39.479606+0000","last_peered":"2024-09-21T06:31:39.479606+0000","last_clean":"2024-09-21T06:31:39.479606+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:31:39.479606+0000","last_undegraded":"2024-09-21T06:31:39.479606+0000","last_fullsized":"2024-09-21T06:31:39.479606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":362361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":364625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":362943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":811008,"data_stored":362521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":811008,"data_stored":362417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1400832,"data_stored":952763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":362361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1404928,"data_stored":947761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:31:43.482 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:31:43.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:31:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 -- 172.21.15.37:0/2120137079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff180074cb0 msgr2=0x7ff180073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 --2- 172.21.15.37:0/2120137079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff180074cb0 0x7ff180073dd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff18015d980 tx=0x7ff17002a560 comp rx=0 tx=0).stop 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 -- 172.21.15.37:0/2120137079 shutdown_connections 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 --2- 172.21.15.37:0/2120137079 >> v2:172.21.15.150:3300/0 conn(0x7ff18006e500 0x7ff1800cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 --2- 172.21.15.37:0/2120137079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff180074cb0 0x7ff180073dd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 -- 172.21.15.37:0/2120137079 >> 172.21.15.37:0/2120137079 conn(0x7ff1800cca40 msgr2=0x7ff1800cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 -- 172.21.15.37:0/2120137079 shutdown_connections 2024-09-21T06:31:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.748+0000 7ff185200640 1 -- 172.21.15.37:0/2120137079 wait complete. 2024-09-21T06:31:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.750+0000 7ff185200640 1 Processor -- start 2024-09-21T06:31:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.750+0000 7ff185200640 1 -- start start 2024-09-21T06:31:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 0x7ff1801f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1800cbc10 0x7ff18009b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff18006f380 con 0x7ff1800cdb90 2024-09-21T06:31:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff18006f200 con 0x7ff1800cbc10 2024-09-21T06:31:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff185200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff18006f500 con 0x7ff180074cb0 2024-09-21T06:31:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff17e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff17e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34880/0 (socket says 172.21.15.37:34880) 2024-09-21T06:31:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff17e800640 1 -- 172.21.15.37:0/118529603 learned_addr learned my addr 172.21.15.37:0/118529603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.752+0000 7ff175400640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1800cbc10 0x7ff18009b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17de00640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 0x7ff1801f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 -- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 msgr2=0x7ff1801f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 0x7ff1801f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 -- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1800cbc10 msgr2=0x7ff18009b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1800cbc10 0x7ff18009b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 -- 172.21.15.37:0/118529603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1801f6200 con 0x7ff1800cdb90 2024-09-21T06:31:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17de00640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 0x7ff1801f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff17e800640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff17800ba10 tx=0x7ff17800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178019070 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff17800cc50 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1800ce300 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1780146a0 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.753+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1800ce8c0 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.754+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff180072860 con 0x7ff1800cdb90 2024-09-21T06:31:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.756+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff17801c070 con 0x7ff1800cdb90 2024-09-21T06:31:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.757+0000 7ff176a00640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.757+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff17809ab10 con 0x7ff1800cdb90 2024-09-21T06:31:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.757+0000 7ff17de00640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.758+0000 7ff17de00640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 0x7ff15c07b0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff170033000 tx=0x7ff170004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:43.761+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff180072860 con 0x7ff1800cdb90 2024-09-21T06:31:44.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.041+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff18006fa30 con 0x7ff1800cdb90 2024-09-21T06:31:44.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.044+0000 7ff176a00640 1 -- 172.21.15.37:0/118529603 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7ff1801f6200 con 0x7ff1800cdb90 2024-09-21T06:31:44.046 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:31:44.046 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:31:44.046 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:31:44.047 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:31:44.047 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:31:44.047 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 msgr2=0x7ff15c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 0x7ff15c07b0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff170033000 tx=0x7ff170004730 comp rx=0 tx=0).stop 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 msgr2=0x7ff18009b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff17800ba10 tx=0x7ff17800bed0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 shutdown_connections 2024-09-21T06:31:44.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1800cdb90 0x7ff18009b840 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1800cbc10 0x7ff18009b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 --2- 172.21.15.37:0/118529603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff180074cb0 0x7ff1801f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 >> 172.21.15.37:0/118529603 conn(0x7ff1800cca40 msgr2=0x7ff18015b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 shutdown_connections 2024-09-21T06:31:44.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.047+0000 7ff185200640 1 -- 172.21.15.37:0/118529603 wait complete. 2024-09-21T06:31:44.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 -- 172.21.15.37:0/2266568847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 msgr2=0x7f0130073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 --2- 172.21.15.37:0/2266568847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f0130073dd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0130076390 tx=0x7f012402a560 comp rx=0 tx=0).stop 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 -- 172.21.15.37:0/2266568847 shutdown_connections 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 --2- 172.21.15.37:0/2266568847 >> v2:172.21.15.150:3300/0 conn(0x7f013006ddf0 0x7f013006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 --2- 172.21.15.37:0/2266568847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01300cbc10 0x7f01300cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 --2- 172.21.15.37:0/2266568847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f0130073dd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 -- 172.21.15.37:0/2266568847 >> 172.21.15.37:0/2266568847 conn(0x7f01300cca40 msgr2=0x7f01300cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 -- 172.21.15.37:0/2266568847 shutdown_connections 2024-09-21T06:31:44.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.296+0000 7f0137600640 1 -- 172.21.15.37:0/2266568847 wait complete. 2024-09-21T06:31:44.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.299+0000 7f0137600640 1 Processor -- start 2024-09-21T06:31:44.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.299+0000 7f0137600640 1 -- start start 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f013006ddf0 0x7f0130155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01300cbc10 0x7f0130157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f013006f170 con 0x7f0130074cb0 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f013006eff0 con 0x7f01300cbc10 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.300+0000 7f0137600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f013006f2f0 con 0x7f013006ddf0 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55924/0 (socket says 172.21.15.37:55924) 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 -- 172.21.15.37:0/3442419356 learned_addr learned my addr 172.21.15.37:0/3442419356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f0134c00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f013006ddf0 0x7f0130155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f0135600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01300cbc10 0x7f0130157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 -- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f013006ddf0 msgr2=0x7f0130155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f013006ddf0 0x7f0130155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 -- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01300cbc10 msgr2=0x7f0130157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01300cbc10 0x7f0130157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 -- 172.21.15.37:0/3442419356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301561d0 con 0x7f0130074cb0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.301+0000 7f012fe00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c9a0 tx=0x7f011c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.302+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c007bb0 con 0x7f0130074cb0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.302+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f011c007e40 con 0x7f0130074cb0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.302+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c00f690 con 0x7f0130074cb0 2024-09-21T06:31:44.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.302+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01300b38c0 con 0x7f0130074cb0 2024-09-21T06:31:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.302+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01300b3e80 con 0x7f0130074cb0 2024-09-21T06:31:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.303+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130073dd0 con 0x7f0130074cb0 2024-09-21T06:31:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.304+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f011c00f830 con 0x7f0130074cb0 2024-09-21T06:31:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.305+0000 7f012cc00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 0x7f011007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.305+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f011c09b7d0 con 0x7f0130074cb0 2024-09-21T06:31:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.306+0000 7f0134c00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 0x7f011007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.307+0000 7f0134c00640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 0x7f011007b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0124033000 tx=0x7f0124004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.310+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f011c014030 con 0x7f0130074cb0 2024-09-21T06:31:44.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.545+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0130155f60 con 0x7f0130074cb0 2024-09-21T06:31:44.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.545+0000 7f012cc00640 1 -- 172.21.15.37:0/3442419356 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5230 (secure 0 0 0) 0x7f011c0639e0 con 0x7f0130074cb0 2024-09-21T06:31:44.548 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"btime":"2024-09-21T06:31:41:488546+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:31:41.005513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":4}]} 2024-09-21T06:31:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-09-21T06:31:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 msgr2=0x7f011007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 0x7f011007b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0124033000 tx=0x7f0124004730 comp rx=0 tx=0).stop 2024-09-21T06:31:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 msgr2=0x7f01300b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c9a0 tx=0x7f011c00ce60 comp rx=0 tx=0).stop 2024-09-21T06:31:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 shutdown_connections 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0110078b90 0x7f011007b0a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0130074cb0 0x7f01300b0fc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01300cbc10 0x7f0130157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 --2- 172.21.15.37:0/3442419356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f013006ddf0 0x7f0130155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 >> 172.21.15.37:0/3442419356 conn(0x7f01300cca40 msgr2=0x7f0130070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 shutdown_connections 2024-09-21T06:31:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:44.548+0000 7f0137600640 1 -- 172.21.15.37:0/3442419356 wait complete. 2024-09-21T06:31:44.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:31:37.988979+0000', 'modified': '2024-09-21T06:31:41.005513+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6106}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6106': {'gid': 6106, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1569973003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1569973003}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1569973003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6106, 'qdb_cluster': [6106]} 2024-09-21T06:31:44.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:31:44.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.202+0000 7f9538600640 1 -- 172.21.15.37:0/1743839479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 msgr2=0x7f953006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.202+0000 7f9538600640 1 --2- 172.21.15.37:0/1743839479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f953006e200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f951c005fd0 tx=0x7f951c02a670 comp rx=0 tx=0).stop 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.202+0000 7f9538600640 1 -- 172.21.15.37:0/1743839479 shutdown_connections 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.202+0000 7f9538600640 1 --2- 172.21.15.37:0/1743839479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f953006e200 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.202+0000 7f9538600640 1 -- 172.21.15.37:0/1743839479 >> 172.21.15.37:0/1743839479 conn(0x7f95300b1b90 msgr2=0x7f95300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.203+0000 7f9538600640 1 -- 172.21.15.37:0/1743839479 shutdown_connections 2024-09-21T06:31:45.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.203+0000 7f9538600640 1 -- 172.21.15.37:0/1743839479 wait complete. 2024-09-21T06:31:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.205+0000 7f9538600640 1 Processor -- start 2024-09-21T06:31:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.205+0000 7f9538600640 1 -- start start 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95300cef00 0x7f95300b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9530156a50 0x7f953015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9530160fd0 con 0x7f953006ddf0 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9530160e50 con 0x7f9530156a50 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.206+0000 7f9538600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f95300cfa20 con 0x7f95300cef00 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55936/0 (socket says 172.21.15.37:55936) 2024-09-21T06:31:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 -- 172.21.15.37:0/256405075 learned_addr learned my addr 172.21.15.37:0/256405075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535200640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95300cef00 0x7f95300b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9536600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9530156a50 0x7f953015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 -- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95300cef00 msgr2=0x7f95300b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95300cef00 0x7f95300b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 -- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9530156a50 msgr2=0x7f953015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9530156a50 0x7f953015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.207+0000 7f9535c00640 1 -- 172.21.15.37:0/256405075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95301f9d70 con 0x7f953006ddf0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f9535c00640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f952400c400 tx=0x7f952400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f952d600640 1 -- 172.21.15.37:0/256405075 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524017070 con 0x7f953006ddf0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f952d600640 1 -- 172.21.15.37:0/256405075 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95240053c0 con 0x7f953006ddf0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95301f8fb0 con 0x7f953006ddf0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f952d600640 1 -- 172.21.15.37:0/256405075 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524003850 con 0x7f953006ddf0 2024-09-21T06:31:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.208+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301f9570 con 0x7f953006ddf0 2024-09-21T06:31:45.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.212+0000 7f952d600640 1 -- 172.21.15.37:0/256405075 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f952401a070 con 0x7f953006ddf0 2024-09-21T06:31:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.212+0000 7f952d600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 0x7f951007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.213+0000 7f952d600640 1 -- 172.21.15.37:0/256405075 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f952409a2b0 con 0x7f953006ddf0 2024-09-21T06:31:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.213+0000 7f953f67f740 1 Processor -- start 2024-09-21T06:31:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.213+0000 7f953f67f740 1 -- start start 2024-09-21T06:31:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.213+0000 7f9535200640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 0x7f951007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f9535200640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 0x7f951007b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f951c033000 tx=0x7f951c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e975ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e972cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592e96b6430 con 0x5592e938c270 2024-09-21T06:31:45.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e96a65d0 con 0x5592e936b590 2024-09-21T06:31:45.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.214+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5592e9695a00 con 0x5592e93633b0 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55944/0 (socket says 172.21.15.37:55944) 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 -- 172.21.15.37:0/1091134766 learned_addr learned my addr 172.21.15.37:0/1091134766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535200640 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e975ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 -- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 msgr2=0x5592e975ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e975ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 -- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 msgr2=0x5592e972cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e972cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 -- 172.21.15.37:0/1091134766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e965e520 con 0x5592e938c270 2024-09-21T06:31:45.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9535c00640 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f95240010c0 tx=0x7f9524001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985de34616981b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.215+0000 7f9522000640 1 -- 172.21.15.37:0/1091134766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240668b0 con 0x5592e938c270 2024-09-21T06:31:45.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f9522000640 1 -- 172.21.15.37:0/1091134766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f952406b760 con 0x5592e938c270 2024-09-21T06:31:45.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f9522000640 1 -- 172.21.15.37:0/1091134766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240610e0 con 0x5592e938c270 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 -- 172.21.15.37:0/1091134766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 msgr2=0x5592e975c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f95240010c0 tx=0x7f9524001260 comp rx=0 tx=0).stop 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 -- 172.21.15.37:0/1091134766 shutdown_connections 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e972cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e975ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 --2- 172.21.15.37:0/1091134766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e938c270 0x5592e975c970 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.216+0000 7f953f67f740 1 -- 172.21.15.37:0/1091134766 >> 172.21.15.37:0/1091134766 conn(0x5592e93803c0 msgr2=0x5592e9732970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.217+0000 7f953f67f740 1 -- 172.21.15.37:0/1091134766 shutdown_connections 2024-09-21T06:31:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.217+0000 7f953f67f740 1 -- 172.21.15.37:0/1091134766 wait complete. 2024-09-21T06:31:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.218+0000 7f953f67f740 1 -- start start 2024-09-21T06:31:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.218+0000 7f953f67f740 1 Processor -- start 2024-09-21T06:31:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e9756b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e938c270 0x5592e9757090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592e963c330 con 0x5592e93633b0 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e9699810 con 0x5592e936b590 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5592e9406620 con 0x5592e938c270 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.219+0000 7f953f67f740 10 client.0 _ll_register_callbacks cb 0x5592e9641420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55958/0 (socket says 172.21.15.37:55958) 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 -- 172.21.15.37:0/332465307 learned_addr learned my addr 172.21.15.37:0/332465307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9536600640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e938c270 0x5592e9757090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535200640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e9756b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 -- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e938c270 msgr2=0x5592e9757090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e938c270 0x5592e9757090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 -- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 msgr2=0x5592e9756b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e9756b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 -- 172.21.15.37:0/332465307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e93f7850 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9535c00640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9524000f80 tx=0x7f952405e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6becd05d613cc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9516a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952407c060 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.220+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f9524084250 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f952406b760 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9524082050 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 10 client.6372 fetch_fsmap learned FSMap version 52 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5592e946e6b0 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f952406b760 con 0x5592e93633b0 2024-09-21T06:31:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f95240611d0 con 0x5592e93633b0 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 10 client.6372 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 10 client.6372 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 20 client.6372 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e934ae40 0x5592e976f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 4 client.6372 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.221+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e96a8520 con 0x5592e934ae40 2024-09-21T06:31:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.222+0000 7f9535200640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e934ae40 0x5592e976f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.222+0000 7f9535200640 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e934ae40 0x5592e976f1e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca71be40b968a9b server_cookie=f1291885bd73c12 in_seq=0 out_seq=0 2024-09-21T06:31:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.222+0000 7f9516a00640 10 client.6372 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f9516a00640 1 -- 172.21.15.37:0/332465307 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5592e96a8520 con 0x5592e934ae40 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f9516a00640 10 client.6372 handle_command_reply: tid=0 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 client.6372 shutdown 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 msgr2=0x5592e9752630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9524000f80 tx=0x7f952405e3a0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 shutdown_connections 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e934ae40 0x5592e976f1e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e938c270 0x5592e9757090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e936b590 0x5592e9756b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f9516a00640 0 client.6372 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 --2- 172.21.15.37:0/332465307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e93633b0 0x5592e9752630 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.223+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 >> 172.21.15.37:0/332465307 conn(0x5592e93803c0 msgr2=0x5592e97798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.224+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 shutdown_connections 2024-09-21T06:31:45.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.224+0000 7f953f67f740 1 -- 172.21.15.37:0/332465307 wait complete. 2024-09-21T06:31:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.224+0000 7f953f67f740 20 client.6372 trim_cache size 0 max 16384 2024-09-21T06:31:45.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.243+0000 7f953f67f740 1 Processor -- start 2024-09-21T06:31:45.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.243+0000 7f953f67f740 1 -- start start 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e9779bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e97712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95302017f0 con 0x5592e936b590 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f95300a0ba0 con 0x5592e938c270 2024-09-21T06:31:45.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.244+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f95302011a0 con 0x5592e93633b0 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55962/0 (socket says 172.21.15.37:55962) 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 -- 172.21.15.37:0/1891447126 learned_addr learned my addr 172.21.15.37:0/1891447126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535c00640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e9779bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9536600640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e97712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 -- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 msgr2=0x5592e9779bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e9779bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 -- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 msgr2=0x5592e97712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e97712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 -- 172.21.15.37:0/1891447126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e9754090 con 0x5592e936b590 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535c00640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e9779bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:45.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.245+0000 7f9535200640 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f951c0096c0 tx=0x7f951c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a110059ec1846db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f9521600640 1 -- 172.21.15.37:0/1891447126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c056070 con 0x5592e936b590 2024-09-21T06:31:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f9521600640 1 -- 172.21.15.37:0/1891447126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f951c0039c0 con 0x5592e936b590 2024-09-21T06:31:45.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f9521600640 1 -- 172.21.15.37:0/1891447126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c035040 con 0x5592e936b590 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 -- 172.21.15.37:0/1891447126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 msgr2=0x5592e9770db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f951c0096c0 tx=0x7f951c03d000 comp rx=0 tx=0).stop 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 -- 172.21.15.37:0/1891447126 shutdown_connections 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e9770db0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e97712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 --2- 172.21.15.37:0/1891447126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e9779bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.246+0000 7f953f67f740 1 -- 172.21.15.37:0/1891447126 >> 172.21.15.37:0/1891447126 conn(0x5592e93803c0 msgr2=0x5592e9786460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.247+0000 7f953f67f740 1 -- 172.21.15.37:0/1891447126 shutdown_connections 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.247+0000 7f953f67f740 1 -- 172.21.15.37:0/1891447126 wait complete. 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.247+0000 7f953f67f740 1 -- start start 2024-09-21T06:31:45.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.248+0000 7f953f67f740 1 Processor -- start 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e97588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e9728500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592e963c330 con 0x5592e936b590 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e9699810 con 0x5592e938c270 2024-09-21T06:31:45.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5592e9754b60 con 0x5592e93633b0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.249+0000 7f953f67f740 10 client.0 _ll_register_callbacks cb 0x7f9524061680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e97588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55978/0 (socket says 172.21.15.37:55978) 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 -- 172.21.15.37:0/1694973724 learned_addr learned my addr 172.21.15.37:0/1694973724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9536600640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e9728500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 -- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 msgr2=0x5592e97588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e97588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 -- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 msgr2=0x5592e9728500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e9728500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 -- 172.21.15.37:0/1694973724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e979b150 con 0x5592e936b590 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9535200640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55167752178ed194 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.250+0000 7f9516a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:45.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c056030 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f951c050050 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951c0039c0 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f951c003e20 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 10 client.6378 fetch_fsmap learned FSMap version 52 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5592e979b730 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f951c0039c0 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f951c0047a0 con 0x5592e936b590 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 10 client.6378 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 10 client.6378 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 20 client.6378 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e97904d0 0x5592e97908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 4 client.6378 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.251+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "status"}) -- 0x5592e96a3fd0 con 0x5592e97904d0 2024-09-21T06:31:45.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.252+0000 7f9535c00640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e97904d0 0x5592e97908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f9516a00640 10 client.6378 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f9535c00640 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e97904d0 0x5592e97908e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc65f1319fe8237d server_cookie=b4d7f728c9ea338b in_seq=0 out_seq=0 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f9516a00640 1 -- 172.21.15.37:0/1694973724 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5592e96a3fd0 con 0x5592e97904d0 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f9516a00640 10 client.6378 handle_command_reply: tid=0 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 client.6378 shutdown 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 msgr2=0x5592e97f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c003b70 comp rx=0 tx=0).stop 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 shutdown_connections 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x5592e97904d0 0x5592e97908e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592e936b590 0x5592e97f66c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e938c270 0x5592e9728500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 --2- 172.21.15.37:0/1694973724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5592e93633b0 0x5592e97588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f9516a00640 0 client.6378 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 >> 172.21.15.37:0/1694973724 conn(0x5592e93803c0 msgr2=0x5592e9770450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 shutdown_connections 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 1 -- 172.21.15.37:0/1694973724 wait complete. 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.253+0000 7f953f67f740 20 client.6378 trim_cache size 0 max 16384 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6106, 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:31:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 6.7944894549999999, 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 52, 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 67, 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 67, 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.767842183000001 2024-09-21T06:31:45.260 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 msgr2=0x7f951007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 0x7f951007b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f951c033000 tx=0x7f951c004730 comp rx=0 tx=0).stop 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 msgr2=0x7f95301571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f952400c400 tx=0x7f952400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 shutdown_connections 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9510078b90 0x7f951007b0a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9530156a50 0x7f953015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95300cef00 0x7f95300b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 --2- 172.21.15.37:0/256405075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f953006ddf0 0x7f95301571e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 >> 172.21.15.37:0/256405075 conn(0x7f95300b1b90 msgr2=0x7f95300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 shutdown_connections 2024-09-21T06:31:45.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.254+0000 7f9538600640 1 -- 172.21.15.37:0/256405075 wait complete. 2024-09-21T06:31:45.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.508+0000 7f31dac00640 1 -- 172.21.15.37:0/2079522601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 msgr2=0x7f31d40ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.508+0000 7f31dac00640 1 --2- 172.21.15.37:0/2079522601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d40ce320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009860 tx=0x7f31c002a5a0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.508+0000 7f31dac00640 1 -- 172.21.15.37:0/2079522601 shutdown_connections 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.508+0000 7f31dac00640 1 --2- 172.21.15.37:0/2079522601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d40ce320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.508+0000 7f31dac00640 1 -- 172.21.15.37:0/2079522601 >> 172.21.15.37:0/2079522601 conn(0x7f31d40cca40 msgr2=0x7f31d40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.509+0000 7f31dac00640 1 -- 172.21.15.37:0/2079522601 shutdown_connections 2024-09-21T06:31:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.509+0000 7f31dac00640 1 -- 172.21.15.37:0/2079522601 wait complete. 2024-09-21T06:31:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.511+0000 7f31dac00640 1 Processor -- start 2024-09-21T06:31:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.511+0000 7f31dac00640 1 -- start start 2024-09-21T06:31:45.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.512+0000 7f31dac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d40ce860 0x7f31d41f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31dac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d409fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31dac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31dac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31d406e180 con 0x7f31d415b9f0 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31dac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31d406e000 con 0x7f31d40ce860 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31dac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f31d406e300 con 0x7f31d40cf930 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55988/0 (socket says 172.21.15.37:55988) 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d3400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d409fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 -- 172.21.15.37:0/777354755 learned_addr learned my addr 172.21.15.37:0/777354755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d3e00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d40ce860 0x7f31d41f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 -- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 msgr2=0x7f31d409fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d409fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 -- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d40ce860 msgr2=0x7f31d41f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d40ce860 0x7f31d41f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.513+0000 7f31d8c00640 1 -- 172.21.15.37:0/777354755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d40a0940 con 0x7f31d415b9f0 2024-09-21T06:31:45.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31d8c00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00cad0 tx=0x7f31cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc016020 con 0x7f31d415b9f0 2024-09-21T06:31:45.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31cc00ad40 con 0x7f31d415b9f0 2024-09-21T06:31:45.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d40a0c20 con 0x7f31d415b9f0 2024-09-21T06:31:45.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc012680 con 0x7f31d415b9f0 2024-09-21T06:31:45.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.514+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d40c3d60 con 0x7f31d415b9f0 2024-09-21T06:31:45.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.515+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d40a0660 con 0x7f31d415b9f0 2024-09-21T06:31:45.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.516+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31cc019020 con 0x7f31d415b9f0 2024-09-21T06:31:45.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.517+0000 7f31cbe00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 0x7f31b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:45.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.517+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f31cc09b430 con 0x7f31d415b9f0 2024-09-21T06:31:45.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.518+0000 7f31d3e00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 0x7f31b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:45.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.519+0000 7f31d3e00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 0x7f31b407b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f31c4007fd0 tx=0x7f31c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:45.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.522+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31cc020020 con 0x7f31d415b9f0 2024-09-21T06:31:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.759+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f31d40b0450 con 0x7f31d415b9f0 2024-09-21T06:31:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.759+0000 7f31cbe00640 1 -- 172.21.15.37:0/777354755 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f31cc063080 con 0x7f31d415b9f0 2024-09-21T06:31:45.762 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:31:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 msgr2=0x7f31b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 0x7f31b407b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f31c4007fd0 tx=0x7f31c4009040 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 msgr2=0x7f31d40c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00cad0 tx=0x7f31cc00cf90 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 shutdown_connections 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31b4078bc0 0x7f31b407b0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d415b9f0 0x7f31d40c5440 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40cf930 0x7f31d409fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 --2- 172.21.15.37:0/777354755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d40ce860 0x7f31d41f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 >> 172.21.15.37:0/777354755 conn(0x7f31d40cca40 msgr2=0x7f31d40ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 shutdown_connections 2024-09-21T06:31:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:45.762+0000 7f31dac00640 1 -- 172.21.15.37:0/777354755 wait complete. 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:31:45.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:31:45.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:45.790 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:31:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:31:45.826 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42486sec preferred_lft 42486sec 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:31:45.827 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:31:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:45.829 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:31:45.920 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:31:45.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:45.923 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:31:45.960 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:31:45.960 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:31:45.960 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:31:45.960 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:31:45.960 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:31:45.961 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:31:46.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:31:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:31:46.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:46.159 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:31:46.194 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:31:46.194 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:31:46.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:31:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:46.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:31:46.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:31:46.290 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:31:46.325 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:31:46.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:31:46.326 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:31:46.401 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:31:46.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:46.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:31:46.602 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:31:46.602 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:31:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:31:46.623 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:31:46.623 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:31:46.623 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:31:46.623 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:31:46.623 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:31:46.655 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:31:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:31:46.657 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:31:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:31:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:31:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:31:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:31:46.660 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:31:46.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:46.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:31:46.717 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:31:46.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:31:46.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:31:46.814 INFO:teuthology.orchestra.run.smithi037.stdout:client6220 2024-09-21T06:31:46.815 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:31:46.815 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:31:46.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 -- 172.21.15.37:0/1957197432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f306c003a30 msgr2=0x7f306c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 --2- 172.21.15.37:0/1957197432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f306c003a30 0x7f306c003ea0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f30780016c0 tx=0x7f306c005830 comp rx=0 tx=0).stop 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308d200640 1 -- 172.21.15.37:0/1957197432 reap_dead start 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 -- 172.21.15.37:0/1957197432 shutdown_connections 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 -- 172.21.15.37:0/1957197432 >> 172.21.15.37:0/1957197432 conn(0x7f308806f920 msgr2=0x7f30880b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 -- 172.21.15.37:0/1957197432 shutdown_connections 2024-09-21T06:31:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.117+0000 7f308dc00640 1 -- 172.21.15.37:0/1957197432 wait complete. 2024-09-21T06:31:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.120+0000 7f308dc00640 1 Processor -- start 2024-09-21T06:31:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.120+0000 7f308dc00640 1 -- start start 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f306c003a30 0x7f30881f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f308812e520 0x7f30881f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30880cddc0 con 0x7f30881f5df0 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f308806dd40 con 0x7f306c003a30 2024-09-21T06:31:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.121+0000 7f308dc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f30880cdf40 con 0x7f308812e520 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56008/0 (socket says 172.21.15.37:56008) 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 -- 172.21.15.37:0/3542485632 learned_addr learned my addr 172.21.15.37:0/3542485632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f3087e00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f306c003a30 0x7f30881f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f308d200640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f308812e520 0x7f30881f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 -- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f308812e520 msgr2=0x7f30881f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f308812e520 0x7f30881f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 -- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f306c003a30 msgr2=0x7f30881f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f306c003a30 0x7f30881f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 -- 172.21.15.37:0/3542485632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30880cc0a0 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.122+0000 7f307fe00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f307000f7f0 tx=0x7f307000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.123+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070015070 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.123+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3070004460 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.123+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000bb00 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.123+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30880cb2e0 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.123+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30880cb8a0 con 0x7f30881f5df0 2024-09-21T06:31:47.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.124+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30881f6390 con 0x7f30881f5df0 2024-09-21T06:31:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.125+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f307000bca0 con 0x7f30881f5df0 2024-09-21T06:31:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.126+0000 7f3084c00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 0x7f306407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.126+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f307009b890 con 0x7f30881f5df0 2024-09-21T06:31:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.127+0000 7f3087e00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 0x7f306407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.128+0000 7f3087e00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 0x7f306407b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3078032000 tx=0x7f3078001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:47.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.131+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30700634e0 con 0x7f30881f5df0 2024-09-21T06:31:47.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.367+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f30880ce4d0 con 0x7f30881f5df0 2024-09-21T06:31:47.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.516+0000 7f3084c00640 1 -- 172.21.15.37:0/3542485632 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v53) ==== 105+0+0 (secure 0 0 0) 0x7f3070067f10 con 0x7f30881f5df0 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 msgr2=0x7f306407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 0x7f306407b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3078032000 tx=0x7f3078001190 comp rx=0 tx=0).stop 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 msgr2=0x7f308812dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f307000f7f0 tx=0x7f307000fcb0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 shutdown_connections 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3064078bc0 0x7f306407b0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30881f5df0 0x7f308812dfe0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f308812e520 0x7f30881f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 --2- 172.21.15.37:0/3542485632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f306c003a30 0x7f30881f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 >> 172.21.15.37:0/3542485632 conn(0x7f308806f920 msgr2=0x7f3088073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:47.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 shutdown_connections 2024-09-21T06:31:47.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.518+0000 7f308dc00640 1 -- 172.21.15.37:0/3542485632 wait complete. 2024-09-21T06:31:47.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:47.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 -- 172.21.15.37:0/1371899844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a10073f20 msgr2=0x7f6a1015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 --2- 172.21.15.37:0/1371899844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a10073f20 0x7f6a1015ca00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6a100d0260 tx=0x7f6a0002a560 comp rx=0 tx=0).stop 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 -- 172.21.15.37:0/1371899844 shutdown_connections 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 --2- 172.21.15.37:0/1371899844 >> v2:172.21.15.150:3300/0 conn(0x7f6a100afb80 0x7f6a1006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 --2- 172.21.15.37:0/1371899844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a10073f20 0x7f6a1015ca00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 -- 172.21.15.37:0/1371899844 >> 172.21.15.37:0/1371899844 conn(0x7f6a10073400 msgr2=0x7f6a10073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 -- 172.21.15.37:0/1371899844 shutdown_connections 2024-09-21T06:31:47.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.774+0000 7f6a15400640 1 -- 172.21.15.37:0/1371899844 wait complete. 2024-09-21T06:31:47.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.776+0000 7f6a15400640 1 Processor -- start 2024-09-21T06:31:47.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.776+0000 7f6a15400640 1 -- start start 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a10073f20 0x7f6a101f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a100afb80 0x7f6a101f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10160700 con 0x7f6a1006e330 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a100cfb80 con 0x7f6a10073f20 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a15400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6a10160880 con 0x7f6a100afb80 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56024/0 (socket says 172.21.15.37:56024) 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a0fe00640 1 -- 172.21.15.37:0/4226445027 learned_addr learned my addr 172.21.15.37:0/4226445027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:47.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a14a00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a100afb80 0x7f6a101f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.778+0000 7f6a07e00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a10073f20 0x7f6a101f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 -- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a100afb80 msgr2=0x7f6a101f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a100afb80 0x7f6a101f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 -- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a10073f20 msgr2=0x7f6a101f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a10073f20 0x7f6a101f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 -- 172.21.15.37:0/4226445027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a10146cc0 con 0x7f6a1006e330 2024-09-21T06:31:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0fe00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a00034040 tx=0x7f6a000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0003e030 con 0x7f6a1006e330 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a000047a0 con 0x7f6a1006e330 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00035040 con 0x7f6a1006e330 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a10149bf0 con 0x7f6a1006e330 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.779+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a100b4950 con 0x7f6a1006e330 2024-09-21T06:31:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.780+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1015ca00 con 0x7f6a1006e330 2024-09-21T06:31:47.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.782+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a00041030 con 0x7f6a1006e330 2024-09-21T06:31:47.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.783+0000 7f6a0cc00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 0x7f69ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:47.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.784+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6a000bb700 con 0x7f6a1006e330 2024-09-21T06:31:47.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.784+0000 7f6a07e00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 0x7f69ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.785+0000 7f6a07e00640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 0x7f69ec07b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005fd0 tx=0x7f69f8005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:47.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:47.788+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a1006e740 con 0x7f6a1006e330 2024-09-21T06:31:48.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.023+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a100b01f0 con 0x7f6a1006e330 2024-09-21T06:31:48.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.024+0000 7f6a0cc00640 1 -- 172.21.15.37:0/4226445027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5246 (secure 0 0 0) 0x7f6a000839a0 con 0x7f6a1006e330 2024-09-21T06:31:48.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:48.026 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":54,"btime":"2024-09-21T06:31:47:520576+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:31:47.520562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6106,"mds_1":6100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:creating","state_seq":1,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":4}]} 2024-09-21T06:31:48.027 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 54 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.026+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 msgr2=0x7f69ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.026+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 0x7f69ec07b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005fd0 tx=0x7f69f8005850 comp rx=0 tx=0).stop 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.026+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 msgr2=0x7f6a101461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.026+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a00034040 tx=0x7f6a000039e0 comp rx=0 tx=0).stop 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 shutdown_connections 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f69ec078b90 0x7f69ec07b0a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a100afb80 0x7f6a101f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a10073f20 0x7f6a101f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 --2- 172.21.15.37:0/4226445027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1006e330 0x7f6a101461e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 >> 172.21.15.37:0/4226445027 conn(0x7f6a10073400 msgr2=0x7f6a100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 shutdown_connections 2024-09-21T06:31:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:48.027+0000 7f6a15400640 1 -- 172.21.15.37:0/4226445027 wait complete. 2024-09-21T06:31:48.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:31:37.988979+0000', 'modified': '2024-09-21T06:31:47.520562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6106, 'mds_1': 6100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6100': {'gid': 6100, 'name': 'd', 'rank': 1, 'incarnation': 54, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.150:6837/2893186945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 2893186945}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2893186945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6106': {'gid': 6106, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1569973003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1569973003}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1569973003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6106, 'qdb_cluster': [6106]} 2024-09-21T06:31:48.453 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6100:up:creating 2024-09-21T06:31:49.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 -- 172.21.15.37:0/1900558025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7108074cb0 msgr2=0x7f7108073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 --2- 172.21.15.37:0/1900558025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7108074cb0 0x7f7108073dd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009830 tx=0x7f70fc02a510 comp rx=0 tx=0).stop 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 -- 172.21.15.37:0/1900558025 shutdown_connections 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 --2- 172.21.15.37:0/1900558025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 0x7f71080cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 --2- 172.21.15.37:0/1900558025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7108074cb0 0x7f7108073dd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.692+0000 7f710fc00640 1 -- 172.21.15.37:0/1900558025 >> 172.21.15.37:0/1900558025 conn(0x7f71080cca40 msgr2=0x7f71080cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.693+0000 7f710fc00640 1 -- 172.21.15.37:0/1900558025 shutdown_connections 2024-09-21T06:31:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.693+0000 7f710fc00640 1 -- 172.21.15.37:0/1900558025 wait complete. 2024-09-21T06:31:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.695+0000 7f710fc00640 1 Processor -- start 2024-09-21T06:31:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.695+0000 7f710fc00640 1 -- start start 2024-09-21T06:31:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 0x7f71080cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 0x7f71080ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f710806f360 con 0x7f710806ddf0 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f710806f1e0 con 0x7f7108074cb0 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.696+0000 7f710fc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f710806f4e0 con 0x7f71080cbc10 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56034/0 (socket says 172.21.15.37:56034) 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 -- 172.21.15.37:0/2784962206 learned_addr learned my addr 172.21.15.37:0/2784962206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f7107e00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 0x7f71080cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710dc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 0x7f71080ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 -- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 msgr2=0x7f71080ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 0x7f71080ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 -- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 msgr2=0x7f71080cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 0x7f71080cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 -- 172.21.15.37:0/2784962206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71080cdf50 con 0x7f710806ddf0 2024-09-21T06:31:49.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f7107e00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 0x7f71080cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:49.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.697+0000 7f710d200640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009950 tx=0x7f70fc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:49.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.698+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc031d60 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.698+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70fc033e10 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.698+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc005260 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.698+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71080ce230 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.698+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71080ce7f0 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.699+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7108072860 con 0x7f710806ddf0 2024-09-21T06:31:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.700+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70fc005400 con 0x7f710806ddf0 2024-09-21T06:31:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.701+0000 7f7104c00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 0x7f70e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.701+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f70fc0bc310 con 0x7f710806ddf0 2024-09-21T06:31:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.702+0000 7f7107e00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 0x7f70e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.702+0000 7f7107e00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 0x7f70e807b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005fd0 tx=0x7f70f4005ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:49.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.706+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7108072860 con 0x7f710806ddf0 2024-09-21T06:31:49.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.944+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f710806fa00 con 0x7f710806ddf0 2024-09-21T06:31:49.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.945+0000 7f7104c00640 1 -- 172.21.15.37:0/2784962206 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5268 (secure 0 0 0) 0x7f70fc03f030 con 0x7f710806ddf0 2024-09-21T06:31:49.947 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:49.947 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"btime":"2024-09-21T06:31:49:535251+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:31:49.535243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:31:49.948 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 57 2024-09-21T06:31:49.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 msgr2=0x7f70e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:49.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 0x7f70e807b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005fd0 tx=0x7f70f4005ea0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 msgr2=0x7f71081f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009950 tx=0x7f70fc033c20 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 shutdown_connections 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f70e8078b90 0x7f70e807b0a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71080cbc10 0x7f71080ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108074cb0 0x7f71080cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 --2- 172.21.15.37:0/2784962206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f710806ddf0 0x7f71081f5390 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 >> 172.21.15.37:0/2784962206 conn(0x7f71080cca40 msgr2=0x7f710806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 shutdown_connections 2024-09-21T06:31:49.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:49.948+0000 7f710fc00640 1 -- 172.21.15.37:0/2784962206 wait complete. 2024-09-21T06:31:50.020 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:31:50.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:31:50.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:31:37.988979+0000', 'modified': '2024-09-21T06:31:49.535243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6106, 'mds_1': 6100, 'mds_2': 6097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6097': {'gid': 6097, 'name': 'b', 'rank': 2, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.150:6836/4023630261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 4023630261}, {'type': 'v1', 'addr': '172.21.15.150:6836', 'nonce': 4023630261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6100': {'gid': 6100, 'name': 'd', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.150:6837/2893186945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 2893186945}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2893186945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6106': {'gid': 6106, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1569973003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1569973003}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1569973003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6106, 'qdb_cluster': [6097, 6100, 6106]} 2024-09-21T06:31:50.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:31:50.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:31:50.433 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:31:50.431+0000 7f34ec400640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.606+0000 7fa1a7200640 1 -- 172.21.15.37:0/1463410817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a80cc0e0 msgr2=0x7fa1a80702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.606+0000 7fa1a7200640 1 --2- 172.21.15.37:0/1463410817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a80cc0e0 0x7fa1a80702a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ce670 tx=0x7fa19802a4d0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 -- 172.21.15.37:0/1463410817 shutdown_connections 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 --2- 172.21.15.37:0/1463410817 >> v2:172.21.15.150:3300/0 conn(0x7fa1a80cf6d0 0x7fa1a815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 --2- 172.21.15.37:0/1463410817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a80cc0e0 0x7fa1a80702a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 -- 172.21.15.37:0/1463410817 >> 172.21.15.37:0/1463410817 conn(0x7fa1a80d05c0 msgr2=0x7fa1a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 -- 172.21.15.37:0/1463410817 shutdown_connections 2024-09-21T06:31:50.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.607+0000 7fa1a7200640 1 -- 172.21.15.37:0/1463410817 wait complete. 2024-09-21T06:31:50.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.609+0000 7fa1a7200640 1 Processor -- start 2024-09-21T06:31:50.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.609+0000 7fa1a7200640 1 -- start start 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.610+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1a80cc0e0 0x7fa1a8072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.610+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a80cf6d0 0x7fa1a81ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a7200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a815d000 con 0x7fa1a8160420 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a7200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80b43e0 con 0x7fa1a80cf6d0 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a7200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a815d180 con 0x7fa1a80cc0e0 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56052/0 (socket says 172.21.15.37:56052) 2024-09-21T06:31:50.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 -- 172.21.15.37:0/2897706799 learned_addr learned my addr 172.21.15.37:0/2897706799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1a80cc0e0 0x7fa1a8072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa19d400640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a80cf6d0 0x7fa1a81ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 -- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1a80cc0e0 msgr2=0x7fa1a8072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1a80cc0e0 0x7fa1a8072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 -- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a80cf6d0 msgr2=0x7fa1a81ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a80cf6d0 0x7fa1a81ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.611+0000 7fa1a6800640 1 -- 172.21.15.37:0/2897706799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8073450 con 0x7fa1a8160420 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b9e0 tx=0x7fa1a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa19ea00640 1 -- 172.21.15.37:0/2897706799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003b70 con 0x7fa1a8160420 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa19ea00640 1 -- 172.21.15.37:0/2897706799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1a0003e00 con 0x7fa1a8160420 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa19ea00640 1 -- 172.21.15.37:0/2897706799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000cb80 con 0x7fa1a8160420 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8073730 con 0x7fa1a8160420 2024-09-21T06:31:50.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.612+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a80c6720 con 0x7fa1a8160420 2024-09-21T06:31:50.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.616+0000 7fa19ea00640 1 -- 172.21.15.37:0/2897706799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1a0004000 con 0x7fa1a8160420 2024-09-21T06:31:50.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.617+0000 7fa19ea00640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 0x7fa18407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.617+0000 7fa19ea00640 1 -- 172.21.15.37:0/2897706799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa1a009ac60 con 0x7fa1a8160420 2024-09-21T06:31:50.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.617+0000 7fa1b3abe740 1 Processor -- start 2024-09-21T06:31:50.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.617+0000 7fa1b3abe740 1 -- start start 2024-09-21T06:31:50.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.617+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 0x7fa18407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.618+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 0x7fa18407b0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa198033000 tx=0x7fa198004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.618+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.618+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5cf3b0 0x55b7ba9c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba922430 con 0x55b7ba5d7590 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba9125d0 con 0x55b7ba5f8270 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b7ba901a00 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56066/0 (socket says 172.21.15.37:56066) 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 -- 172.21.15.37:0/2045567933 learned_addr learned my addr 172.21.15.37:0/2045567933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa19d400640 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5cf3b0 0x55b7ba9c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 -- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5cf3b0 msgr2=0x55b7ba9c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5cf3b0 0x55b7ba9c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 -- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 msgr2=0x55b7ba9c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.619+0000 7fa1a6800640 1 -- 172.21.15.37:0/2045567933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ba8ca520 con 0x55b7ba5d7590 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa1a6800640 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0062840 tx=0x7fa1a00620e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3fc3a96b91095f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa196600640 1 -- 172.21.15.37:0/2045567933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00672e0 con 0x55b7ba5d7590 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa196600640 1 -- 172.21.15.37:0/2045567933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1a006c190 con 0x55b7ba5d7590 2024-09-21T06:31:50.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa196600640 1 -- 172.21.15.37:0/2045567933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a006bd00 con 0x55b7ba5d7590 2024-09-21T06:31:50.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa1b3abe740 1 -- 172.21.15.37:0/2045567933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 msgr2=0x55b7ba999150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.620+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0062840 tx=0x7fa1a00620e0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 -- 172.21.15.37:0/2045567933 shutdown_connections 2024-09-21T06:31:50.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba999150 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5cf3b0 0x55b7ba9c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/2045567933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 -- 172.21.15.37:0/2045567933 >> 172.21.15.37:0/2045567933 conn(0x55b7ba5ec3c0 msgr2=0x55b7ba99eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 -- 172.21.15.37:0/2045567933 shutdown_connections 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.621+0000 7fa1b3abe740 1 -- 172.21.15.37:0/2045567933 wait complete. 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.622+0000 7fa1b3abe740 1 -- start start 2024-09-21T06:31:50.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.622+0000 7fa1b3abe740 1 Processor -- start 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7ba9bf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba8a8330 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba905810 con 0x55b7ba5f8270 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.623+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b7ba672620 con 0x55b7ba5d7590 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1b3abe740 10 client.0 _ll_register_callbacks cb 0x55b7ba8ad420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56080/0 (socket says 172.21.15.37:56080) 2024-09-21T06:31:50.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 -- 172.21.15.37:0/1759813846 learned_addr learned my addr 172.21.15.37:0/1759813846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a6800640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa19d400640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7ba9bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 -- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 msgr2=0x55b7ba9bf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7ba9bf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 -- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 msgr2=0x55b7ba9c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa1a5e00640 1 -- 172.21.15.37:0/1759813846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ba663850 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.624+0000 7fa19d400640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7ba9bf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa1980099c0 tx=0x7fa198009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8c71aecac83621 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198035040 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa198003d40 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a006c190 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa1980047a0 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa1b3abe740 10 client.6417 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b7ba6da6b0 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1a006c190 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.625+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7fa198004a30 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 10 client.6417 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 10 client.6417 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 20 client.6417 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba5b6e40 0x55b7ba9f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 4 client.6417 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7ba914520 con 0x55b7ba5b6e40 2024-09-21T06:31:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.626+0000 7fa19d400640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba5b6e40 0x55b7ba9f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.627+0000 7fa19d400640 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba5b6e40 0x55b7ba9f30e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ab7fee440a0b3bd server_cookie=6a358d071aa70e36 in_seq=0 out_seq=0 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.627+0000 7fa18aa00640 10 client.6417 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa18aa00640 1 -- 172.21.15.37:0/1759813846 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7ba914520 con 0x55b7ba5b6e40 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa18aa00640 10 client.6417 handle_command_reply: tid=0 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 client.6417 shutdown 2024-09-21T06:31:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 msgr2=0x55b7ba9c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa1980099c0 tx=0x7fa198009260 comp rx=0 tx=0).stop 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 shutdown_connections 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba5b6e40 0x55b7ba9f30e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7ba9bf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1759813846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c41b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 >> 172.21.15.37:0/1759813846 conn(0x55b7ba5ec3c0 msgr2=0x55b7ba9e6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa18aa00640 0 client.6417 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 shutdown_connections 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.628+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1759813846 wait complete. 2024-09-21T06:31:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.629+0000 7fa1b3abe740 20 client.6417 trim_cache size 0 max 16384 2024-09-21T06:31:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.648+0000 7fa1b3abe740 1 Processor -- start 2024-09-21T06:31:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.648+0000 7fa1b3abe740 1 -- start start 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5f8270 0x55b7ba9e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8202580 con 0x55b7ba5d7590 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a82017e0 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.649+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8201eb0 con 0x55b7ba5f8270 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56086/0 (socket says 172.21.15.37:56086) 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 -- 172.21.15.37:0/1141679778 learned_addr learned my addr 172.21.15.37:0/1141679778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa1a6800640 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5f8270 0x55b7ba9e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 -- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5f8270 msgr2=0x55b7ba9e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5f8270 0x55b7ba9e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 -- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5cf3b0 msgr2=0x55b7ba9e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 -- 172.21.15.37:0/1141679778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ba9bf000 con 0x55b7ba5d7590 2024-09-21T06:31:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.650+0000 7fa19d400640 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8072940 tx=0x7fa19000b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389cf5572eeb6f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa195c00640 1 -- 172.21.15.37:0/1141679778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190028070 con 0x55b7ba5d7590 2024-09-21T06:31:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa195c00640 1 -- 172.21.15.37:0/1141679778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa190003f30 con 0x55b7ba5d7590 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa195c00640 1 -- 172.21.15.37:0/1141679778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900116a0 con 0x55b7ba5d7590 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1141679778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 msgr2=0x55b7ba9e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8072940 tx=0x7fa19000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1141679778 shutdown_connections 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5d7590 0x55b7ba9e7150 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5f8270 0x55b7ba9e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/1141679778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1141679778 >> 172.21.15.37:0/1141679778 conn(0x55b7ba5ec3c0 msgr2=0x55b7ba9946c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.651+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1141679778 shutdown_connections 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.652+0000 7fa1b3abe740 1 -- 172.21.15.37:0/1141679778 wait complete. 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.652+0000 7fa1b3abe740 1 -- start start 2024-09-21T06:31:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.653+0000 7fa1b3abe740 1 Processor -- start 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7baa63400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b7ba9dee90 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80c6130 con 0x55b7ba5f8270 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b7ba9f6520 con 0x55b7ba5d7590 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1b3abe740 10 client.0 _ll_register_callbacks cb 0x7fa1980040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56096/0 (socket says 172.21.15.37:56096) 2024-09-21T06:31:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.654+0000 7fa1a5e00640 1 -- 172.21.15.37:0/3744435575 learned_addr learned my addr 172.21.15.37:0/3744435575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa19d400640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7baa63400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a6800640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 -- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 msgr2=0x55b7baa63400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7baa63400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 -- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 msgr2=0x55b7ba9c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 -- 172.21.15.37:0/3744435575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ba9987b0 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a6800640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa1a5e00640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa19803f040 tx=0x7fa1980049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d95f8238b55ef79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa18aa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.655+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198062020 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa198003dd0 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa190003f30 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa198035070 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 10 client.6423 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b7baa63a40 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa190003f30 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7fa19806f020 con 0x55b7ba5cf3b0 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 10 client.6423 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 10 client.6423 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 20 client.6423 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba9e7320 0x55b7baa058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 4 client.6423 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.656+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "status"}) -- 0x55b7ba90ffd0 con 0x55b7ba9e7320 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.657+0000 7fa19d400640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba9e7320 0x55b7baa058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.657+0000 7fa19d400640 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba9e7320 0x55b7baa058b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e0567edf1bf353e server_cookie=f43200c9acdf235d in_seq=0 out_seq=0 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.657+0000 7fa18aa00640 10 client.6423 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa18aa00640 1 -- 172.21.15.37:0/3744435575 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b7ba90ffd0 con 0x55b7ba9e7320 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa18aa00640 10 client.6423 handle_command_reply: tid=0 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 client.6423 shutdown 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 msgr2=0x55b7ba9c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa19803f040 tx=0x7fa1980049c0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 shutdown_connections 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x55b7ba9e7320 0x55b7baa058b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7ba5f8270 0x55b7ba9c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b7ba5d7590 0x55b7baa63400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa18aa00640 0 client.6423 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 --2- 172.21.15.37:0/3744435575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b7ba5cf3b0 0x55b7ba9c5f90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 >> 172.21.15.37:0/3744435575 conn(0x55b7ba5ec3c0 msgr2=0x55b7ba9e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 shutdown_connections 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 1 -- 172.21.15.37:0/3744435575 wait complete. 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.658+0000 7fa1b3abe740 20 client.6423 trim_cache size 0 max 16384 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6097, 2024-09-21T06:31:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1229177849999998, 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 57, 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 67, 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 67, 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.180675053000002 2024-09-21T06:31:50.664 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 msgr2=0x7fa18407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 0x7fa18407b0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa198033000 tx=0x7fa198004730 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 msgr2=0x7fa1a815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b9e0 tx=0x7fa1a000bea0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 shutdown_connections 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa184078bc0 0x7fa18407b0d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1a8160420 0x7fa1a815b5d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a80cf6d0 0x7fa1a81ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 --2- 172.21.15.37:0/2897706799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1a80cc0e0 0x7fa1a8072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 >> 172.21.15.37:0/2897706799 conn(0x7fa1a80d05c0 msgr2=0x7fa1a815f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 shutdown_connections 2024-09-21T06:31:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.659+0000 7fa1a7200640 1 -- 172.21.15.37:0/2897706799 wait complete. 2024-09-21T06:31:50.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 -- 172.21.15.37:0/2924760086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc20073a00 msgr2=0x7fdc20162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 --2- 172.21.15.37:0/2924760086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc20073a00 0x7fdc20162340 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdc08005fd0 tx=0x7fdc0802a530 comp rx=0 tx=0).stop 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 -- 172.21.15.37:0/2924760086 shutdown_connections 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 --2- 172.21.15.37:0/2924760086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc20073a00 0x7fdc20162340 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 --2- 172.21.15.37:0/2924760086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc200aef80 0x7fdc200ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 -- 172.21.15.37:0/2924760086 >> 172.21.15.37:0/2924760086 conn(0x7fdc200d05c0 msgr2=0x7fdc2006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 -- 172.21.15.37:0/2924760086 shutdown_connections 2024-09-21T06:31:50.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.908+0000 7fdc25a00640 1 -- 172.21.15.37:0/2924760086 wait complete. 2024-09-21T06:31:50.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.911+0000 7fdc25a00640 1 Processor -- start 2024-09-21T06:31:50.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.911+0000 7fdc25a00640 1 -- start start 2024-09-21T06:31:50.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc20073a00 0x7fdc201574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc20162880 0x7fdc201f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201550c0 con 0x7fdc200aef80 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20155240 con 0x7fdc20162880 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc25a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdc201553c0 con 0x7fdc20073a00 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56114/0 (socket says 172.21.15.37:56114) 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.912+0000 7fdc17e00640 1 -- 172.21.15.37:0/172572901 learned_addr learned my addr 172.21.15.37:0/172572901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc25000640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc20162880 0x7fdc201f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc20073a00 0x7fdc201574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 -- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc20073a00 msgr2=0x7fdc201574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc20073a00 0x7fdc201574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 -- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc20162880 msgr2=0x7fdc201f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc20162880 0x7fdc201f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 -- 172.21.15.37:0/172572901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc201fa7d0 con 0x7fdc200aef80 2024-09-21T06:31:50.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.913+0000 7fdc17e00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdc08034040 tx=0x7fdc080038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.914+0000 7fdc1cc00640 1 -- 172.21.15.37:0/172572901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0803e030 con 0x7fdc200aef80 2024-09-21T06:31:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.914+0000 7fdc1cc00640 1 -- 172.21.15.37:0/172572901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc08003d90 con 0x7fdc200aef80 2024-09-21T06:31:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.914+0000 7fdc1cc00640 1 -- 172.21.15.37:0/172572901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08035040 con 0x7fdc200aef80 2024-09-21T06:31:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.914+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc201f9f30 con 0x7fdc200aef80 2024-09-21T06:31:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.914+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc201fa4f0 con 0x7fdc200aef80 2024-09-21T06:31:50.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.916+0000 7fdc1cc00640 1 -- 172.21.15.37:0/172572901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc08041030 con 0x7fdc200aef80 2024-09-21T06:31:50.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.917+0000 7fdc1cc00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 0x7fdbfc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.917+0000 7fdc1cc00640 1 -- 172.21.15.37:0/172572901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fdc080bb690 con 0x7fdc200aef80 2024-09-21T06:31:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.918+0000 7fdc2e24d740 1 Processor -- start 2024-09-21T06:31:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.918+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 0x7fdbfc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.918+0000 7fdc2e24d740 1 -- start start 2024-09-21T06:31:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.919+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 0x7fdbfc07b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc10004630 tx=0x7fdc1000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.919+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.919+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.919+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x56231519e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562315127430 con 0x562314dfd270 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5623151175d0 con 0x562314dd43b0 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562315106a00 con 0x562314ddc590 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56128/0 (socket says 172.21.15.37:56128) 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 -- 172.21.15.37:0/425483333 learned_addr learned my addr 172.21.15.37:0/425483333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc25000640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x56231519e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc17e00640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 -- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 msgr2=0x56231519e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x56231519e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 -- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 msgr2=0x5623151ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc1fe00640 1 -- 172.21.15.37:0/425483333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623150cf520 con 0x562314dfd270 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.920+0000 7fdc17e00640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.921+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000bbc0 tx=0x7fdc10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99eeed53c2bb9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.921+0000 7fdc14c00640 1 -- 172.21.15.37:0/425483333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10011e00 con 0x562314dfd270 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.921+0000 7fdc14c00640 1 -- 172.21.15.37:0/425483333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc1001e510 con 0x562314dfd270 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.921+0000 7fdc14c00640 1 -- 172.21.15.37:0/425483333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1001e920 con 0x562314dfd270 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 -- 172.21.15.37:0/425483333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 msgr2=0x5623151cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000bbc0 tx=0x7fdc10013000 comp rx=0 tx=0).stop 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 -- 172.21.15.37:0/425483333 shutdown_connections 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x56231519e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/425483333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151cdb60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 -- 172.21.15.37:0/425483333 >> 172.21.15.37:0/425483333 conn(0x562314df13c0 msgr2=0x5623151a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 -- 172.21.15.37:0/425483333 shutdown_connections 2024-09-21T06:31:50.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.922+0000 7fdc2e24d740 1 -- 172.21.15.37:0/425483333 wait complete. 2024-09-21T06:31:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.923+0000 7fdc2e24d740 1 -- start start 2024-09-21T06:31:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.923+0000 7fdc2e24d740 1 Processor -- start 2024-09-21T06:31:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x5623151e2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5623150ad330 con 0x562314dfd270 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56231510a810 con 0x562314dd43b0 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562314e77620 con 0x562314ddc590 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc2e24d740 10 client.0 _ll_register_callbacks cb 0x5623150b2420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56138/0 (socket says 172.21.15.37:56138) 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 -- 172.21.15.37:0/1354727949 learned_addr learned my addr 172.21.15.37:0/1354727949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc17e00640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x5623151e2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 -- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 msgr2=0x5623151e2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x5623151e2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 -- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 msgr2=0x5623151e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.925+0000 7fdc25000640 1 -- 172.21.15.37:0/1354727949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562314e68850 con 0x562314dfd270 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc25000640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800bb80 tx=0x7fdc1800a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4fdb3538448a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc0d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1801a020 con 0x562314dfd270 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1001e510 con 0x562314dfd270 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fdc18016670 con 0x562314dfd270 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdc18024050 con 0x562314dfd270 2024-09-21T06:31:50.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc2e24d740 10 client.6435 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.926+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x562314edf6b0 con 0x562314dfd270 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc1001e510 con 0x562314dfd270 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7fdc18027020 con 0x562314dfd270 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 10 client.6435 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 10 client.6435 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 20 client.6435 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x562314dbbe40 0x5623151f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 4 client.6435 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562315119520 con 0x562314dbbe40 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.927+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x562314dbbe40 0x5623151f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.928+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x562314dbbe40 0x5623151f9180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a56175b8ad7e5cc server_cookie=a2675ca3564ae3e4 in_seq=0 out_seq=0 2024-09-21T06:31:50.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.928+0000 7fdc0d600640 10 client.6435 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:31:50.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc0d600640 1 -- 172.21.15.37:0/1354727949 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562315119520 con 0x562314dbbe40 2024-09-21T06:31:50.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc0d600640 10 client.6435 handle_command_reply: tid=0 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc2e24d740 1 client.6435 shutdown 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 msgr2=0x5623151e9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800bb80 tx=0x7fdc1800a7b0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 shutdown_connections 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.929+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x562314dbbe40 0x5623151f9180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x5623151e9a10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314ddc590 0x5623151e2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/1354727949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x5623151e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 >> 172.21.15.37:0/1354727949 conn(0x562314df13c0 msgr2=0x5623151c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc0d600640 0 client.6435 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 shutdown_connections 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 1 -- 172.21.15.37:0/1354727949 wait complete. 2024-09-21T06:31:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.930+0000 7fdc2e24d740 20 client.6435 trim_cache size 0 max 16384 2024-09-21T06:31:50.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.949+0000 7fdc2e24d740 1 Processor -- start 2024-09-21T06:31:50.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.949+0000 7fdc2e24d740 1 -- start start 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dd43b0 0x5623151fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314ddc590 0x56231519f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20161d10 con 0x562314dfd270 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20201ee0 con 0x562314ddc590 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.950+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdc201517d0 con 0x562314dd43b0 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56148/0 (socket says 172.21.15.37:56148) 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 -- 172.21.15.37:0/3988155480 learned_addr learned my addr 172.21.15.37:0/3988155480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc17e00640 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314ddc590 0x56231519f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 -- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dd43b0 msgr2=0x5623151fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dd43b0 0x5623151fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 -- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314ddc590 msgr2=0x56231519f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314ddc590 0x56231519f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 -- 172.21.15.37:0/3988155480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623151cb060 con 0x562314dfd270 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.951+0000 7fdc25000640 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f80 tx=0x7fdc18009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6536ae0f3ce1ad66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc0fe00640 1 -- 172.21.15.37:0/3988155480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1802c070 con 0x562314dfd270 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc0fe00640 1 -- 172.21.15.37:0/3988155480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc18016bb0 con 0x562314dfd270 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc0fe00640 1 -- 172.21.15.37:0/3988155480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1801a070 con 0x562314dfd270 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 -- 172.21.15.37:0/3988155480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 msgr2=0x56231519f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f80 tx=0x7fdc18009fc0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 -- 172.21.15.37:0/3988155480 shutdown_connections 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314dfd270 0x56231519f650 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314ddc590 0x56231519f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/3988155480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dd43b0 0x5623151fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 -- 172.21.15.37:0/3988155480 >> 172.21.15.37:0/3988155480 conn(0x562314df13c0 msgr2=0x5623151e2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.952+0000 7fdc2e24d740 1 -- 172.21.15.37:0/3988155480 shutdown_connections 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.953+0000 7fdc2e24d740 1 -- 172.21.15.37:0/3988155480 wait complete. 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.953+0000 7fdc2e24d740 1 -- start start 2024-09-21T06:31:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.954+0000 7fdc2e24d740 1 Processor -- start 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x562315269640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dfd270 0x5623151c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5623151c6d50 con 0x562314ddc590 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5623151e2cb0 con 0x562314dd43b0 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5623151c72e0 con 0x562314dfd270 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc2e24d740 10 client.0 _ll_register_callbacks cb 0x7fdc1800b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56150/0 (socket says 172.21.15.37:56150) 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc17e00640 1 -- 172.21.15.37:0/204456690 learned_addr learned my addr 172.21.15.37:0/204456690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x562315269640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.955+0000 7fdc25000640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dfd270 0x5623151c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 -- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dfd270 msgr2=0x5623151c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dfd270 0x5623151c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 -- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 msgr2=0x562315269640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x562315269640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 -- 172.21.15.37:0/204456690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623151e2a40 con 0x562314ddc590 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc17e00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc080804c0 tx=0x7fdc08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb792d6d1349ac2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc0d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08048040 con 0x562314ddc590 2024-09-21T06:31:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fdc0808c930 con 0x562314ddc590 2024-09-21T06:31:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc18016bb0 con 0x562314ddc590 2024-09-21T06:31:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdc08087c80 con 0x562314ddc590 2024-09-21T06:31:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc2e24d740 10 client.6441 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.956+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x5623152630e0 con 0x562314ddc590 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc18016bb0 con 0x562314ddc590 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7fdc0806ab90 con 0x562314ddc590 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 10 client.6441 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 10 client.6441 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 20 client.6441 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5623151f3970 0x56231526c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 4 client.6441 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "status"}) -- 0x562315114fd0 con 0x5623151f3970 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.957+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5623151f3970 0x56231526c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc1fe00640 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5623151f3970 0x56231526c320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96ae5d3a481e4884 server_cookie=31f54e2077e3ed70 in_seq=0 out_seq=0 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc0d600640 10 client.6441 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:31:50.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc0d600640 1 -- 172.21.15.37:0/204456690 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562315114fd0 con 0x5623151f3970 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc0d600640 10 client.6441 handle_command_reply: tid=0 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 client.6441 shutdown 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 msgr2=0x562315262ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc080804c0 tx=0x7fdc08009260 comp rx=0 tx=0).stop 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 shutdown_connections 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5623151f3970 0x56231526c320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562314ddc590 0x562315262ba0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562314dfd270 0x5623151c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc0d600640 0 client.6441 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 --2- 172.21.15.37:0/204456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562314dd43b0 0x562315269640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.958+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 >> 172.21.15.37:0/204456690 conn(0x562314df13c0 msgr2=0x5623151e97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 shutdown_connections 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc2e24d740 1 -- 172.21.15.37:0/204456690 wait complete. 2024-09-21T06:31:50.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc2e24d740 20 client.6441 trim_cache size 0 max 16384 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6100, 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.4331537810000001, 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 57, 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 67, 2024-09-21T06:31:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 67, 2024-09-21T06:31:50.965 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.47985443 2024-09-21T06:31:50.965 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 msgr2=0x7fdbfc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 0x7fdbfc07b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc10004630 tx=0x7fdc1000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 msgr2=0x7fdc20142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdc08034040 tx=0x7fdc080038a0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 shutdown_connections 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdbfc078b90 0x7fdbfc07b0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc200aef80 0x7fdc20142980 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc20162880 0x7fdc201f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 --2- 172.21.15.37:0/172572901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc20073a00 0x7fdc201574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 >> 172.21.15.37:0/172572901 conn(0x7fdc200d05c0 msgr2=0x7fdc200b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.959+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 shutdown_connections 2024-09-21T06:31:50.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:50.960+0000 7fdc25a00640 1 -- 172.21.15.37:0/172572901 wait complete. 2024-09-21T06:31:50.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f0420800640 1 -- 172.21.15.37:0/1880030501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0400003c10 msgr2=0x7f0400004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f0420800640 1 --2- 172.21.15.37:0/1880030501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0400003c10 0x7f0400004060 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f040c001830 tx=0x7f0400005830 comp rx=0 tx=0).stop 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f041e800640 1 -- 172.21.15.37:0/1880030501 reap_dead start 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f0420800640 1 -- 172.21.15.37:0/1880030501 shutdown_connections 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f0420800640 1 -- 172.21.15.37:0/1880030501 >> 172.21.15.37:0/1880030501 conn(0x7f04180b1b90 msgr2=0x7f04180b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.213+0000 7f0420800640 1 -- 172.21.15.37:0/1880030501 shutdown_connections 2024-09-21T06:31:51.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.214+0000 7f0420800640 1 -- 172.21.15.37:0/1880030501 wait complete. 2024-09-21T06:31:51.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.216+0000 7f0420800640 1 Processor -- start 2024-09-21T06:31:51.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.216+0000 7f0420800640 1 -- start start 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0400003c10 0x7f04180bb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04181f8550 0x7f04181f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0418073320 con 0x7f04181f8960 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f04180731a0 con 0x7f0400003c10 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.217+0000 7f0420800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f04180734a0 con 0x7f04181f8550 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56172/0 (socket says 172.21.15.37:56172) 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 -- 172.21.15.37:0/1956277729 learned_addr learned my addr 172.21.15.37:0/1956277729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041de00640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0400003c10 0x7f04180bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041d400640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04181f8550 0x7f04181f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 -- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04181f8550 msgr2=0x7f04181f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04181f8550 0x7f04181f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 -- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0400003c10 msgr2=0x7f04180bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0400003c10 0x7f04180bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.218+0000 7f041e800640 1 -- 172.21.15.37:0/1956277729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181f9160 con 0x7f04181f8960 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f041e800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f041806fe80 tx=0x7f0414010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f0412000640 1 -- 172.21.15.37:0/1956277729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414003970 con 0x7f04181f8960 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f0412000640 1 -- 172.21.15.37:0/1956277729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0414015070 con 0x7f04181f8960 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f0412000640 1 -- 172.21.15.37:0/1956277729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414004030 con 0x7f04181f8960 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181f9440 con 0x7f04181f8960 2024-09-21T06:31:51.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.219+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04180bae60 con 0x7f04181f8960 2024-09-21T06:31:51.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.221+0000 7f0412000640 1 -- 172.21.15.37:0/1956277729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04140041d0 con 0x7f04181f8960 2024-09-21T06:31:51.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.222+0000 7f0412000640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 0x7f03fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.223+0000 7f0412000640 1 -- 172.21.15.37:0/1956277729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f041409b9a0 con 0x7f04181f8960 2024-09-21T06:31:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.223+0000 7f041de00640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 0x7f03fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.223+0000 7f042784f740 1 Processor -- start 2024-09-21T06:31:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.223+0000 7f042784f740 1 -- start start 2024-09-21T06:31:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f041de00640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 0x7f03fc07b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c032000 tx=0x7f040c001360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d3476970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d3476eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8d33d0430 con 0x55d8d3085590 2024-09-21T06:31:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8d33c05d0 con 0x55d8d30a6270 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.224+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d8d33afa00 con 0x55d8d307d3b0 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56188/0 (socket says 172.21.15.37:56188) 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 -- 172.21.15.37:0/597001488 learned_addr learned my addr 172.21.15.37:0/597001488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041d400640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d3476eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041de00640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d3476970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 -- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 msgr2=0x55d8d3476eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d3476eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 -- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 msgr2=0x55d8d3476970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d3476970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041e800640 1 -- 172.21.15.37:0/597001488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d3378520 con 0x55d8d3085590 2024-09-21T06:31:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.225+0000 7f041de00640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d3476970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f041e800640 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0414063580 tx=0x7f041405f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e268f2ce269953ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f0406600640 1 -- 172.21.15.37:0/597001488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414021030 con 0x55d8d3085590 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f0406600640 1 -- 172.21.15.37:0/597001488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04140635f0 con 0x55d8d3085590 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f0406600640 1 -- 172.21.15.37:0/597001488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041406a450 con 0x55d8d3085590 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f042784f740 1 -- 172.21.15.37:0/597001488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 msgr2=0x55d8d3446f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.226+0000 7f042784f740 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0414063580 tx=0x7f041405f590 comp rx=0 tx=0).stop 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 -- 172.21.15.37:0/597001488 shutdown_connections 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d3446f60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d3476eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 --2- 172.21.15.37:0/597001488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d3476970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 -- 172.21.15.37:0/597001488 >> 172.21.15.37:0/597001488 conn(0x55d8d309a3c0 msgr2=0x55d8d344c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 -- 172.21.15.37:0/597001488 shutdown_connections 2024-09-21T06:31:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.227+0000 7f042784f740 1 -- 172.21.15.37:0/597001488 wait complete. 2024-09-21T06:31:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.228+0000 7f042784f740 1 -- start start 2024-09-21T06:31:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.228+0000 7f042784f740 1 Processor -- start 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d348b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d34a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8d3356330 con 0x55d8d3085590 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8d33b3810 con 0x55d8d30a6270 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d8d3120620 con 0x55d8d307d3b0 2024-09-21T06:31:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.229+0000 7f042784f740 10 client.0 _ll_register_callbacks cb 0x55d8d335b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56198/0 (socket says 172.21.15.37:56198) 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 -- 172.21.15.37:0/338276636 learned_addr learned my addr 172.21.15.37:0/338276636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041de00640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d348b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041e800640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d34a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 -- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 msgr2=0x55d8d348b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d348b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 -- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 msgr2=0x55d8d34a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d34a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.230+0000 7f041d400640 1 -- 172.21.15.37:0/338276636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d3111850 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f041d400640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0408000fd0 tx=0x7f0408013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8793dab059a79d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040801c640 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04140635f0 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f040801c7e0 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f0408024050 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f042784f740 10 client.6453 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55d8d31886b0 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04140635f0 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.231+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7f0408027020 con 0x55d8d3085590 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 10 client.6453 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 10 client.6453 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 20 client.6453 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d3064e40 0x55d8d3472450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 4 client.6453 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:31:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8d33c2520 con 0x55d8d3064e40 2024-09-21T06:31:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f041de00640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d3064e40 0x55d8d3472450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f041de00640 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d3064e40 0x55d8d3472450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc5f4e63b8d9008 server_cookie=5b52a918b26e1080 in_seq=0 out_seq=0 2024-09-21T06:31:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.232+0000 7f03faa00640 10 client.6453 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.233+0000 7f03faa00640 1 -- 172.21.15.37:0/338276636 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8d33c2520 con 0x55d8d3064e40 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.233+0000 7f03faa00640 10 client.6453 handle_command_reply: tid=0 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.233+0000 7f042784f740 1 client.6453 shutdown 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 msgr2=0x55d8d34a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0408000fd0 tx=0x7f0408013820 comp rx=0 tx=0).stop 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 shutdown_connections 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d3064e40 0x55d8d3472450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d3085590 0x55d8d34a4660 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d30a6270 0x55d8d34a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 --2- 172.21.15.37:0/338276636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d307d3b0 0x55d8d348b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f03faa00640 0 client.6453 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 >> 172.21.15.37:0/338276636 conn(0x55d8d309a3c0 msgr2=0x55d8d3491fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 shutdown_connections 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 1 -- 172.21.15.37:0/338276636 wait complete. 2024-09-21T06:31:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.234+0000 7f042784f740 20 client.6453 trim_cache size 0 max 16384 2024-09-21T06:31:51.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.253+0000 7f042784f740 1 Processor -- start 2024-09-21T06:31:51.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.253+0000 7f042784f740 1 -- start start 2024-09-21T06:31:51.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 0x55d8d3447c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 0x55d8d34481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0418201de0 con 0x55d8d30a6270 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f04182024b0 con 0x55d8d307d3b0 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f04180c7e50 con 0x55d8d3085590 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56210/0 (socket says 172.21.15.37:56210) 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 -- 172.21.15.37:0/3726708933 learned_addr learned my addr 172.21.15.37:0/3726708933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041de00640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 0x55d8d3447c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041d400640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 0x55d8d34481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 -- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 msgr2=0x55d8d34481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 0x55d8d34481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 -- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 msgr2=0x55d8d3447c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 0x55d8d3447c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041e800640 1 -- 172.21.15.37:0/3726708933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d344d650 con 0x55d8d30a6270 2024-09-21T06:31:51.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041de00640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 0x55d8d3447c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:51.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.255+0000 7f041d400640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 0x55d8d34481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:31:51.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f041e800640 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0414063580 tx=0x7f0414099b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b870b4cbfd1b7159 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f0405c00640 1 -- 172.21.15.37:0/3726708933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414021040 con 0x55d8d30a6270 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f0405c00640 1 -- 172.21.15.37:0/3726708933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0414068020 con 0x55d8d30a6270 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f0405c00640 1 -- 172.21.15.37:0/3726708933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041406a450 con 0x55d8d30a6270 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f042784f740 1 -- 172.21.15.37:0/3726708933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 msgr2=0x55d8d34b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.256+0000 7f042784f740 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0414063580 tx=0x7f0414099b70 comp rx=0 tx=0).stop 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 -- 172.21.15.37:0/3726708933 shutdown_connections 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d30a6270 0x55d8d34b4b00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d3085590 0x55d8d34481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 --2- 172.21.15.37:0/3726708933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d307d3b0 0x55d8d3447c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 -- 172.21.15.37:0/3726708933 >> 172.21.15.37:0/3726708933 conn(0x55d8d309a3c0 msgr2=0x55d8d344cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 -- 172.21.15.37:0/3726708933 shutdown_connections 2024-09-21T06:31:51.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.257+0000 7f042784f740 1 -- 172.21.15.37:0/3726708933 wait complete. 2024-09-21T06:31:51.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.258+0000 7f042784f740 1 -- start start 2024-09-21T06:31:51.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.258+0000 7f042784f740 1 Processor -- start 2024-09-21T06:31:51.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d3085590 0x55d8d3447c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d30a6270 0x55d8d34481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8d346fb60 con 0x55d8d307d3b0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8d3473360 con 0x55d8d3085590 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d8d34701b0 con 0x55d8d30a6270 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.259+0000 7f042784f740 10 client.0 _ll_register_callbacks cb 0x7f04080048d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56216/0 (socket says 172.21.15.37:56216) 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 -- 172.21.15.37:0/257107829 learned_addr learned my addr 172.21.15.37:0/257107829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041e800640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d30a6270 0x55d8d34481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041d400640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d3085590 0x55d8d3447c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 -- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d30a6270 msgr2=0x55d8d34481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d30a6270 0x55d8d34481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 -- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d3085590 msgr2=0x55d8d3447c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d3085590 0x55d8d3447c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 -- 172.21.15.37:0/257107829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d3495870 con 0x55d8d307d3b0 2024-09-21T06:31:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f041de00640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f040c001830 tx=0x7f040c006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf9e6d32d5e64be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.260+0000 7f03faa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c052030 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f040c003de0 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0414068020 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f040c045040 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 10 client.6459 fetch_fsmap learned FSMap version 57 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55d8d3516250 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f040c059030 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 57) ==== 3542+0+0 (secure 0 0 0) 0x7f040c04b070 con 0x55d8d307d3b0 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 10 client.6459 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T06:31:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 10 client.6459 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 20 client.6459 populate_metadata read hostname 'smithi037' 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d34881a0 0x55d8d3515000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 4 client.6459 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.261+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "status"}) -- 0x55d8d33bdfd0 con 0x55d8d34881a0 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f041d400640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d34881a0 0x55d8d3515000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f03faa00640 10 client.6459 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f041d400640 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d34881a0 0x55d8d3515000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22549d2b38e315f1 server_cookie=39fd8635fef72c64 in_seq=0 out_seq=0 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f03faa00640 1 -- 172.21.15.37:0/257107829 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d8d33bdfd0 con 0x55d8d34881a0 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f03faa00640 10 client.6459 handle_command_reply: tid=0 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.262+0000 7f042784f740 1 client.6459 shutdown 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 msgr2=0x55d8d3494380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f040c001830 tx=0x7f040c006b10 comp rx=0 tx=0).stop 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 shutdown_connections 2024-09-21T06:31:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55d8d34881a0 0x55d8d3515000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8d30a6270 0x55d8d34481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f03faa00640 0 client.6459 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8d3085590 0x55d8d3447c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 --2- 172.21.15.37:0/257107829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8d307d3b0 0x55d8d3494380 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 >> 172.21.15.37:0/257107829 conn(0x55d8d309a3c0 msgr2=0x55d8d344c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 shutdown_connections 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 1 -- 172.21.15.37:0/257107829 wait complete. 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.263+0000 7f042784f740 20 client.6459 trim_cache size 0 max 16384 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:31:51.268 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6106, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 12.80415951, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 57, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 67, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 67, 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.777513129999999 2024-09-21T06:31:51.269 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 msgr2=0x7f03fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 0x7f03fc07b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c032000 tx=0x7f040c001360 comp rx=0 tx=0).stop 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 msgr2=0x7f04181f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f041806fe80 tx=0x7f0414010f40 comp rx=0 tx=0).stop 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 shutdown_connections 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f03fc078b90 0x7f03fc07b0a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04181f8960 0x7f04181f54a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04181f8550 0x7f04181f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 --2- 172.21.15.37:0/1956277729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0400003c10 0x7f04180bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 >> 172.21.15.37:0/1956277729 conn(0x7f04180b1b90 msgr2=0x7f0418070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 shutdown_connections 2024-09-21T06:31:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.264+0000 7f0420800640 1 -- 172.21.15.37:0/1956277729 wait complete. 2024-09-21T06:31:51.291 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:31:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 -- 172.21.15.37:0/860041406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6954073dd0 msgr2=0x7f695406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 --2- 172.21.15.37:0/860041406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6954073dd0 0x7f695406ddf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6940009830 tx=0x7f694002a4a0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 -- 172.21.15.37:0/860041406 shutdown_connections 2024-09-21T06:31:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 --2- 172.21.15.37:0/860041406 >> v2:172.21.15.150:3300/0 conn(0x7f695406e330 0x7f695406ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 --2- 172.21.15.37:0/860041406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6954073dd0 0x7f695406ddf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 -- 172.21.15.37:0/860041406 >> 172.21.15.37:0/860041406 conn(0x7f69540cca40 msgr2=0x7f69540cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 -- 172.21.15.37:0/860041406 shutdown_connections 2024-09-21T06:31:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.516+0000 7f695b800640 1 -- 172.21.15.37:0/860041406 wait complete. 2024-09-21T06:31:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.518+0000 7f695b800640 1 Processor -- start 2024-09-21T06:31:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.518+0000 7f695b800640 1 -- start start 2024-09-21T06:31:51.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f695406e330 0x7f6954158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 0x7f69541e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f695406f4a0 con 0x7f6954074cb0 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f695406f320 con 0x7f6954073dd0 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f695b800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f695406f620 con 0x7f695406e330 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f6959800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f6959800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56232/0 (socket says 172.21.15.37:56232) 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f6959800640 1 -- 172.21.15.37:0/4039956811 learned_addr learned my addr 172.21.15.37:0/4039956811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f6958e00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f695406e330 0x7f6954158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.520+0000 7f6953e00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 0x7f69541e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 -- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f695406e330 msgr2=0x7f6954158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f695406e330 0x7f6954158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 -- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 msgr2=0x7f69541e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 0x7f69541e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 -- 172.21.15.37:0/4039956811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540cdf50 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6953e00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 0x7f69541e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6959800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f694c00ea80 tx=0x7f694c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c004b20 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f694c004ce0 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69540ce230 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.521+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c005280 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.522+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69540ce7f0 con 0x7f6954074cb0 2024-09-21T06:31:51.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.522+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69540cea30 con 0x7f6954074cb0 2024-09-21T06:31:51.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.524+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f694c005420 con 0x7f6954074cb0 2024-09-21T06:31:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.525+0000 7f6950c00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 0x7f693407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.525+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f694c09bca0 con 0x7f6954074cb0 2024-09-21T06:31:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.525+0000 7f6958e00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 0x7f693407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.526+0000 7f6958e00640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 0x7f693407b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6948001040 tx=0x7f6948000fd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.529+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69540cea30 con 0x7f6954074cb0 2024-09-21T06:31:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.765+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f695406fb60 con 0x7f6954074cb0 2024-09-21T06:31:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.765+0000 7f6950c00640 1 -- 172.21.15.37:0/4039956811 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5268 (secure 0 0 0) 0x7f694c063e30 con 0x7f6954074cb0 2024-09-21T06:31:51.768 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:31:51.768 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"btime":"2024-09-21T06:31:49:535251+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:31:49.535243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:31:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 57 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 msgr2=0x7f693407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 0x7f693407b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6948001040 tx=0x7f6948000fd0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 msgr2=0x7f69540b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f694c00ea80 tx=0x7f694c00ef40 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 shutdown_connections 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6934078b90 0x7f693407b0a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6954074cb0 0x7f69540b0760 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6954073dd0 0x7f69541e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 --2- 172.21.15.37:0/4039956811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f695406e330 0x7f6954158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 >> 172.21.15.37:0/4039956811 conn(0x7f69540cca40 msgr2=0x7f695415bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 shutdown_connections 2024-09-21T06:31:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:31:51.768+0000 7f695b800640 1 -- 172.21.15.37:0/4039956811 wait complete. 2024-09-21T06:31:51.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:51.795 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:31:51.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:51.843 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:31:51.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:51.890 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:31:51.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:51.972 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:31:52.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:31:52.053 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:31:52.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:31:52.415 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:31:52.415 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:31:52.416 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:31:54.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:31:54.677 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:31:54.678 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:31:54.678 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:31:56.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:31:56.938 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:31:56.938 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:31:58.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:31:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:31:59.196 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:31:59.196 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:31:59.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:31:59.442 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:31:59.443 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:31:59.443 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:32:01.444 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:32:01.694 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:32:01.694 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:32:01.694 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:32:03.696 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:32:03.943 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:32:03.944 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:32:03.944 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:32:05.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:32:06.192 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T06:32:06.193 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:32:06.193 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:32:06.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:32:06.433 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:32:06.433 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:32:06.433 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:32:06.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-21T06:32:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 -- 172.21.15.37:0/2235841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540aef90 msgr2=0x7fa6540af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 --2- 172.21.15.37:0/2235841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540aef90 0x7fa6540af3a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa640009830 tx=0x7fa64002a4d0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 -- 172.21.15.37:0/2235841220 shutdown_connections 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 --2- 172.21.15.37:0/2235841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540aef90 0x7fa6540af3a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 --2- 172.21.15.37:0/2235841220 >> v2:172.21.15.150:3300/0 conn(0x7fa65406cde0 0x7fa654155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 -- 172.21.15.37:0/2235841220 >> 172.21.15.37:0/2235841220 conn(0x7fa65406f920 msgr2=0x7fa6540b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 -- 172.21.15.37:0/2235841220 shutdown_connections 2024-09-21T06:32:06.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.672+0000 7fa65a600640 1 -- 172.21.15.37:0/2235841220 wait complete. 2024-09-21T06:32:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.674+0000 7fa65a600640 1 Processor -- start 2024-09-21T06:32:06.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.674+0000 7fa65a600640 1 -- start start 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa65406cde0 0x7fa654147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6540aef90 0x7fa654144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa65415f490 con 0x7fa6540cc8d0 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa65415f310 con 0x7fa65406cde0 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa65a600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa65415f610 con 0x7fa6540aef90 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa659c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa659c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50392/0 (socket says 172.21.15.37:50392) 2024-09-21T06:32:06.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa659c00640 1 -- 172.21.15.37:0/2417634770 learned_addr learned my addr 172.21.15.37:0/2417634770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa659200640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa65406cde0 0x7fa654147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.676+0000 7fa653e00640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6540aef90 0x7fa654144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 -- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6540aef90 msgr2=0x7fa654144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6540aef90 0x7fa654144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 -- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa65406cde0 msgr2=0x7fa654147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa65406cde0 0x7fa654147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 -- 172.21.15.37:0/2417634770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6541f8bf0 con 0x7fa6540cc8d0 2024-09-21T06:32:06.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa659c00640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c760 tx=0x7fa64c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa650c00640 1 -- 172.21.15.37:0/2417634770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c017070 con 0x7fa6540cc8d0 2024-09-21T06:32:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa650c00640 1 -- 172.21.15.37:0/2417634770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64c00ce10 con 0x7fa6540cc8d0 2024-09-21T06:32:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa650c00640 1 -- 172.21.15.37:0/2417634770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0052b0 con 0x7fa6540cc8d0 2024-09-21T06:32:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6541f8ed0 con 0x7fa6540cc8d0 2024-09-21T06:32:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.677+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6541f9490 con 0x7fa6540cc8d0 2024-09-21T06:32:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.680+0000 7fa650c00640 1 -- 172.21.15.37:0/2417634770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa64c01a070 con 0x7fa6540cc8d0 2024-09-21T06:32:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.681+0000 7fa650c00640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 0x7fa63407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.681+0000 7fa650c00640 1 -- 172.21.15.37:0/2417634770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa64c09a520 con 0x7fa6540cc8d0 2024-09-21T06:32:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.681+0000 7fa662c84740 1 Processor -- start 2024-09-21T06:32:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.681+0000 7fa662c84740 1 -- start start 2024-09-21T06:32:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.681+0000 7fa659200640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 0x7fa63407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.682+0000 7fa659200640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 0x7fa63407b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa648009b30 tx=0x7fa648005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e6740730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7e66591c0 con 0x55a7e63316b0 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a7e666c870 con 0x55a7e63294d0 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a7e665bca0 con 0x55a7e6352390 2024-09-21T06:32:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa659c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa659c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50398/0 (socket says 172.21.15.37:50398) 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.683+0000 7fa659c00640 1 -- 172.21.15.37:0/1940782874 learned_addr learned my addr 172.21.15.37:0/1940782874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659200640 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e6740730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa653e00640 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 -- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 msgr2=0x55a7e6740730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e6740730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 -- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 msgr2=0x55a7e6723400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 -- 172.21.15.37:0/1940782874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e64347d0 con 0x55a7e63316b0 2024-09-21T06:32:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa659c00640 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa64c061b00 tx=0x7fa64c061910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64ef341394fefb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa645600640 1 -- 172.21.15.37:0/1940782874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c06ba50 con 0x55a7e63316b0 2024-09-21T06:32:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa645600640 1 -- 172.21.15.37:0/1940782874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64c09a0c0 con 0x55a7e63316b0 2024-09-21T06:32:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.684+0000 7fa645600640 1 -- 172.21.15.37:0/1940782874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c06b5c0 con 0x55a7e63316b0 2024-09-21T06:32:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 -- 172.21.15.37:0/1940782874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 msgr2=0x55a7e6723940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa64c061b00 tx=0x7fa64c061910 comp rx=0 tx=0).stop 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 -- 172.21.15.37:0/1940782874 shutdown_connections 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e6723940 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 --2- 172.21.15.37:0/1940782874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e6740730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 -- 172.21.15.37:0/1940782874 >> 172.21.15.37:0/1940782874 conn(0x55a7e63464e0 msgr2=0x55a7e66f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 -- 172.21.15.37:0/1940782874 shutdown_connections 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.685+0000 7fa662c84740 1 -- 172.21.15.37:0/1940782874 wait complete. 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.686+0000 7fa662c84740 1 -- start start 2024-09-21T06:32:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.686+0000 7fa662c84740 1 Processor -- start 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e6711b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e673b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7e6602410 con 0x55a7e63316b0 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a7e665fab0 con 0x55a7e6352390 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a7e63cc740 con 0x55a7e63294d0 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa662c84740 10 client.0 _ll_register_callbacks cb 0x55a7e65df360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa653e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa653e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50410/0 (socket says 172.21.15.37:50410) 2024-09-21T06:32:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.688+0000 7fa653e00640 1 -- 172.21.15.37:0/88789783 learned_addr learned my addr 172.21.15.37:0/88789783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa659200640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e6711b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa659c00640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e673b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 -- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 msgr2=0x55a7e6711b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e6711b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 -- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 msgr2=0x55a7e673b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e673b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 -- 172.21.15.37:0/88789783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e63bd970 con 0x55a7e63316b0 2024-09-21T06:32:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa653e00640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa640009950 tx=0x7fa6400045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9fa28f96572a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa63a000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400039c0 con 0x55a7e63316b0 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa640003b60 con 0x55a7e63316b0 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64c09a0c0 con 0x55a7e63316b0 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.689+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa640003d60 con 0x55a7e63316b0 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 10 client.6480 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55a7e639b6a0 con 0x55a7e63316b0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64c09a0c0 con 0x55a7e63316b0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7fa640005020 con 0x55a7e63316b0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 10 client.6480 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 10 client.6480 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 20 client.6480 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6310f60 0x55a7e673c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 4 client.6480 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e666e7c0 con 0x55a7e6310f60 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.690+0000 7fa659200640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6310f60 0x55a7e673c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.691+0000 7fa659200640 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6310f60 0x55a7e673c0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f08446d337505 server_cookie=fb361405c9e4af8d in_seq=0 out_seq=0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.691+0000 7fa63a000640 10 client.6480 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa63a000640 1 -- 172.21.15.37:0/88789783 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7e666e7c0 con 0x55a7e6310f60 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa63a000640 10 client.6480 handle_command_reply: tid=0 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 client.6480 shutdown 2024-09-21T06:32:06.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 msgr2=0x55a7e673b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa640009950 tx=0x7fa6400045d0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 shutdown_connections 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6310f60 0x55a7e673c0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e673b000 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e673b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 --2- 172.21.15.37:0/88789783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e6711b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa63a000640 0 client.6480 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 >> 172.21.15.37:0/88789783 conn(0x55a7e63464e0 msgr2=0x55a7e6723400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 shutdown_connections 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 1 -- 172.21.15.37:0/88789783 wait complete. 2024-09-21T06:32:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.692+0000 7fa662c84740 20 client.6480 trim_cache size 0 max 16384 2024-09-21T06:32:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.708+0000 7fa662c84740 1 Processor -- start 2024-09-21T06:32:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.708+0000 7fa662c84740 1 -- start start 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e66f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa654202570 con 0x55a7e63316b0 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa654201760 con 0x55a7e63294d0 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.709+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa654201e40 con 0x55a7e6352390 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50412/0 (socket says 172.21.15.37:50412) 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 -- 172.21.15.37:0/3942214692 learned_addr learned my addr 172.21.15.37:0/3942214692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa659c00640 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e66f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa659200640 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 -- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 msgr2=0x55a7e66f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e66f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 -- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 msgr2=0x55a7e6723a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 -- 172.21.15.37:0/3942214692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e673d3a0 con 0x55a7e63316b0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.710+0000 7fa653e00640 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa640009780 tx=0x7fa640005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e469c84ea817ce9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa644c00640 1 -- 172.21.15.37:0/3942214692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400039c0 con 0x55a7e63316b0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa644c00640 1 -- 172.21.15.37:0/3942214692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa640046470 con 0x55a7e63316b0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa644c00640 1 -- 172.21.15.37:0/3942214692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640046860 con 0x55a7e63316b0 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 -- 172.21.15.37:0/3942214692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 msgr2=0x55a7e66f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa640009780 tx=0x7fa640005f10 comp rx=0 tx=0).stop 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 -- 172.21.15.37:0/3942214692 shutdown_connections 2024-09-21T06:32:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e66f3d30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e6352390 0x55a7e66f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 --2- 172.21.15.37:0/3942214692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e63294d0 0x55a7e6723a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.711+0000 7fa662c84740 1 -- 172.21.15.37:0/3942214692 >> 172.21.15.37:0/3942214692 conn(0x55a7e63464e0 msgr2=0x55a7e673db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.712+0000 7fa662c84740 1 -- 172.21.15.37:0/3942214692 shutdown_connections 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.712+0000 7fa662c84740 1 -- 172.21.15.37:0/3942214692 wait complete. 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.712+0000 7fa662c84740 1 -- start start 2024-09-21T06:32:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.713+0000 7fa662c84740 1 Processor -- start 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e674c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e674c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541584b0 con 0x55a7e63316b0 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a7e66fae30 con 0x55a7e6352390 2024-09-21T06:32:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a7e67bd820 con 0x55a7e63294d0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa662c84740 10 client.0 _ll_register_callbacks cb 0x7fa6400096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa653e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e674c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.714+0000 7fa659200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:53274/0 (socket says 172.21.15.37:53274) 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 -- 172.21.15.37:0/904023019 learned_addr learned my addr 172.21.15.37:0/904023019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659c00640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e674c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 -- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 msgr2=0x55a7e674c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e674c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 -- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 msgr2=0x55a7e674c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e674c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 -- 172.21.15.37:0/904023019 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e674cc50 con 0x55a7e63294d0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa659200640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa648004160 tx=0x7fa648013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839ef83764fa3773 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.715+0000 7fa63a000640 10 client.0 ms_handle_connect on v2:172.21.15.150:3301/0 2024-09-21T06:32:06.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648003a80 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mon.2 v2:172.21.15.150:3301/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa648003c20 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa640046470 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa648038050 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 10 client.5861 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55a7e67619a0 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mon.2 v2:172.21.15.150:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa640046470 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mon.2 v2:172.21.15.150:3301/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7fa64803b020 con 0x55a7e63294d0 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 10 client.5861 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 10 client.5861 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.716+0000 7fa662c84740 20 client.5861 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6720180 0x55a7e6760b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa662c84740 4 client.5861 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7e666a270 con 0x55a7e6720180 2024-09-21T06:32:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa653e00640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6720180 0x55a7e6760b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa653e00640 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6720180 0x55a7e6760b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f338103dfe9bf68 server_cookie=68584151d2f14955 in_seq=0 out_seq=0 2024-09-21T06:32:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.717+0000 7fa63a000640 10 client.5861 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:06.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa63a000640 1 -- 172.21.15.37:0/904023019 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7e666a270 con 0x55a7e6720180 2024-09-21T06:32:06.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa63a000640 10 client.5861 handle_command_reply: tid=0 2024-09-21T06:32:06.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 client.5861 shutdown 2024-09-21T06:32:06.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 msgr2=0x55a7e67bcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa648004160 tx=0x7fa648013040 comp rx=0 tx=0).stop 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 shutdown_connections 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55a7e6720180 0x55a7e6760b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7e63316b0 0x55a7e674c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a7e6352390 0x55a7e674c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 --2- 172.21.15.37:0/904023019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a7e63294d0 0x55a7e67bcbd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa63a000640 0 client.5861 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 >> 172.21.15.37:0/904023019 conn(0x55a7e63464e0 msgr2=0x55a7e66f3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 shutdown_connections 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 1 -- 172.21.15.37:0/904023019 wait complete. 2024-09-21T06:32:06.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.735+0000 7fa662c84740 20 client.5861 trim_cache size 0 max 16384 2024-09-21T06:32:06.739 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:06.739 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:32:06.739 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:32:06.739 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:06.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 msgr2=0x7fa63407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 0x7fa63407b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa648009b30 tx=0x7fa648005f90 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 msgr2=0x7fa6540b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c760 tx=0x7fa64c00cc20 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 shutdown_connections 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa634078b90 0x7fa63407b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6540cc8d0 0x7fa6540b3b70 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6540aef90 0x7fa654144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 --2- 172.21.15.37:0/2417634770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa65406cde0 0x7fa654147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 >> 172.21.15.37:0/2417634770 conn(0x7fa65406f920 msgr2=0x7fa654073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 shutdown_connections 2024-09-21T06:32:06.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.736+0000 7fa65a600640 1 -- 172.21.15.37:0/2417634770 wait complete. 2024-09-21T06:32:06.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 -- 172.21.15.37:0/3673276341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b41456e0 msgr2=0x7f32b40b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 --2- 172.21.15.37:0/3673276341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b41456e0 0x7f32b40b4f90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b3470 tx=0x7f32a402a560 comp rx=0 tx=0).stop 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 -- 172.21.15.37:0/3673276341 shutdown_connections 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 --2- 172.21.15.37:0/3673276341 >> v2:172.21.15.150:3300/0 conn(0x7f32b40cc8d0 0x7f32b40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 --2- 172.21.15.37:0/3673276341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b41456e0 0x7f32b40b4f90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 -- 172.21.15.37:0/3673276341 >> 172.21.15.37:0/3673276341 conn(0x7f32b406f920 msgr2=0x7f32b40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 -- 172.21.15.37:0/3673276341 shutdown_connections 2024-09-21T06:32:06.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.988+0000 7f32b9c00640 1 -- 172.21.15.37:0/3673276341 wait complete. 2024-09-21T06:32:06.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.990+0000 7f32b9c00640 1 Processor -- start 2024-09-21T06:32:06.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.990+0000 7f32b9c00640 1 -- start start 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b4142910 0x7f32b415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32b41456e0 0x7f32b41e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40ce150 con 0x7f32b40cc8d0 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40cdfd0 con 0x7f32b4142910 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f32b40ce2d0 con 0x7f32b41456e0 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50442/0 (socket says 172.21.15.37:50442) 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b3e00640 1 -- 172.21.15.37:0/4049900275 learned_addr learned my addr 172.21.15.37:0/4049900275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:06.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32b9200640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32b41456e0 0x7f32b41e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.992+0000 7f32abe00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b4142910 0x7f32b415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 -- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32b41456e0 msgr2=0x7f32b41e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32b41456e0 0x7f32b41e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 -- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b4142910 msgr2=0x7f32b415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b4142910 0x7f32b415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 -- 172.21.15.37:0/4049900275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b413c630 con 0x7f32b40cc8d0 2024-09-21T06:32:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b3e00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034040 tx=0x7f32a4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:06.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b0c00640 1 -- 172.21.15.37:0/4049900275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a403e030 con 0x7f32b40cc8d0 2024-09-21T06:32:06.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b0c00640 1 -- 172.21.15.37:0/4049900275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4003e00 con 0x7f32b40cc8d0 2024-09-21T06:32:06.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b0c00640 1 -- 172.21.15.37:0/4049900275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4035040 con 0x7f32b40cc8d0 2024-09-21T06:32:06.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.993+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b4073e60 con 0x7f32b40cc8d0 2024-09-21T06:32:06.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.994+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b4074420 con 0x7f32b40cc8d0 2024-09-21T06:32:06.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.996+0000 7f32b0c00640 1 -- 172.21.15.37:0/4049900275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f32a4041030 con 0x7f32b40cc8d0 2024-09-21T06:32:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.997+0000 7f32b0c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 0x7f329007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.997+0000 7f32b0c00640 1 -- 172.21.15.37:0/4049900275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f32a40bb6f0 con 0x7f32b40cc8d0 2024-09-21T06:32:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.997+0000 7f32c2419740 1 Processor -- start 2024-09-21T06:32:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.997+0000 7f32abe00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 0x7f329007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.998+0000 7f32c2419740 1 -- start start 2024-09-21T06:32:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.998+0000 7f32abe00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 0x7f329007b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32b4144de0 tx=0x7f329c009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x5588724ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x558872490400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5588723c61c0 con 0x55887209e6b0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5588723d9870 con 0x5588720964d0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5588723c8ca0 con 0x5588720bf390 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32b9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32b9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50454/0 (socket says 172.21.15.37:50454) 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:06.999+0000 7f32b9200640 1 -- 172.21.15.37:0/3624159706 learned_addr learned my addr 172.21.15.37:0/3624159706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b3e00640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x5588724ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32abe00640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x558872490400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 -- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 msgr2=0x5588724ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x5588724ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 -- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 msgr2=0x558872490400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x558872490400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 -- 172.21.15.37:0/3624159706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588721a17d0 con 0x55887209e6b0 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b3e00640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x5588724ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32b9200640 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009f80 tx=0x7f32ac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e9493883eadd15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32a8c00640 1 -- 172.21.15.37:0/3624159706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac016070 con 0x55887209e6b0 2024-09-21T06:32:07.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32a8c00640 1 -- 172.21.15.37:0/3624159706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32ac004e10 con 0x55887209e6b0 2024-09-21T06:32:07.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.000+0000 7f32a8c00640 1 -- 172.21.15.37:0/3624159706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac012750 con 0x55887209e6b0 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 -- 172.21.15.37:0/3624159706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 msgr2=0x558872490940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009f80 tx=0x7f32ac010800 comp rx=0 tx=0).stop 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 -- 172.21.15.37:0/3624159706 shutdown_connections 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x558872490940 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x558872490400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 --2- 172.21.15.37:0/3624159706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x5588724ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 -- 172.21.15.37:0/3624159706 >> 172.21.15.37:0/3624159706 conn(0x5588720b34e0 msgr2=0x558872466620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 -- 172.21.15.37:0/3624159706 shutdown_connections 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.001+0000 7f32c2419740 1 -- 172.21.15.37:0/3624159706 wait complete. 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.002+0000 7f32c2419740 1 -- start start 2024-09-21T06:32:07.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.003+0000 7f32c2419740 1 Processor -- start 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55887209e6b0 0x55887247e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588720bf390 0x5588724aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55887236f410 con 0x5588720bf390 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5588723ccab0 con 0x55887209e6b0 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558872139740 con 0x5588720964d0 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.004+0000 7f32c2419740 10 client.0 _ll_register_callbacks cb 0x55887234c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32abe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55887209e6b0 0x55887247e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:53314/0 (socket says 172.21.15.37:53314) 2024-09-21T06:32:07.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588720bf390 0x5588724aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 -- 172.21.15.37:0/2301181061 learned_addr learned my addr 172.21.15.37:0/2301181061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 -- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55887209e6b0 msgr2=0x55887247e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55887209e6b0 0x55887247e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 -- 172.21.15.37:0/2301181061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588720bf390 msgr2=0x5588724aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588720bf390 0x5588724aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 -- 172.21.15.37:0/2301181061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55887212a970 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32b3e00640 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4082cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894dbcb3f80b006a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32a1600640 10 client.0 ms_handle_connect on v2:172.21.15.150:3301/0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.005+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40a9070 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mon.2 v2:172.21.15.150:3301/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f32a4082510 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ac004e10 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f32a4048040 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32c2419740 10 client.5870 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5588721086a0 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.006+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mon.2 v2:172.21.15.150:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32ac004e10 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mon.2 v2:172.21.15.150:3301/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7f32a407ec00 con 0x5588720964d0 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 10 client.5870 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 10 client.5870 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.d 2024-09-21T06:32:07.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 20 client.5870 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:07.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x55887207df60 0x5588724aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 4 client.5870 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:07.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588723db7c0 con 0x55887207df60 2024-09-21T06:32:07.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.007+0000 7f32abe00640 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x55887207df60 0x5588724aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.008+0000 7f32abe00640 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x55887207df60 0x5588724aaa40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c82822fb65842d00 server_cookie=391fe7295683cc64 in_seq=0 out_seq=0 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.008+0000 7f32a1600640 10 client.5870 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32a1600640 1 -- 172.21.15.37:0/2301181061 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588723db7c0 con 0x55887207df60 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32a1600640 10 client.5870 handle_command_reply: tid=0 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 client.5870 shutdown 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 msgr2=0x5588724a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4082cc0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 shutdown_connections 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x55887207df60 0x5588724aaa40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588720bf390 0x5588724aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55887209e6b0 0x55887247e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32a1600640 0 client.5870 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 --2- 172.21.15.37:0/2301181061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x5588724a4970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 >> 172.21.15.37:0/2301181061 conn(0x5588720b34e0 msgr2=0x5588724ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 shutdown_connections 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 1 -- 172.21.15.37:0/2301181061 wait complete. 2024-09-21T06:32:07.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.009+0000 7f32c2419740 20 client.5870 trim_cache size 0 max 16384 2024-09-21T06:32:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.025+0000 7f32c2419740 1 Processor -- start 2024-09-21T06:32:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.025+0000 7f32c2419740 1 -- start start 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x5588724a4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x55887245fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4202610 con 0x55887209e6b0 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32b42017e0 con 0x5588720964d0 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.026+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f32b4201ee0 con 0x5588720bf390 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50484/0 (socket says 172.21.15.37:50484) 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x5588724a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 -- 172.21.15.37:0/3633399697 learned_addr learned my addr 172.21.15.37:0/3633399697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32b9200640 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x55887245fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 -- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 msgr2=0x55887245fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x55887245fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 -- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 msgr2=0x5588724a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x5588724a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.027+0000 7f32abe00640 1 -- 172.21.15.37:0/3633399697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588724ab260 con 0x55887209e6b0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32abe00640 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590045085d68afd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32a3e00640 1 -- 172.21.15.37:0/3633399697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c01b040 con 0x55887209e6b0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32a3e00640 1 -- 172.21.15.37:0/3633399697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f329c00ee80 con 0x55887209e6b0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32a3e00640 1 -- 172.21.15.37:0/3633399697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c004260 con 0x55887209e6b0 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32c2419740 1 -- 172.21.15.37:0/3633399697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 msgr2=0x55887245f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.028+0000 7f32c2419740 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c015000 comp rx=0 tx=0).stop 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 -- 172.21.15.37:0/3633399697 shutdown_connections 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887245f960 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720bf390 0x55887245fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 --2- 172.21.15.37:0/3633399697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720964d0 0x5588724a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 -- 172.21.15.37:0/3633399697 >> 172.21.15.37:0/3633399697 conn(0x5588720b34e0 msgr2=0x55887248cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 -- 172.21.15.37:0/3633399697 shutdown_connections 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.029+0000 7f32c2419740 1 -- 172.21.15.37:0/3633399697 wait complete. 2024-09-21T06:32:07.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.030+0000 7f32c2419740 1 -- start start 2024-09-21T06:32:07.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.030+0000 7f32c2419740 1 Processor -- start 2024-09-21T06:32:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x558872529fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 0x558872464080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32b413dc70 con 0x55887209e6b0 2024-09-21T06:32:07.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5588724ac7b0 con 0x5588720bf390 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55887247e980 con 0x5588720964d0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.031+0000 7f32c2419740 10 client.0 _ll_register_callbacks cb 0x7f32ac009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50488/0 (socket says 172.21.15.37:50488) 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 -- 172.21.15.37:0/914363854 learned_addr learned my addr 172.21.15.37:0/914363854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32b9200640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 0x558872464080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32b3e00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x558872529fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 -- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 msgr2=0x558872529fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x558872529fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 -- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 msgr2=0x558872464080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 0x558872464080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 -- 172.21.15.37:0/914363854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887247e0e0 con 0x55887209e6b0 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32b9200640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 0x558872464080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:07.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32abe00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f329c006d50 tx=0x7f329c006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abcb46e4ae2171d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.032+0000 7f32a1600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c037060 con 0x55887209e6b0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f329c004030 con 0x55887209e6b0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588723d7270 con 0x55887209e6b0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f329c03d050 con 0x55887209e6b0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 10 client.6504 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x558872467dd0 con 0x55887209e6b0 2024-09-21T06:32:07.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5588723d7270 con 0x55887209e6b0 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7f329c040020 con 0x55887209e6b0 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 10 client.6504 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 10 client.6504 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.d 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 20 client.6504 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5588724645c0 0x55887245f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 4 client.6504 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.033+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588723d46a0 con 0x5588724645c0 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.034+0000 7f32b3e00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5588724645c0 0x55887245f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.034+0000 7f32b3e00640 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5588724645c0 0x55887245f960 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1971ab9885aaff55 server_cookie=f5e6e71f3d07bbfa in_seq=0 out_seq=0 2024-09-21T06:32:07.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.034+0000 7f32a1600640 10 client.6504 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.044+0000 7f32a1600640 1 -- 172.21.15.37:0/914363854 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588723d46a0 con 0x5588724645c0 2024-09-21T06:32:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.044+0000 7f32a1600640 10 client.6504 handle_command_reply: tid=0 2024-09-21T06:32:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.044+0000 7f32c2419740 1 client.6504 shutdown 2024-09-21T06:32:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 msgr2=0x55887247dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f329c006d50 tx=0x7f329c006d80 comp rx=0 tx=0).stop 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 shutdown_connections 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x5588724645c0 0x55887245f960 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887209e6b0 0x55887247dba0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588720bf390 0x558872464080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 --2- 172.21.15.37:0/914363854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588720964d0 0x558872529fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 >> 172.21.15.37:0/914363854 conn(0x5588720b34e0 msgr2=0x55887248cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32a1600640 0 client.6504 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 shutdown_connections 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 1 -- 172.21.15.37:0/914363854 wait complete. 2024-09-21T06:32:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.045+0000 7f32c2419740 20 client.6504 trim_cache size 0 max 16384 2024-09-21T06:32:07.049 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:07.049 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:32:07.049 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:32:07.049 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:07.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 msgr2=0x7f329007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 0x7f329007b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32b4144de0 tx=0x7f329c009060 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 msgr2=0x7f32b40735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034040 tx=0x7f32a4003910 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 shutdown_connections 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3290078b90 0x7f329007b0a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32b41456e0 0x7f32b41e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b4142910 0x7f32b415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 --2- 172.21.15.37:0/4049900275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32b40cc8d0 0x7f32b40735e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 >> 172.21.15.37:0/4049900275 conn(0x7f32b406f920 msgr2=0x7f32b40b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 shutdown_connections 2024-09-21T06:32:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.046+0000 7f32b9c00640 1 -- 172.21.15.37:0/4049900275 wait complete. 2024-09-21T06:32:07.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2024-09-21T06:32:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 -- 172.21.15.37:0/1754128857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 msgr2=0x7f2bf806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 --2- 172.21.15.37:0/1754128857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf806ddf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80b2c70 tx=0x7f2be802a560 comp rx=0 tx=0).stop 2024-09-21T06:32:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 -- 172.21.15.37:0/1754128857 shutdown_connections 2024-09-21T06:32:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 --2- 172.21.15.37:0/1754128857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf806e330 0x7f2bf80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 --2- 172.21.15.37:0/1754128857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf806ddf0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 --2- 172.21.15.37:0/1754128857 >> v2:172.21.15.150:3300/0 conn(0x7f2bf8156a50 0x7f2bf8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 -- 172.21.15.37:0/1754128857 >> 172.21.15.37:0/1754128857 conn(0x7f2bf80b1b90 msgr2=0x7f2bf80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 -- 172.21.15.37:0/1754128857 shutdown_connections 2024-09-21T06:32:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.301+0000 7f2c00800640 1 -- 172.21.15.37:0/1754128857 wait complete. 2024-09-21T06:32:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.304+0000 7f2c00800640 1 Processor -- start 2024-09-21T06:32:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.304+0000 7f2c00800640 1 -- start start 2024-09-21T06:32:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf806e330 0x7f2bf81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf8156a50 0x7f2bf80aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf80ce330 con 0x7f2bf8074b60 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf80ce1b0 con 0x7f2bf806e330 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2c00800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf80ce4b0 con 0x7f2bf8156a50 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2bfd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2bfd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50510/0 (socket says 172.21.15.37:50510) 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.305+0000 7f2bfd400640 1 -- 172.21.15.37:0/1610859078 learned_addr learned my addr 172.21.15.37:0/1610859078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf806e330 0x7f2bf81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfe800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf8156a50 0x7f2bf80aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 -- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf8156a50 msgr2=0x7f2bf80aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf8156a50 0x7f2bf80aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 -- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf806e330 msgr2=0x7f2bf81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf806e330 0x7f2bf81571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 -- 172.21.15.37:0/1610859078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf8159900 con 0x7f2bf8074b60 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034040 tx=0x7f2be8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.306+0000 7f2bf2000640 1 -- 172.21.15.37:0/1610859078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be803e030 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.307+0000 7f2bf2000640 1 -- 172.21.15.37:0/1610859078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2be8003e00 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.307+0000 7f2bf2000640 1 -- 172.21.15.37:0/1610859078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8035040 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.307+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf8159be0 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.307+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf80b3a70 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.310+0000 7f2bf2000640 1 -- 172.21.15.37:0/1610859078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2be8041030 con 0x7f2bf8074b60 2024-09-21T06:32:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.310+0000 7f2bf2000640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 0x7f2bdc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.311+0000 7f2bf2000640 1 -- 172.21.15.37:0/1610859078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2be80bb5b0 con 0x7f2bf8074b60 2024-09-21T06:32:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.311+0000 7f2c077c8740 1 Processor -- start 2024-09-21T06:32:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.311+0000 7f2c077c8740 1 -- start start 2024-09-21T06:32:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.311+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 0x7f2bdc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 0x7f2bdc07b0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00bb50 tx=0x7f2bec00b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a2484c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a2484c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555a247821c0 con 0x555a2447b390 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555a24795870 con 0x555a2445a6b0 2024-09-21T06:32:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.312+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555a24784ca0 con 0x555a244524d0 2024-09-21T06:32:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50522/0 (socket says 172.21.15.37:50522) 2024-09-21T06:32:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 -- 172.21.15.37:0/1835682150 learned_addr learned my addr 172.21.15.37:0/1835682150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfe800640 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a2484c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a2484c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 -- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 msgr2=0x555a2484c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a2484c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 -- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 msgr2=0x555a2484c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a2484c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 -- 172.21.15.37:0/1835682150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a2455d7d0 con 0x555a2447b390 2024-09-21T06:32:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.313+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2bec010fe0 tx=0x7f2bec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e61cdf394aa27d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.314+0000 7f2be6600640 1 -- 172.21.15.37:0/1835682150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec01e620 con 0x555a2447b390 2024-09-21T06:32:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.314+0000 7f2be6600640 1 -- 172.21.15.37:0/1835682150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bec01e8b0 con 0x555a2447b390 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.314+0000 7f2be6600640 1 -- 172.21.15.37:0/1835682150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec01ec60 con 0x555a2447b390 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 -- 172.21.15.37:0/1835682150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 msgr2=0x555a24869550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2bec010fe0 tx=0x7f2bec013000 comp rx=0 tx=0).stop 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 -- 172.21.15.37:0/1835682150 shutdown_connections 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a2484c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a2484c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1835682150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24869550 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 -- 172.21.15.37:0/1835682150 >> 172.21.15.37:0/1835682150 conn(0x555a2446f4e0 msgr2=0x555a24822440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 -- 172.21.15.37:0/1835682150 shutdown_connections 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.315+0000 7f2c077c8740 1 -- 172.21.15.37:0/1835682150 wait complete. 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.316+0000 7f2c077c8740 1 -- start start 2024-09-21T06:32:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.316+0000 7f2c077c8740 1 Processor -- start 2024-09-21T06:32:07.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.317+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 0x555a2483a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.317+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.317+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 0x555a2484b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2c077c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555a2472b410 con 0x555a2445a6b0 2024-09-21T06:32:07.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555a24788ab0 con 0x555a244524d0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555a244f5740 con 0x555a2447b390 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2c077c8740 10 client.0 _ll_register_callbacks cb 0x555a24708360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50524/0 (socket says 172.21.15.37:50524) 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 -- 172.21.15.37:0/282620336 learned_addr learned my addr 172.21.15.37:0/282620336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 -- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 msgr2=0x555a2484b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfe800640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 0x555a2484b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfde00640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 0x555a2483a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 0x555a2484b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 -- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 msgr2=0x555a2483a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 0x555a2483a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfd400640 1 -- 172.21.15.37:0/282620336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a244e6970 con 0x555a2445a6b0 2024-09-21T06:32:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfe800640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 0x555a2484b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.318+0000 7f2bfde00640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 0x555a2483a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bfd400640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2be8000c00 tx=0x7f2be8083100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd3824e0ed55caf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8048040 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2be808c850 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bec01e8b0 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2be8083200 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2c077c8740 10 client.6516 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x555a244c46a0 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bec01e8b0 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7f2be80bb0b0 con 0x555a2445a6b0 2024-09-21T06:32:07.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2c077c8740 10 client.6516 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.319+0000 7f2c077c8740 10 client.6516 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.320+0000 7f2c077c8740 20 client.6516 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.320+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24439f60 0x555a24864b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.320+0000 7f2c077c8740 4 client.6516 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.320+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a247977c0 con 0x555a24439f60 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.320+0000 7f2bfde00640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24439f60 0x555a24864b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.321+0000 7f2bfde00640 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24439f60 0x555a24864b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a08e0f319ad96ab0 server_cookie=f495a16c44defbe2 in_seq=0 out_seq=0 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.321+0000 7f2bdaa00640 10 client.6516 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.321+0000 7f2bdaa00640 1 -- 172.21.15.37:0/282620336 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a247977c0 con 0x555a24439f60 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.321+0000 7f2bdaa00640 10 client.6516 handle_command_reply: tid=0 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 client.6516 shutdown 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 msgr2=0x555a24863480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2be8000c00 tx=0x7f2be8083100 comp rx=0 tx=0).stop 2024-09-21T06:32:07.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 shutdown_connections 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24439f60 0x555a24864b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2445a6b0 0x555a24863480 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2447b390 0x555a2484b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 --2- 172.21.15.37:0/282620336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a244524d0 0x555a2483a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2bdaa00640 0 client.6516 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 >> 172.21.15.37:0/282620336 conn(0x555a2446f4e0 msgr2=0x555a2481b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 shutdown_connections 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 1 -- 172.21.15.37:0/282620336 wait complete. 2024-09-21T06:32:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.322+0000 7f2c077c8740 20 client.6516 trim_cache size 0 max 16384 2024-09-21T06:32:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.338+0000 7f2c077c8740 1 Processor -- start 2024-09-21T06:32:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.338+0000 7f2c077c8740 1 -- start start 2024-09-21T06:32:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a24889d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8202740 con 0x555a2447b390 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8201930 con 0x555a2445a6b0 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.339+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf8202010 con 0x555a244524d0 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfde00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a24889d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50532/0 (socket says 172.21.15.37:50532) 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 -- 172.21.15.37:0/567197547 learned_addr learned my addr 172.21.15.37:0/567197547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfd400640 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 -- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 msgr2=0x555a24889d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a24889d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 -- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 msgr2=0x555a24823120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 -- 172.21.15.37:0/567197547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a2481ed50 con 0x555a2447b390 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.340+0000 7f2bfe800640 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009f70 tx=0x7f2bf4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16d61bd42a6030f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2be5c00640 1 -- 172.21.15.37:0/567197547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4004b20 con 0x555a2447b390 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2be5c00640 1 -- 172.21.15.37:0/567197547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bf4004cc0 con 0x555a2447b390 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2be5c00640 1 -- 172.21.15.37:0/567197547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4003bd0 con 0x555a2447b390 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 -- 172.21.15.37:0/567197547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 msgr2=0x555a24823660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009f70 tx=0x7f2bf4013830 comp rx=0 tx=0).stop 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 -- 172.21.15.37:0/567197547 shutdown_connections 2024-09-21T06:32:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 --2- 172.21.15.37:0/567197547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a244524d0 0x555a24889d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.341+0000 7f2c077c8740 1 -- 172.21.15.37:0/567197547 >> 172.21.15.37:0/567197547 conn(0x555a2446f4e0 msgr2=0x555a2484b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.342+0000 7f2c077c8740 1 -- 172.21.15.37:0/567197547 shutdown_connections 2024-09-21T06:32:07.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.342+0000 7f2c077c8740 1 -- 172.21.15.37:0/567197547 wait complete. 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.342+0000 7f2c077c8740 1 -- start start 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.343+0000 7f2c077c8740 1 Processor -- start 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf80b4d70 con 0x555a244524d0 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555a24868d90 con 0x555a2447b390 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555a248621c0 con 0x555a2445a6b0 2024-09-21T06:32:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2c077c8740 10 client.0 _ll_register_callbacks cb 0x7f2be807f850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2bfde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2bfde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50536/0 (socket says 172.21.15.37:50536) 2024-09-21T06:32:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.344+0000 7f2bfde00640 1 -- 172.21.15.37:0/1536600944 learned_addr learned my addr 172.21.15.37:0/1536600944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfe800640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 -- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 msgr2=0x555a24823120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 -- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2447b390 msgr2=0x555a24823660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 -- 172.21.15.37:0/1536600944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a24888920 con 0x555a244524d0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bfde00640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00bfd0 tx=0x7f2bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9bf7521ea67e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec01eab0 con 0x555a244524d0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2bec01ec50 con 0x555a244524d0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.345+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf4004cc0 con 0x555a244524d0 2024-09-21T06:32:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2bec01ee50 con 0x555a244524d0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 10 client.6522 fetch_fsmap learned FSMap version 59 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x555a248654c0 con 0x555a244524d0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bf4004cc0 con 0x555a244524d0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 59) ==== 3554+0+0 (secure 0 0 0) 0x7f2bec042030 con 0x555a244524d0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 10 client.6522 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 10 client.6522 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 20 client.6522 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24849f10 0x555a2484a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 4 client.6522 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a2481d4d0 con 0x555a24849f10 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.346+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24849f10 0x555a2484a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.347+0000 7f2bfd400640 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24849f10 0x555a2484a320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17c9acdd93c1aa94 server_cookie=bc9ba2e7a95c9db8 in_seq=0 out_seq=0 2024-09-21T06:32:07.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.347+0000 7f2bdaa00640 10 client.6522 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:07.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.360+0000 7f2bdaa00640 1 -- 172.21.15.37:0/1536600944 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a2481d4d0 con 0x555a24849f10 2024-09-21T06:32:07.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.360+0000 7f2bdaa00640 10 client.6522 handle_command_reply: tid=0 2024-09-21T06:32:07.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.360+0000 7f2c077c8740 1 client.6522 shutdown 2024-09-21T06:32:07.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.360+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 msgr2=0x555a24866bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.360+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00bfd0 tx=0x7f2bec0046c0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 shutdown_connections 2024-09-21T06:32:07.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x555a24849f10 0x555a2484a320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a2447b390 0x555a24823660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a2445a6b0 0x555a24823120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2bdaa00640 0 client.6522 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 --2- 172.21.15.37:0/1536600944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a244524d0 0x555a24866bf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 >> 172.21.15.37:0/1536600944 conn(0x555a2446f4e0 msgr2=0x555a2481b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 shutdown_connections 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 1 -- 172.21.15.37:0/1536600944 wait complete. 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c077c8740 20 client.6522 trim_cache size 0 max 16384 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:07.364 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:32:07.365 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:32:07.365 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:07.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 msgr2=0x7f2bdc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 0x7f2bdc07b0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00bb50 tx=0x7f2bec00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 msgr2=0x7f2bf80afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.361+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034040 tx=0x7f2be8003910 comp rx=0 tx=0).stop 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 shutdown_connections 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2bdc078bc0 0x7f2bdc07b0d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf8074b60 0x7f2bf80afa00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bf8156a50 0x7f2bf80aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 --2- 172.21.15.37:0/1610859078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf806e330 0x7f2bf81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 >> 172.21.15.37:0/1610859078 conn(0x7f2bf80b1b90 msgr2=0x7f2bf806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 shutdown_connections 2024-09-21T06:32:07.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07.362+0000 7f2c00800640 1 -- 172.21.15.37:0/1610859078 wait complete. 2024-09-21T06:32:07.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:07.388 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/ 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-21T06:32:07.436 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-21T06:32:07.437 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:07.437 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:07.437 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:07.438 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:07.438 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-21T06:32:07.438 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:07.581 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-09-21T06:32:07.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:07.592 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:07.592 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:07.592 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:07.592 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:07.592 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-21T06:32:07.593 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:07.735 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-09-21T06:32:07.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-21T06:32:07.747 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:07.887 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-09-21T06:32:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:07.898 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:07.898 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:07.898 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:07.899 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:07.899 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-21T06:32:07.899 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:08.039 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627781 2024-09-21T06:32:08.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-21T06:32:08.051 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:08.191 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627782 2024-09-21T06:32:08.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:08.202 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:32:08.202 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:32:08.202 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:32:08.203 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:32:08.203 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-21T06:32:08.203 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:32:08.343 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627783 2024-09-21T06:32:08.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:32:08.355 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-21T06:32:08.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.540+0000 7ff3eb145f40 1 Processor -- start 2024-09-21T06:32:08.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.540+0000 7ff3eb145f40 1 -- start start 2024-09-21T06:32:08.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.541+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.541+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556ac12c1e0 0x5556ac12c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 0x5556ac1d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556ac0aa1d0 con 0x7ff3c80022f0 2024-09-21T06:32:08.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5556ac0b6f90 con 0x5556ac12c1e0 2024-09-21T06:32:08.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5556ac04bc60 con 0x5556ac1d2470 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50552/0 (socket says 172.21.15.37:50552) 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 -- 172.21.15.37:0/1740900340 learned_addr learned my addr 172.21.15.37:0/1740900340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e3800640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556ac12c1e0 0x5556ac12c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4c00640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 0x5556ac1d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 -- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 msgr2=0x5556ac1d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 0x5556ac1d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 -- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556ac12c1e0 msgr2=0x5556ac12c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556ac12c1e0 0x5556ac12c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4200640 1 -- 172.21.15.37:0/1740900340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ac1b7560 con 0x7ff3c80022f0 2024-09-21T06:32:08.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.542+0000 7ff3e4c00640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 0x5556ac1d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:08.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.543+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40467c0 tx=0x5556ac147aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4546ad5ab124b951 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.543+0000 7ff3e2400640 1 -- 172.21.15.37:0/1740900340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40016f0 con 0x7ff3c80022f0 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.543+0000 7ff3e2400640 1 -- 172.21.15.37:0/1740900340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3d4064070 con 0x7ff3c80022f0 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.543+0000 7ff3e2400640 1 -- 172.21.15.37:0/1740900340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d404dc50 con 0x7ff3c80022f0 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1740900340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 msgr2=0x5556ac1cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40467c0 tx=0x5556ac147aa0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1740900340 shutdown_connections 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac1d2470 0x5556ac1d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556ac12c1e0 0x5556ac12c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1740900340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80022f0 0x5556ac1cf0e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1740900340 >> 172.21.15.37:0/1740900340 conn(0x5556ac01cac0 msgr2=0x5556ac129020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1740900340 shutdown_connections 2024-09-21T06:32:08.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.544+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1740900340 wait complete. 2024-09-21T06:32:08.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.545+0000 7ff3eb145f40 1 Processor -- start 2024-09-21T06:32:08.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.545+0000 7ff3eb145f40 1 -- start start 2024-09-21T06:32:08.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac12c1e0 0x5556ac1cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 0x5556ac1b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c80022f0 0x5556ac12c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556ac1ceaa0 con 0x5556ac1d2470 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5556ac1b7310 con 0x7ff3c80022f0 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.546+0000 7ff3eb145f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5556ac1c1b90 con 0x5556ac12c1e0 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 0x5556ac1b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 0x5556ac1b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50554/0 (socket says 172.21.15.37:50554) 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 -- 172.21.15.37:0/1581983303 learned_addr learned my addr 172.21.15.37:0/1581983303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e4c00640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c80022f0 0x5556ac12c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac12c1e0 0x5556ac1cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 -- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac12c1e0 msgr2=0x5556ac1cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac12c1e0 0x5556ac1cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 -- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c80022f0 msgr2=0x5556ac12c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c80022f0 0x5556ac12c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 -- 172.21.15.37:0/1581983303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ac1d3a50 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556ac12c1e0 0x5556ac1cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.547+0000 7ff3e3800640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 0x5556ac1b4180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0002db0 tx=0x7ff3d000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.548+0000 7ff3e1000640 1 -- 172.21.15.37:0/1581983303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0002440 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.548+0000 7ff3e1000640 1 -- 172.21.15.37:0/1581983303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3d0001740 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.548+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5556ac1d3d30 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.548+0000 7ff3e1000640 1 -- 172.21.15.37:0/1581983303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0001b70 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.548+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ac1b46c0 con 0x5556ac1d2470 2024-09-21T06:32:08.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.550+0000 7ff3e1000640 1 -- 172.21.15.37:0/1581983303 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3d0001d10 con 0x5556ac1d2470 2024-09-21T06:32:08.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3e1000640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3c4077570 0x7ff3c4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3e1000640 1 -- 172.21.15.37:0/1581983303 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff3d009a560 con 0x5556ac1d2470 2024-09-21T06:32:08.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x5556ac1b6eb0 0x5556ac1c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 9.12 9:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x5556ac276a80 con 0x5556ac1b6eb0 2024-09-21T06:32:08.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3c4077570 0x7ff3c4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.551+0000 7ff3e4c00640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x5556ac1b6eb0 0x5556ac1c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.552+0000 7ff3e4c00640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x5556ac1b6eb0 0x5556ac1c1180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.552+0000 7ff3e4200640 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3c4077570 0x7ff3c4079a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4063000 tx=0x7ff3d40092a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3e4c00640 1 -- 172.21.15.37:0/1581983303 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v67'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3dc00b040 con 0x5556ac1b6eb0 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x5556ac1b6eb0 msgr2=0x5556ac1c1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x5556ac1b6eb0 0x5556ac1c1180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3c4077570 msgr2=0x7ff3c4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3c4077570 0x7ff3c4079a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4063000 tx=0x7ff3d40092a0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 msgr2=0x5556ac1b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 --2- 172.21.15.37:0/1581983303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556ac1d2470 0x5556ac1b4180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0002db0 tx=0x7ff3d000d720 comp rx=0 tx=0).stop 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3e4c00640 1 -- 172.21.15.37:0/1581983303 reap_dead start 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 shutdown_connections 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 >> 172.21.15.37:0/1581983303 conn(0x5556ac01cac0 msgr2=0x5556ac129020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 shutdown_connections 2024-09-21T06:32:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.554+0000 7ff3eb145f40 1 -- 172.21.15.37:0/1581983303 wait complete. 2024-09-21T06:32:08.570 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-21T06:32:08.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.713+0000 7f4a34235f40 1 Processor -- start 2024-09-21T06:32:08.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.714+0000 7f4a34235f40 1 -- start start 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f6988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f698df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a87f511c60 con 0x7f4a140030f0 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a87f5701d0 con 0x55a87f5f11c0 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a34235f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a87f60d110 con 0x55a87f5f0db0 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a2d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a2d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50572/0 (socket says 172.21.15.37:50572) 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a2d400640 1 -- 172.21.15.37:0/1448302036 learned_addr learned my addr 172.21.15.37:0/1448302036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a2de00640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f698df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.715+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f6988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 -- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 msgr2=0x55a87f6988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f6988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 -- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 msgr2=0x55a87f698df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f698df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 -- 172.21.15.37:0/1448302036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87f60a9e0 con 0x7f4a140030f0 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f6988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a2d400640 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a200017d0 tx=0x7f4a20009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05846ff05dbb460 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a27400640 1 -- 172.21.15.37:0/1448302036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000d7e0 con 0x7f4a140030f0 2024-09-21T06:32:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a27400640 1 -- 172.21.15.37:0/1448302036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a20014020 con 0x7f4a140030f0 2024-09-21T06:32:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.716+0000 7f4a27400640 1 -- 172.21.15.37:0/1448302036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000dc80 con 0x7f4a140030f0 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 -- 172.21.15.37:0/1448302036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 msgr2=0x55a87f67e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a200017d0 tx=0x7f4a20009ac0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 -- 172.21.15.37:0/1448302036 shutdown_connections 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f698df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f6988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 --2- 172.21.15.37:0/1448302036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f67e310 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 -- 172.21.15.37:0/1448302036 >> 172.21.15.37:0/1448302036 conn(0x55a87f4e2ac0 msgr2=0x55a87f696780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 -- 172.21.15.37:0/1448302036 shutdown_connections 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.717+0000 7f4a34235f40 1 -- 172.21.15.37:0/1448302036 wait complete. 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.718+0000 7f4a34235f40 1 Processor -- start 2024-09-21T06:32:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.718+0000 7f4a34235f40 1 -- start start 2024-09-21T06:32:08.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.719+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f725850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.719+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f67d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a34235f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f71f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a34235f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a87f5ee040 con 0x7f4a140030f0 2024-09-21T06:32:08.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a34235f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a87f60a740 con 0x55a87f5f11c0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a34235f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a87f5eda30 con 0x55a87f5f0db0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f71f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f71f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50580/0 (socket says 172.21.15.37:50580) 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 -- 172.21.15.37:0/4142000731 learned_addr learned my addr 172.21.15.37:0/4142000731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2d400640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f725850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f67d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 -- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 msgr2=0x55a87f725850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a87f5f0db0 0x55a87f725850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 -- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 msgr2=0x55a87f67d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f67d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2de00640 1 -- 172.21.15.37:0/4142000731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87f5c1cc0 con 0x7f4a140030f0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.720+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a87f5f11c0 0x55a87f67d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a2de00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f71f560 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55a87f5c38e0 tx=0x7f4a28010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a26000640 1 -- 172.21.15.37:0/4142000731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800f430 con 0x7f4a140030f0 2024-09-21T06:32:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a26000640 1 -- 172.21.15.37:0/4142000731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a2800f5d0 con 0x7f4a140030f0 2024-09-21T06:32:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a87f5c1fa0 con 0x7f4a140030f0 2024-09-21T06:32:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a26000640 1 -- 172.21.15.37:0/4142000731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800fa00 con 0x7f4a140030f0 2024-09-21T06:32:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.721+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87f609230 con 0x7f4a140030f0 2024-09-21T06:32:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.723+0000 7f4a26000640 1 -- 172.21.15.37:0/4142000731 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a2800fba0 con 0x7f4a140030f0 2024-09-21T06:32:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.724+0000 7f4a26000640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a100775a0 0x7f4a10079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.724+0000 7f4a26000640 1 -- 172.21.15.37:0/4142000731 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4a28016020 con 0x7f4a140030f0 2024-09-21T06:32:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.724+0000 7f4a34235f40 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55a87f7342e0 0x55a87f7346f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.724+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 9.12 9:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x55a87f609f40 con 0x55a87f7342e0 2024-09-21T06:32:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.725+0000 7f4a2d400640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a100775a0 0x7f4a10079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.725+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55a87f7342e0 0x55a87f7346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.725+0000 7f4a2ca00640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55a87f7342e0 0x55a87f7346f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.725+0000 7f4a2d400640 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a100775a0 0x7f4a10079ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a200017d0 tx=0x7f4a200092a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a2ca00640 1 -- 172.21.15.37:0/4142000731 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v67'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4a1c0557b0 con 0x55a87f7342e0 2024-09-21T06:32:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55a87f7342e0 msgr2=0x55a87f7346f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x55a87f7342e0 0x55a87f7346f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a100775a0 msgr2=0x7f4a10079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a100775a0 0x7f4a10079ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a200017d0 tx=0x7f4a200092a0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 msgr2=0x55a87f71f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 --2- 172.21.15.37:0/4142000731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a140030f0 0x55a87f71f560 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55a87f5c38e0 tx=0x7f4a28010f90 comp rx=0 tx=0).stop 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a2de00640 1 -- 172.21.15.37:0/4142000731 reap_dead start 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 shutdown_connections 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 >> 172.21.15.37:0/4142000731 conn(0x55a87f4e2ac0 msgr2=0x55a87f67e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 shutdown_connections 2024-09-21T06:32:08.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.728+0000 7f4a34235f40 1 -- 172.21.15.37:0/4142000731 wait complete. 2024-09-21T06:32:08.744 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-21T06:32:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.886+0000 7fb578d03f40 1 Processor -- start 2024-09-21T06:32:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.886+0000 7fb578d03f40 1 -- start start 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97c205a0 0x55da97cafa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c23630 0x55da97caaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da97c22e30 con 0x55da97bdb470 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55da97b27c60 con 0x55da97c23630 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.887+0000 7fb578d03f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55da97c23110 con 0x55da97c205a0 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50590/0 (socket says 172.21.15.37:50590) 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 -- 172.21.15.37:0/2656537648 learned_addr learned my addr 172.21.15.37:0/2656537648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571400640 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97c205a0 0x55da97cafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb572800640 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c23630 0x55da97caaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 -- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97c205a0 msgr2=0x55da97cafa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97c205a0 0x55da97cafa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 -- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c23630 msgr2=0x55da97caaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c23630 0x55da97caaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 -- 172.21.15.37:0/2656537648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da97afff10 con 0x55da97bdb470 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.888+0000 7fb571e00640 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c550 tx=0x7fb56400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b47f1f793a868e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb56be00640 1 -- 172.21.15.37:0/2656537648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56400a960 con 0x55da97bdb470 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb56be00640 1 -- 172.21.15.37:0/2656537648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb56400ab00 con 0x55da97bdb470 2024-09-21T06:32:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb56be00640 1 -- 172.21.15.37:0/2656537648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564015440 con 0x55da97bdb470 2024-09-21T06:32:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 -- 172.21.15.37:0/2656537648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 msgr2=0x55da97d37fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c550 tx=0x7fb56400c950 comp rx=0 tx=0).stop 2024-09-21T06:32:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 -- 172.21.15.37:0/2656537648 shutdown_connections 2024-09-21T06:32:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c23630 0x55da97caaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97c205a0 0x55da97cafa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 --2- 172.21.15.37:0/2656537648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97bdb470 0x55da97d37fc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.889+0000 7fb578d03f40 1 -- 172.21.15.37:0/2656537648 >> 172.21.15.37:0/2656537648 conn(0x55da97af8ac0 msgr2=0x55da97c03810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.890+0000 7fb578d03f40 1 -- 172.21.15.37:0/2656537648 shutdown_connections 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.890+0000 7fb578d03f40 1 -- 172.21.15.37:0/2656537648 wait complete. 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.891+0000 7fb578d03f40 1 Processor -- start 2024-09-21T06:32:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.891+0000 7fb578d03f40 1 -- start start 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97bdb470 0x55da97c93fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c205a0 0x55da97caa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 0x55da97caae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da97d37570 con 0x55da97c23630 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55da97d37b30 con 0x55da97c205a0 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.892+0000 7fb578d03f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55da97c1f880 con 0x55da97bdb470 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 0x55da97caae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 0x55da97caae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50606/0 (socket says 172.21.15.37:50606) 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 -- 172.21.15.37:0/3897410244 learned_addr learned my addr 172.21.15.37:0/3897410244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb571400640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c205a0 0x55da97caa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb571e00640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97bdb470 0x55da97c93fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 -- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97bdb470 msgr2=0x55da97c93fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55da97bdb470 0x55da97c93fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 -- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c205a0 msgr2=0x55da97caa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55da97c205a0 0x55da97caa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 -- 172.21.15.37:0/3897410244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da97c947e0 con 0x55da97c23630 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.893+0000 7fb572800640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 0x55da97caae50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb56c057040 tx=0x7fb56c004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.894+0000 7fb56aa00640 1 -- 172.21.15.37:0/3897410244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c064030 con 0x55da97c23630 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.894+0000 7fb56aa00640 1 -- 172.21.15.37:0/3897410244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb56c060440 con 0x55da97c23630 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.894+0000 7fb56aa00640 1 -- 172.21.15.37:0/3897410244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c060630 con 0x55da97c23630 2024-09-21T06:32:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.894+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da97d47020 con 0x55da97c23630 2024-09-21T06:32:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.894+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da97caff90 con 0x55da97c23630 2024-09-21T06:32:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.896+0000 7fb56aa00640 1 -- 172.21.15.37:0/3897410244 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb56c067030 con 0x55da97c23630 2024-09-21T06:32:08.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.897+0000 7fb56aa00640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5540775a0 0x7fb554079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.897+0000 7fb56aa00640 1 -- 172.21.15.37:0/3897410244 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb56c0ddcc0 con 0x55da97c23630 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.897+0000 7fb578d03f40 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55da97caba90 0x55da97c1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.897+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- osd_op(unknown.0.0:1 9.5 9:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x55da97d52990 con 0x55da97caba90 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.897+0000 7fb571e00640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5540775a0 0x7fb554079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.898+0000 7fb571400640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55da97caba90 0x55da97c1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.898+0000 7fb571e00640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5540775a0 0x7fb554079ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564017040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.898+0000 7fb571400640 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55da97caba90 0x55da97c1f1c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb571400640 1 -- 172.21.15.37:0/3897410244 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v67'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb56000c650 con 0x55da97caba90 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55da97caba90 msgr2=0x55da97c1f1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x55da97caba90 0x55da97c1f1c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5540775a0 msgr2=0x7fb554079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5540775a0 0x7fb554079ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564017040 comp rx=0 tx=0).stop 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 msgr2=0x55da97caae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.900+0000 7fb578d03f40 1 --2- 172.21.15.37:0/3897410244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da97c23630 0x55da97caae50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb56c057040 tx=0x7fb56c004550 comp rx=0 tx=0).stop 2024-09-21T06:32:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.901+0000 7fb572800640 1 -- 172.21.15.37:0/3897410244 reap_dead start 2024-09-21T06:32:08.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.901+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 shutdown_connections 2024-09-21T06:32:08.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.901+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 >> 172.21.15.37:0/3897410244 conn(0x55da97af8ac0 msgr2=0x55da97d36d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:08.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.901+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 shutdown_connections 2024-09-21T06:32:08.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:08.901+0000 7fb578d03f40 1 -- 172.21.15.37:0/3897410244 wait complete. 2024-09-21T06:32:08.917 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-21T06:32:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.058+0000 7f0a82018f40 1 Processor -- start 2024-09-21T06:32:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.058+0000 7f0a82018f40 1 -- start start 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601b1470 0x55626026a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f65a0 0x556260284620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5562600fdc60 con 0x5562601f9630 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55626015c1d0 con 0x5562601b1470 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a82018f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5562601f9110 con 0x5562601f65a0 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50608/0 (socket says 172.21.15.37:50608) 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 -- 172.21.15.37:0/1812395783 learned_addr learned my addr 172.21.15.37:0/1812395783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7b200640 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601b1470 0x55626026a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7a800640 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f65a0 0x556260284620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 -- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f65a0 msgr2=0x556260284620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f65a0 0x556260284620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 -- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601b1470 msgr2=0x55626026a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601b1470 0x55626026a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.060+0000 7f0a7bc00640 1 -- 172.21.15.37:0/1812395783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556260285780 con 0x5562601f9630 2024-09-21T06:32:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.061+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0a740015c0 tx=0x7f0a74001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32f712732a102f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.061+0000 7f0a79400640 1 -- 172.21.15.37:0/1812395783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74016070 con 0x5562601f9630 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.061+0000 7f0a79400640 1 -- 172.21.15.37:0/1812395783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a7400f800 con 0x5562601f9630 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.061+0000 7f0a79400640 1 -- 172.21.15.37:0/1812395783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400fc30 con 0x5562601f9630 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 -- 172.21.15.37:0/1812395783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 msgr2=0x5562603180e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0a740015c0 tx=0x7f0a74001a80 comp rx=0 tx=0).stop 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 -- 172.21.15.37:0/1812395783 shutdown_connections 2024-09-21T06:32:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601f9630 0x5562603180e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f65a0 0x556260284620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 --2- 172.21.15.37:0/1812395783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601b1470 0x55626026a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 -- 172.21.15.37:0/1812395783 >> 172.21.15.37:0/1812395783 conn(0x5562600ceac0 msgr2=0x556260283140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 -- 172.21.15.37:0/1812395783 shutdown_connections 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.062+0000 7f0a82018f40 1 -- 172.21.15.37:0/1812395783 wait complete. 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.063+0000 7f0a82018f40 1 Processor -- start 2024-09-21T06:32:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.063+0000 7f0a82018f40 1 -- start start 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.064+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 0x556260280380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601f65a0 0x556260264fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a82018f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f9630 0x55626031c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a82018f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556260285240 con 0x5562601b1470 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a82018f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556260281880 con 0x5562601f65a0 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a82018f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556260285ff0 con 0x5562601f9630 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 0x556260280380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 0x556260280380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50610/0 (socket says 172.21.15.37:50610) 2024-09-21T06:32:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 -- 172.21.15.37:0/53692657 learned_addr learned my addr 172.21.15.37:0/53692657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f9630 0x55626031c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 -- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f9630 msgr2=0x55626031c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7a800640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601f65a0 0x556260264fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f9630 0x55626031c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 -- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601f65a0 msgr2=0x556260264fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601f65a0 0x556260264fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7b200640 1 -- 172.21.15.37:0/53692657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562603102d0 con 0x5562601b1470 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562601f9630 0x55626031c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.065+0000 7f0a7a800640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562601f65a0 0x556260264fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a7b200640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 0x556260280380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00a6e0 tx=0x7f0a6c00aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a73e00640 1 -- 172.21.15.37:0/53692657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c009b40 con 0x5562601b1470 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a73e00640 1 -- 172.21.15.37:0/53692657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a6c009ce0 con 0x5562601b1470 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562601f55d0 con 0x5562601b1470 2024-09-21T06:32:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a73e00640 1 -- 172.21.15.37:0/53692657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c008680 con 0x5562601b1470 2024-09-21T06:32:09.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.066+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626031d320 con 0x5562601b1470 2024-09-21T06:32:09.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.068+0000 7f0a73e00640 1 -- 172.21.15.37:0/53692657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a6c008820 con 0x5562601b1470 2024-09-21T06:32:09.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.069+0000 7f0a73e00640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a5c0775a0 0x7f0a5c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.069+0000 7f0a73e00640 1 -- 172.21.15.37:0/53692657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f0a6c09ad10 con 0x5562601b1470 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.069+0000 7f0a82018f40 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5562601df980 0x556260265650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.069+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- osd_op(unknown.0.0:1 9.9 9:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x556260265b90 con 0x5562601df980 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.069+0000 7f0a7a800640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a5c0775a0 0x7f0a5c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.070+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5562601df980 0x556260265650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.070+0000 7f0a7a800640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a5c0775a0 0x7f0a5c079ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a68055000 tx=0x7f0a68058040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.070+0000 7f0a7bc00640 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5562601df980 0x556260265650 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.072+0000 7f0a7bc00640 1 -- 172.21.15.37:0/53692657 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v67'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0a7400f070 con 0x5562601df980 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.072+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5562601df980 msgr2=0x556260265650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.072+0000 7f0a82018f40 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5562601df980 0x556260265650 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a5c0775a0 msgr2=0x7f0a5c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0a5c0775a0 0x7f0a5c079ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a68055000 tx=0x7f0a68058040 comp rx=0 tx=0).stop 2024-09-21T06:32:09.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 msgr2=0x556260280380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 --2- 172.21.15.37:0/53692657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562601b1470 0x556260280380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00a6e0 tx=0x7f0a6c00aba0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a7bc00640 1 -- 172.21.15.37:0/53692657 reap_dead start 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 shutdown_connections 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 >> 172.21.15.37:0/53692657 conn(0x5562600ceac0 msgr2=0x556260281cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 shutdown_connections 2024-09-21T06:32:09.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.073+0000 7f0a82018f40 1 -- 172.21.15.37:0/53692657 wait complete. 2024-09-21T06:32:09.086 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-21T06:32:09.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.229+0000 7f991c1eaf40 1 Processor -- start 2024-09-21T06:32:09.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.229+0000 7f991c1eaf40 1 -- start start 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7481b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b74820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5654b72ca1d0 con 0x5654b731fb80 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5654b72d6f90 con 0x5654b73ed280 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5654b726bc60 con 0x5654b73f4f40 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50626/0 (socket says 172.21.15.37:50626) 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 -- 172.21.15.37:0/3488184639 learned_addr learned my addr 172.21.15.37:0/3488184639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915e00640 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b74820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9914a00640 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7481b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 -- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 msgr2=0x5654b74820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b74820b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 -- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 msgr2=0x5654b7481b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7481b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.231+0000 7f9915400640 1 -- 172.21.15.37:0/3488184639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b73f5620 con 0x5654b731fb80 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f9915400640 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9908001d20 tx=0x7f990800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4c0d5536d06609 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f990f400640 1 -- 172.21.15.37:0/3488184639 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800bd80 con 0x5654b731fb80 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f990f400640 1 -- 172.21.15.37:0/3488184639 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f990800f400 con 0x5654b731fb80 2024-09-21T06:32:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f990f400640 1 -- 172.21.15.37:0/3488184639 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800f850 con 0x5654b731fb80 2024-09-21T06:32:09.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f991c1eaf40 1 -- 172.21.15.37:0/3488184639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 msgr2=0x5654b73f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9908001d20 tx=0x7f990800b5b0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f991c1eaf40 1 -- 172.21.15.37:0/3488184639 shutdown_connections 2024-09-21T06:32:09.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.232+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b74820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.233+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7481b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.233+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/3488184639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73f4a00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.233+0000 7f991c1eaf40 1 -- 172.21.15.37:0/3488184639 >> 172.21.15.37:0/3488184639 conn(0x5654b723cac0 msgr2=0x5654b73f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.233+0000 7f991c1eaf40 1 -- 172.21.15.37:0/3488184639 shutdown_connections 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.233+0000 7f991c1eaf40 1 -- 172.21.15.37:0/3488184639 wait complete. 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.234+0000 7f991c1eaf40 1 Processor -- start 2024-09-21T06:32:09.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.234+0000 7f991c1eaf40 1 -- start start 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7364e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b73e4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5654b7348980 con 0x5654b731fb80 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5654b73f5900 con 0x5654b73ed280 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f991c1eaf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5654b73e8610 con 0x5654b73f4f40 2024-09-21T06:32:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f9915400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.235+0000 7f9915400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50640/0 (socket says 172.21.15.37:50640) 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 -- 172.21.15.37:0/2532568248 learned_addr learned my addr 172.21.15.37:0/2532568248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915e00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b73e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9914a00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7364e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 -- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 msgr2=0x5654b73e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5654b73f4f40 0x5654b73e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 -- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 msgr2=0x5654b7364e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7364e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 -- 172.21.15.37:0/2532568248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b7347f70 con 0x5654b731fb80 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9914a00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654b73ed280 0x5654b7364e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f9915400640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73ec430 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f990800bab0 tx=0x7f990800ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f990e000640 1 -- 172.21.15.37:0/2532568248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990801b430 con 0x5654b731fb80 2024-09-21T06:32:09.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f990e000640 1 -- 172.21.15.37:0/2532568248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9908020020 con 0x5654b731fb80 2024-09-21T06:32:09.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.236+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654b73f2a50 con 0x5654b731fb80 2024-09-21T06:32:09.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.237+0000 7f990e000640 1 -- 172.21.15.37:0/2532568248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990801b740 con 0x5654b731fb80 2024-09-21T06:32:09.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.237+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b74853e0 con 0x5654b731fb80 2024-09-21T06:32:09.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.239+0000 7f990e000640 1 -- 172.21.15.37:0/2532568248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9908017070 con 0x5654b731fb80 2024-09-21T06:32:09.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f990e000640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f98f8077570 0x7f98f8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f990e000640 1 -- 172.21.15.37:0/2532568248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f990809b170 con 0x5654b731fb80 2024-09-21T06:32:09.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5654b7361a60 0x5654b7361e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- osd_op(unknown.0.0:1 9.5 9:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x5654b73623b0 con 0x5654b7361a60 2024-09-21T06:32:09.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f9914a00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f98f8077570 0x7f98f8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.240+0000 7f9915e00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5654b7361a60 0x5654b7361e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.241+0000 7f9914a00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f98f8077570 0x7f98f8079a80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5654b734b460 tx=0x7f9904002650 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.241+0000 7f9915e00640 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5654b7361a60 0x5654b7361e70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f9915e00640 1 -- 172.21.15.37:0/2532568248 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v67'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f991005e040 con 0x5654b7361a60 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5654b7361a60 msgr2=0x5654b7361e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x5654b7361a60 0x5654b7361e70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f98f8077570 msgr2=0x7f98f8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f98f8077570 0x7f98f8079a80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5654b734b460 tx=0x7f9904002650 comp rx=0 tx=0).stop 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 msgr2=0x5654b73ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f991c1eaf40 1 --2- 172.21.15.37:0/2532568248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5654b731fb80 0x5654b73ec430 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f990800bab0 tx=0x7f990800ff50 comp rx=0 tx=0).stop 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.243+0000 7f9915e00640 1 -- 172.21.15.37:0/2532568248 reap_dead start 2024-09-21T06:32:09.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.244+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 shutdown_connections 2024-09-21T06:32:09.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.244+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 >> 172.21.15.37:0/2532568248 conn(0x5654b723cac0 msgr2=0x5654b73f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.244+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 shutdown_connections 2024-09-21T06:32:09.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.244+0000 7f991c1eaf40 1 -- 172.21.15.37:0/2532568248 wait complete. 2024-09-21T06:32:09.257 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-21T06:32:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.399+0000 7fb5950f8f40 1 Processor -- start 2024-09-21T06:32:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.400+0000 7fb5950f8f40 1 -- start start 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 0x55f55b36bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f55b2421d0 con 0x55f55b34a320 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f55b24ef90 con 0x55f55b36b120 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f55b1e3c60 con 0x7fb574002340 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb58ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb58ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50672/0 (socket says 172.21.15.37:50672) 2024-09-21T06:32:09.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb58ec00640 1 -- 172.21.15.37:0/2266160475 learned_addr learned my addr 172.21.15.37:0/2266160475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb58e200640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 0x55f55b36bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.401+0000 7fb58d800640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 -- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 msgr2=0x55f55b36bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 0x55f55b36bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 -- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 msgr2=0x55f55b2c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 -- 172.21.15.37:0/2266160475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55b36ba70 con 0x55f55b34a320 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58e200640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 0x55f55b36bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58d800640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb58ec00640 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55f55b2999d0 tx=0x7fb588009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674057644553210f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb587e00640 1 -- 172.21.15.37:0/2266160475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5880092e0 con 0x55f55b34a320 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb587e00640 1 -- 172.21.15.37:0/2266160475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb588005580 con 0x55f55b34a320 2024-09-21T06:32:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.402+0000 7fb587e00640 1 -- 172.21.15.37:0/2266160475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588005990 con 0x55f55b34a320 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 -- 172.21.15.37:0/2266160475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 msgr2=0x55f55b34a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55f55b2999d0 tx=0x7fb588009d20 comp rx=0 tx=0).stop 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 -- 172.21.15.37:0/2266160475 shutdown_connections 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55b34a320 0x55f55b34a730 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/2266160475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb574002340 0x55f55b36bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 -- 172.21.15.37:0/2266160475 >> 172.21.15.37:0/2266160475 conn(0x55f55b1b4ac0 msgr2=0x55f55b368730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 -- 172.21.15.37:0/2266160475 shutdown_connections 2024-09-21T06:32:09.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.403+0000 7fb5950f8f40 1 -- 172.21.15.37:0/2266160475 wait complete. 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.404+0000 7fb5950f8f40 1 Processor -- start 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.404+0000 7fb5950f8f40 1 -- start start 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b34a320 0x55f55b3495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 0x55f55b2c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f55b36b820 con 0x7fb574002340 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f55b2dbdb0 con 0x55f55b34a320 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb5950f8f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f55b3f9900 con 0x55f55b36b120 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 0x55f55b2c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 0x55f55b2c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50680/0 (socket says 172.21.15.37:50680) 2024-09-21T06:32:09.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 -- 172.21.15.37:0/648478478 learned_addr learned my addr 172.21.15.37:0/648478478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58e200640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b34a320 0x55f55b3495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58d800640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 -- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f55b36b120 msgr2=0x55f55b2c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f55b36b120 0x55f55b2c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 -- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b34a320 msgr2=0x55f55b3495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f55b34a320 0x55f55b3495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.406+0000 7fb58ec00640 1 -- 172.21.15.37:0/648478478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55b34b360 con 0x7fb574002340 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb58ec00640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 0x55f55b2c32f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55f55b299650 tx=0x7fb588008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb586a00640 1 -- 172.21.15.37:0/648478478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5880092e0 con 0x7fb574002340 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb586a00640 1 -- 172.21.15.37:0/648478478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb588008b80 con 0x7fb574002340 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f55b34b640 con 0x7fb574002340 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb586a00640 1 -- 172.21.15.37:0/648478478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588015410 con 0x7fb574002340 2024-09-21T06:32:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.407+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55b3f9640 con 0x7fb574002340 2024-09-21T06:32:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.409+0000 7fb586a00640 1 -- 172.21.15.37:0/648478478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb588013030 con 0x7fb574002340 2024-09-21T06:32:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.410+0000 7fb586a00640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5700775a0 0x7fb570079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.410+0000 7fb586a00640 1 -- 172.21.15.37:0/648478478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb588099ff0 con 0x7fb574002340 2024-09-21T06:32:09.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.411+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55f55b3f9120 0x55f55b34c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.411+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 9.1d 9:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x55f55b40d270 con 0x55f55b3f9120 2024-09-21T06:32:09.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.411+0000 7fb58e200640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5700775a0 0x7fb570079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.411+0000 7fb58d800640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55f55b3f9120 0x55f55b34c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.412+0000 7fb58e200640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5700775a0 0x7fb570079ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb580056000 tx=0x7fb580001a60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.412+0000 7fb58d800640 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55f55b3f9120 0x55f55b34c630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb58d800640 1 -- 172.21.15.37:0/648478478 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v67'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb57c00a430 con 0x55f55b3f9120 2024-09-21T06:32:09.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55f55b3f9120 msgr2=0x55f55b34c630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55f55b3f9120 0x55f55b34c630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5700775a0 msgr2=0x7fb570079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb5700775a0 0x7fb570079ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb580056000 tx=0x7fb580001a60 comp rx=0 tx=0).stop 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 msgr2=0x55f55b2c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 --2- 172.21.15.37:0/648478478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574002340 0x55f55b2c32f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55f55b299650 tx=0x7fb588008780 comp rx=0 tx=0).stop 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb58ec00640 1 -- 172.21.15.37:0/648478478 reap_dead start 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 shutdown_connections 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 >> 172.21.15.37:0/648478478 conn(0x55f55b1b4ac0 msgr2=0x55f55b368730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 shutdown_connections 2024-09-21T06:32:09.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.414+0000 7fb5950f8f40 1 -- 172.21.15.37:0/648478478 wait complete. 2024-09-21T06:32:09.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:09.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2024-09-21T06:32:09.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 -- 172.21.15.37:0/2988508394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c84145cc0 msgr2=0x7f8c841604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 --2- 172.21.15.37:0/2988508394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c84145cc0 0x7f8c841604a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c005fd0 tx=0x7f8c6c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 -- 172.21.15.37:0/2988508394 shutdown_connections 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 --2- 172.21.15.37:0/2988508394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c84145cc0 0x7f8c841604a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 --2- 172.21.15.37:0/2988508394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c840b2e60 0x7f8c8415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 -- 172.21.15.37:0/2988508394 >> 172.21.15.37:0/2988508394 conn(0x7f8c840d0460 msgr2=0x7f8c8406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 -- 172.21.15.37:0/2988508394 shutdown_connections 2024-09-21T06:32:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.659+0000 7f8c83e00640 1 -- 172.21.15.37:0/2988508394 wait complete. 2024-09-21T06:32:09.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.662+0000 7f8c83e00640 1 Processor -- start 2024-09-21T06:32:09.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.662+0000 7f8c83e00640 1 -- start start 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 0x7f8c840c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c84145cc0 0x7f8c841e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8406fec0 con 0x7f8c840ceaf0 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8406fd40 con 0x7f8c840b2e60 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c83e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8c84070040 con 0x7f8c84145cc0 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50698/0 (socket says 172.21.15.37:50698) 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.663+0000 7f8c7be00640 1 -- 172.21.15.37:0/3048826903 learned_addr learned my addr 172.21.15.37:0/3048826903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 0x7f8c840c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c83400640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c84145cc0 0x7f8c841e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 -- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c84145cc0 msgr2=0x7f8c841e89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c84145cc0 0x7f8c841e89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 -- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 msgr2=0x7f8c840c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 0x7f8c840c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 -- 172.21.15.37:0/3048826903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c840c5390 con 0x7f8c840ceaf0 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 0x7f8c840c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c034040 tx=0x7f8c6c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7aa00640 1 -- 172.21.15.37:0/3048826903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c03e030 con 0x7f8c840ceaf0 2024-09-21T06:32:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7aa00640 1 -- 172.21.15.37:0/3048826903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c6c003ca0 con 0x7f8c840ceaf0 2024-09-21T06:32:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.664+0000 7f8c7aa00640 1 -- 172.21.15.37:0/3048826903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c035040 con 0x7f8c840ceaf0 2024-09-21T06:32:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.665+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841f5e40 con 0x7f8c840ceaf0 2024-09-21T06:32:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.665+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c840c50b0 con 0x7f8c840ceaf0 2024-09-21T06:32:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.667+0000 7f8c7aa00640 1 -- 172.21.15.37:0/3048826903 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c6c041030 con 0x7f8c840ceaf0 2024-09-21T06:32:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.668+0000 7f8c7aa00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 0x7f8c6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.668+0000 7f8c7aa00640 1 -- 172.21.15.37:0/3048826903 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8c6c0bb4f0 con 0x7f8c840ceaf0 2024-09-21T06:32:09.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.668+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 0x7f8c6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.669+0000 7f8c90bd9740 1 Processor -- start 2024-09-21T06:32:09.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.669+0000 7f8c90bd9740 1 -- start start 2024-09-21T06:32:09.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.669+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 0x7f8c6007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c7400f360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a17000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a17540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56402094c0c0 con 0x564020645680 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56402095ff90 con 0x56402061c7c0 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.670+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56402094f3c0 con 0x5640206249a0 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50700/0 (socket says 172.21.15.37:50700) 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 -- 172.21.15.37:0/2859049616 learned_addr learned my addr 172.21.15.37:0/2859049616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c83400640 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a17540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c7be00640 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a17000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 -- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 msgr2=0x564020a17540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a17540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 -- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 msgr2=0x564020a17000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a17000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 -- 172.21.15.37:0/2859049616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564020725a50 con 0x564020645680 2024-09-21T06:32:09.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.671+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400ff40 tx=0x7f8c74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ca4cab27b9673f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.672+0000 7f8c73400640 1 -- 172.21.15.37:0/2859049616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74022070 con 0x564020645680 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.672+0000 7f8c73400640 1 -- 172.21.15.37:0/2859049616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c7401e650 con 0x564020645680 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.672+0000 7f8c73400640 1 -- 172.21.15.37:0/2859049616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401ea60 con 0x564020645680 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.672+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2859049616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 msgr2=0x564020a34330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400ff40 tx=0x7f8c74012000 comp rx=0 tx=0).stop 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2859049616 shutdown_connections 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a17540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2859049616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a34330 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2859049616 >> 172.21.15.37:0/2859049616 conn(0x5640206397d0 msgr2=0x5640209ed220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2859049616 shutdown_connections 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.673+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2859049616 wait complete. 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.674+0000 7f8c90bd9740 1 -- start start 2024-09-21T06:32:09.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.674+0000 7f8c90bd9740 1 Processor -- start 2024-09-21T06:32:09.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640208f4af0 con 0x564020645680 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5640209531d0 con 0x56402061c7c0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5640206bfa30 con 0x5640206249a0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c90bd9740 10 client.0 _ll_register_callbacks cb 0x5640208fab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50710/0 (socket says 172.21.15.37:50710) 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 -- 172.21.15.37:0/2629429337 learned_addr learned my addr 172.21.15.37:0/2629429337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c7be00640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 -- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 msgr2=0x564020a2e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a2e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 -- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 msgr2=0x564020a05340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a05340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.676+0000 7f8c83400640 1 -- 172.21.15.37:0/2629429337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402068e990 con 0x564020645680 2024-09-21T06:32:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c83400640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c84072b60 tx=0x7f8c7c001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731a5d27434be23b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0145e0 con 0x564020645680 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8c7c014780 con 0x564020645680 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7401e650 con 0x564020645680 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c022050 con 0x564020645680 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c90bd9740 10 client.6588 fetch_fsmap learned FSMap version 60 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5640207861e0 con 0x564020645680 2024-09-21T06:32:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.677+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c7401e650 con 0x564020645680 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) ==== 3558+0+0 (secure 0 0 0) 0x7f8c7c025020 con 0x564020645680 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 10 client.6588 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 10 client.6588 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 20 client.6588 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020604250 0x564020a2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 4 client.6588 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564020961ee0 con 0x564020604250 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.678+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020604250 0x564020a2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.679+0000 7f8c82a00640 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020604250 0x564020a2a180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9539f34dca43a329 server_cookie=8097bc445135871a in_seq=0 out_seq=0 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.679+0000 7f8c67e00640 10 client.6588 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.679+0000 7f8c67e00640 1 -- 172.21.15.37:0/2629429337 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564020961ee0 con 0x564020604250 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.679+0000 7f8c67e00640 10 client.6588 handle_command_reply: tid=0 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.679+0000 7f8c90bd9740 1 client.6588 shutdown 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 msgr2=0x564020a2ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c84072b60 tx=0x7f8c7c001c10 comp rx=0 tx=0).stop 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 shutdown_connections 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020604250 0x564020a2a180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564020645680 0x564020a2ebd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c67e00640 0 client.6588 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5640206249a0 0x564020a2e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/2629429337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56402061c7c0 0x564020a05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 >> 172.21.15.37:0/2629429337 conn(0x5640206397d0 msgr2=0x564020a067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 shutdown_connections 2024-09-21T06:32:09.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 1 -- 172.21.15.37:0/2629429337 wait complete. 2024-09-21T06:32:09.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.680+0000 7f8c90bd9740 20 client.6588 trim_cache size 0 max 16384 2024-09-21T06:32:09.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.699+0000 7f8c90bd9740 1 Processor -- start 2024-09-21T06:32:09.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.699+0000 7f8c90bd9740 1 -- start start 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.700+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56402061c7c0 0x564020a13a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.700+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.700+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564020645680 0x5640209ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84202400 con 0x5640206249a0 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8c842015a0 con 0x564020645680 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8c84201cd0 con 0x56402061c7c0 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50720/0 (socket says 172.21.15.37:50720) 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 -- 172.21.15.37:0/3949921814 learned_addr learned my addr 172.21.15.37:0/3949921814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56402061c7c0 0x564020a13a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c83400640 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564020645680 0x5640209ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 -- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56402061c7c0 msgr2=0x564020a13a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56402061c7c0 0x564020a13a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 -- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564020645680 msgr2=0x5640209ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564020645680 0x5640209ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.701+0000 7f8c7be00640 1 -- 172.21.15.37:0/3949921814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564020723d50 con 0x5640206249a0 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.702+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c080320 tx=0x7f8c6c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b984742b67bea28f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.702+0000 7f8c72a00640 1 -- 172.21.15.37:0/3949921814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c048040 con 0x5640206249a0 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.702+0000 7f8c72a00640 1 -- 172.21.15.37:0/3949921814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c6c082bc0 con 0x5640206249a0 2024-09-21T06:32:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.702+0000 7f8c72a00640 1 -- 172.21.15.37:0/3949921814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c08ae60 con 0x5640206249a0 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3949921814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 msgr2=0x564020a13fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c080320 tx=0x7f8c6c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3949921814 shutdown_connections 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640206249a0 0x564020a13fa0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564020645680 0x5640209ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3949921814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56402061c7c0 0x564020a13a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3949921814 >> 172.21.15.37:0/3949921814 conn(0x5640206397d0 msgr2=0x564020a04d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3949921814 shutdown_connections 2024-09-21T06:32:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.703+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3949921814 wait complete. 2024-09-21T06:32:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.704+0000 7f8c90bd9740 1 -- start start 2024-09-21T06:32:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.704+0000 7f8c90bd9740 1 Processor -- start 2024-09-21T06:32:09.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5640206249a0 0x564020a3f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 0x564020a3fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c840ae630 con 0x56402061c7c0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564020a32eb0 con 0x5640206249a0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564020ab58e0 con 0x564020645680 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.705+0000 7f8c90bd9740 10 client.0 _ll_register_callbacks cb 0x7f8c7c005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50726/0 (socket says 172.21.15.37:50726) 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 -- 172.21.15.37:0/3914808904 learned_addr learned my addr 172.21.15.37:0/3914808904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5640206249a0 0x564020a3f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c83400640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 0x564020a3fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 -- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 msgr2=0x564020a3fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 0x564020a3fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 -- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5640206249a0 msgr2=0x564020a3f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5640206249a0 0x564020a3f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 -- 172.21.15.37:0/3914808904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564020a3b340 con 0x56402061c7c0 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c83400640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 0x564020a3fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c82a00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c74021000 tx=0x7f8c7400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e68cac9ca11536b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.706+0000 7f8c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74022040 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8c74004220 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6c082bc0 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8c74042050 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 10 client.6594 fetch_fsmap learned FSMap version 60 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x564020a29b50 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c6c082bc0 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) ==== 3558+0+0 (secure 0 0 0) 0x7f8c74045020 con 0x56402061c7c0 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 10 client.6594 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T06:32:09.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 10 client.6594 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 20 client.6594 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.707+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020a29f40 0x564020a2a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.708+0000 7f8c90bd9740 4 client.6594 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.708+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x56402095d990 con 0x564020a29f40 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.708+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020a29f40 0x564020a2a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.708+0000 7f8c7be00640 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020a29f40 0x564020a2a350 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2720cf1d1bc631 server_cookie=bb65766116eaea19 in_seq=0 out_seq=0 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.708+0000 7f8c67e00640 10 client.6594 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c67e00640 1 -- 172.21.15.37:0/3914808904 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56402095d990 con 0x564020a29f40 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c67e00640 10 client.6594 handle_command_reply: tid=0 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 client.6594 shutdown 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 msgr2=0x564020a3ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c74021000 tx=0x7f8c7400fcb0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.714 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f34e7400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1) n(v3 rc2024-09-21T06:31:52.126583+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5644cb46a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:31:52.126583+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 shutdown_connections 2024-09-21T06:32:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x564020a29f40 0x564020a2a350 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564020645680 0x564020a3fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5640206249a0 0x564020a3f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c67e00640 0 client.6594 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 --2- 172.21.15.37:0/3914808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56402061c7c0 0x564020a3ae00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.709+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 >> 172.21.15.37:0/3914808904 conn(0x5640206397d0 msgr2=0x564020a04d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 shutdown_connections 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c90bd9740 1 -- 172.21.15.37:0/3914808904 wait complete. 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c90bd9740 20 client.6594 trim_cache size 0 max 16384 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-21T06:32:09.715 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "1b29e3ab-8472-4965-b8aa-1954b7e38d82", 2024-09-21T06:32:09.716 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-21T06:32:09.716 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:09.716 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:32:09.710+0000 7efc40600640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1) n(v2 rc2024-09-21T06:31:52.126583+0000 4=0+4) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56113f526100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:31:52.126583+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.717 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:32:09.711+0000 7fa37dc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1) n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560c7684e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.717 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:32:09.711+0000 7fa37dc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/AsLsXsFsx@2},l=6220 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560c7684e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.717 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:32:09.712+0000 7efc48800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1) n(v1 rc2024-09-21T06:31:52.126583+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6220=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56113f526680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:31:52.126583+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 msgr2=0x7f8c6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 0x7f8c6007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c7400f360 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 msgr2=0x7f8c84143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c034040 tx=0x7f8c6c0038a0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 shutdown_connections 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c60078b90 0x7f8c6007b0a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c840ceaf0 0x7f8c84143d30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c84145cc0 0x7f8c841e89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 --2- 172.21.15.37:0/3048826903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c840b2e60 0x7f8c840c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.710+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 >> 172.21.15.37:0/3048826903 conn(0x7f8c840d0460 msgr2=0x7f8c840b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.711+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 shutdown_connections 2024-09-21T06:32:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.711+0000 7f8c83e00640 1 -- 172.21.15.37:0/3048826903 wait complete. 2024-09-21T06:32:09.719 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:32:09.711+0000 7f34ef600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1) n(v4 rc2024-09-21T06:31:52.126583+0000 5=0+5) (inest sync->mix g=1,2) caps={6220=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5644cb46d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:31:52.126583+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:32:09.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:09.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-21T06:32:09.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 -- 172.21.15.37:0/979077779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ccfc0 msgr2=0x7f4e6c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 --2- 172.21.15.37:0/979077779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ccfc0 0x7f4e6c0b4f90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cd790 tx=0x7f4e5402a530 comp rx=0 tx=0).stop 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 -- 172.21.15.37:0/979077779 shutdown_connections 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 --2- 172.21.15.37:0/979077779 >> v2:172.21.15.150:3300/0 conn(0x7f4e6c156c60 0x7f4e6c0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 --2- 172.21.15.37:0/979077779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ccfc0 0x7f4e6c0b4f90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 -- 172.21.15.37:0/979077779 >> 172.21.15.37:0/979077779 conn(0x7f4e6c06f920 msgr2=0x7f4e6c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 -- 172.21.15.37:0/979077779 shutdown_connections 2024-09-21T06:32:09.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.966+0000 7f4e71a00640 1 -- 172.21.15.37:0/979077779 wait complete. 2024-09-21T06:32:09.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.968+0000 7f4e71a00640 1 Processor -- start 2024-09-21T06:32:09.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.968+0000 7f4e71a00640 1 -- start start 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 0x7f4e6c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 0x7f4e6c1e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c06d560 con 0x7f4e6c06e370 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c06d3e0 con 0x7f4e6c156c60 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c06d6e0 con 0x7f4e6c0745b0 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50742/0 (socket says 172.21.15.37:50742) 2024-09-21T06:32:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e6be00640 1 -- 172.21.15.37:0/2360598460 learned_addr learned my addr 172.21.15.37:0/2360598460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e71000640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 0x7f4e6c1e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.970+0000 7f4e63e00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 0x7f4e6c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 -- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 msgr2=0x7f4e6c1e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 0x7f4e6c1e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 -- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 msgr2=0x7f4e6c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 0x7f4e6c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 -- 172.21.15.37:0/2360598460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c152fc0 con 0x7f4e6c06e370 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e71000640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 0x7f4e6c1e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e63e00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 0x7f4e6c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e6be00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c3f0 tx=0x7f4e5c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e68c00640 1 -- 172.21.15.37:0/2360598460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c017070 con 0x7f4e6c06e370 2024-09-21T06:32:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e68c00640 1 -- 172.21.15.37:0/2360598460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c0035f0 con 0x7f4e6c06e370 2024-09-21T06:32:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c1f5100 con 0x7f4e6c06e370 2024-09-21T06:32:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e68c00640 1 -- 172.21.15.37:0/2360598460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003a40 con 0x7f4e6c06e370 2024-09-21T06:32:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.971+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c1f56a0 con 0x7f4e6c06e370 2024-09-21T06:32:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.974+0000 7f4e68c00640 1 -- 172.21.15.37:0/2360598460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e5c01a070 con 0x7f4e6c06e370 2024-09-21T06:32:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.975+0000 7f4e68c00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 0x7f4e4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.975+0000 7f4e68c00640 1 -- 172.21.15.37:0/2360598460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4e5c09a260 con 0x7f4e6c06e370 2024-09-21T06:32:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.975+0000 7f4e7a04d740 1 Processor -- start 2024-09-21T06:32:09.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.975+0000 7f4e63e00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 0x7f4e4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.975+0000 7f4e7a04d740 1 -- start start 2024-09-21T06:32:09.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.976+0000 7f4e63e00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 0x7f4e4807b0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e54034000 tx=0x7f4e54004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c6580400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6580940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6c64b61c0 con 0x55b6c61af390 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6c64c9870 con 0x55b6c61864d0 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b6c64b8ca0 con 0x55b6c618e6b0 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50758/0 (socket says 172.21.15.37:50758) 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e6be00640 1 -- 172.21.15.37:0/4104941359 learned_addr learned my addr 172.21.15.37:0/4104941359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e63e00640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c6580400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.977+0000 7f4e71000640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6580940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 -- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 msgr2=0x55b6c6580940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6580940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 -- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 msgr2=0x55b6c6580400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c6580400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 -- 172.21.15.37:0/4104941359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c62917d0 con 0x55b6c61af390 2024-09-21T06:32:09.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e63e00640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c6580400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e6be00640 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0984a0 tx=0x7f4e5c05ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c2067593655d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e60c00640 1 -- 172.21.15.37:0/4104941359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c061e00 con 0x55b6c61af390 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e60c00640 1 -- 172.21.15.37:0/4104941359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c0668e0 con 0x55b6c61af390 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.978+0000 7f4e60c00640 1 -- 172.21.15.37:0/4104941359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c069e10 con 0x55b6c61af390 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 -- 172.21.15.37:0/4104941359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 msgr2=0x55b6c659d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0984a0 tx=0x7f4e5c05ddd0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 -- 172.21.15.37:0/4104941359 shutdown_connections 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6580940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c6580400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/4104941359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c659d730 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 -- 172.21.15.37:0/4104941359 >> 172.21.15.37:0/4104941359 conn(0x55b6c61a34e0 msgr2=0x55b6c6556620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 -- 172.21.15.37:0/4104941359 shutdown_connections 2024-09-21T06:32:09.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.979+0000 7f4e7a04d740 1 -- 172.21.15.37:0/4104941359 wait complete. 2024-09-21T06:32:09.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.980+0000 7f4e7a04d740 1 -- start start 2024-09-21T06:32:09.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.980+0000 7f4e7a04d740 1 Processor -- start 2024-09-21T06:32:09.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 0x55b6c656e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 0x55b6c65970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6c645f410 con 0x55b6c61af390 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6c64bcab0 con 0x55b6c618e6b0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b6c6229740 con 0x55b6c61864d0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e7a04d740 10 client.0 _ll_register_callbacks cb 0x55b6c643c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e71000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e71000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50760/0 (socket says 172.21.15.37:50760) 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e71000640 1 -- 172.21.15.37:0/1294630582 learned_addr learned my addr 172.21.15.37:0/1294630582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e63e00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 0x55b6c65970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.982+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 0x55b6c656e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 -- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 msgr2=0x55b6c656e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 0x55b6c656e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 -- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 msgr2=0x55b6c65970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 0x55b6c65970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 -- 172.21.15.37:0/1294630582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c621a970 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e63e00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 0x55b6c65970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 0x55b6c656e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e71000640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0ce5d0 tx=0x7f4e640108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4904c2d08b3d269 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e59600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004d20 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4e6401ac40 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c0668e0 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4e6401ae40 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e7a04d740 10 client.6609 fetch_fsmap learned FSMap version 60 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.983+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55b6c61f86a0 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c0668e0 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) ==== 3558+0+0 (secure 0 0 0) 0x7f4e64021050 con 0x55b6c61af390 2024-09-21T06:32:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 10 client.6609 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 10 client.6609 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 20 client.6609 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c616df60 0x55b6c65958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 4 client.6609 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c64cb7c0 con 0x55b6c616df60 2024-09-21T06:32:09.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.984+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c616df60 0x55b6c65958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:09.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.985+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c616df60 0x55b6c65958b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be08d95375c75cc1 server_cookie=44b50ef182fd3362 in_seq=0 out_seq=0 2024-09-21T06:32:09.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.985+0000 7f4e59600640 10 client.6609 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.985+0000 7f4e59600640 1 -- 172.21.15.37:0/1294630582 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6c64cb7c0 con 0x55b6c616df60 2024-09-21T06:32:09.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.985+0000 7f4e59600640 10 client.6609 handle_command_reply: tid=0 2024-09-21T06:32:09.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.985+0000 7f4e7a04d740 1 client.6609 shutdown 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 msgr2=0x55b6c6555cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0ce5d0 tx=0x7f4e640108e0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 shutdown_connections 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c616df60 0x55b6c65958b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6555cd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c618e6b0 0x55b6c65970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1294630582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c61864d0 0x55b6c656e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e59600640 0 client.6609 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 >> 172.21.15.37:0/1294630582 conn(0x55b6c61a34e0 msgr2=0x55b6c659e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 shutdown_connections 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1294630582 wait complete. 2024-09-21T06:32:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:09.986+0000 7f4e7a04d740 20 client.6609 trim_cache size 0 max 16384 2024-09-21T06:32:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.005+0000 7f4e7a04d740 1 Processor -- start 2024-09-21T06:32:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.005+0000 7f4e7a04d740 1 -- start start 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6591730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 0x55b6c659e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c202460 con 0x55b6c61864d0 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c201600 con 0x55b6c61af390 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c201d30 con 0x55b6c618e6b0 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50768/0 (socket says 172.21.15.37:50768) 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 -- 172.21.15.37:0/3543219718 learned_addr learned my addr 172.21.15.37:0/3543219718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e63e00640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6591730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e71000640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 0x55b6c659e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 -- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 msgr2=0x55b6c6591730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6591730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 -- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 msgr2=0x55b6c659e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 0x55b6c659e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e6be00640 1 -- 172.21.15.37:0/3543219718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c6591dc0 con 0x55b6c61864d0 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.007+0000 7f4e71000640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 0x55b6c659e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.008+0000 7f4e6be00640 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000fb0 tx=0x7f4e5c098430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a76c1edaba7a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.008+0000 7f4e5be00640 1 -- 172.21.15.37:0/3543219718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c061e00 con 0x55b6c61864d0 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.008+0000 7f4e5be00640 1 -- 172.21.15.37:0/3543219718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c0668e0 con 0x55b6c61864d0 2024-09-21T06:32:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.008+0000 7f4e5be00640 1 -- 172.21.15.37:0/3543219718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c06b560 con 0x55b6c61864d0 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 -- 172.21.15.37:0/3543219718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 msgr2=0x55b6c659bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000fb0 tx=0x7f4e5c098430 comp rx=0 tx=0).stop 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 -- 172.21.15.37:0/3543219718 shutdown_connections 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61af390 0x55b6c659e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6591730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/3543219718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61864d0 0x55b6c659bf20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 -- 172.21.15.37:0/3543219718 >> 172.21.15.37:0/3543219718 conn(0x55b6c61a34e0 msgr2=0x55b6c65578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 -- 172.21.15.37:0/3543219718 shutdown_connections 2024-09-21T06:32:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.009+0000 7f4e7a04d740 1 -- 172.21.15.37:0/3543219718 wait complete. 2024-09-21T06:32:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.010+0000 7f4e7a04d740 1 -- start start 2024-09-21T06:32:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.010+0000 7f4e7a04d740 1 Processor -- start 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.011+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c65928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.011+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6603150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.011+0000 7f4e7a04d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.011+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c073a80 con 0x55b6c61af390 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.011+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6603690 con 0x55b6c61864d0 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e7a04d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b6c6554e10 con 0x55b6c618e6b0 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e7a04d740 10 client.0 _ll_register_callbacks cb 0x7f4e64009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50776/0 (socket says 172.21.15.37:50776) 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 -- 172.21.15.37:0/1809782534 learned_addr learned my addr 172.21.15.37:0/1809782534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c65928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e63e00640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6603150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 -- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 msgr2=0x55b6c6603150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6603150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 -- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 msgr2=0x55b6c65928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c65928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 -- 172.21.15.37:0/1809782534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c6552d10 con 0x55b6c61af390 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e71000640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000f80 tx=0x7f4e64011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4c7c52a5bc32ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.012+0000 7f4e59600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64010de0 con 0x55b6c61af390 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4e6402dc70 con 0x55b6c61af390 2024-09-21T06:32:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c0668e0 con 0x55b6c61af390 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4e6402de70 con 0x55b6c61af390 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 10 client.6615 fetch_fsmap learned FSMap version 60 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55b6c6556ba0 con 0x55b6c61af390 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c0668e0 con 0x55b6c61af390 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) ==== 3558+0+0 (secure 0 0 0) 0x7f4e64021050 con 0x55b6c61af390 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 10 client.6615 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 10 client.6615 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.013+0000 7f4e7a04d740 20 client.6615 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c65545c0 0x55b6c65bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e7a04d740 4 client.6615 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6c64c7270 con 0x55b6c65545c0 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c65545c0 0x55b6c65bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e6be00640 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c65545c0 0x55b6c65bdea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f7ae4323c5508f server_cookie=80d807e4ca7e8541 in_seq=0 out_seq=0 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.014+0000 7f4e59600640 10 client.6615 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e59600640 1 -- 172.21.15.37:0/1809782534 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x55b6c64c7270 con 0x55b6c65545c0 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e59600640 10 client.6615 handle_command_reply: tid=0 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 client.6615 shutdown 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 msgr2=0x55b6c6554080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000f80 tx=0x7f4e64011d70 comp rx=0 tx=0).stop 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 shutdown_connections 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55b6c65545c0 0x55b6c65bdea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6c61af390 0x55b6c6554080 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b6c618e6b0 0x55b6c6603150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 --2- 172.21.15.37:0/1809782534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c61864d0 0x55b6c65928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 >> 172.21.15.37:0/1809782534 conn(0x55b6c61a34e0 msgr2=0x55b6c65944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e59600640 0 client.6615 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 shutdown_connections 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 1 -- 172.21.15.37:0/1809782534 wait complete. 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.015+0000 7f4e7a04d740 20 client.6615 trim_cache size 0 max 16384 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:32:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "1b29e3ab-8472-4965-b8aa-1954b7e38d82": { 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "1b29e3ab-8472-4965-b8aa-1954b7e38d82", 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "1b29e3ab-8472-4965-b8aa-1954b7e38d82": { 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:32:09.709354+0000", 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:32:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 msgr2=0x7f4e4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 0x7f4e4807b0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e54034000 tx=0x7f4e54004730 comp rx=0 tx=0).stop 2024-09-21T06:32:10.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 msgr2=0x7f4e6c152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c3f0 tx=0x7f4e5c00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 shutdown_connections 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e48078bc0 0x7f4e4807b0d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c0745b0 0x7f4e6c1e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c156c60 0x7f4e6c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 --2- 172.21.15.37:0/2360598460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c06e370 0x7f4e6c152a80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 >> 172.21.15.37:0/2360598460 conn(0x7f4e6c06f920 msgr2=0x7f4e6c154840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 shutdown_connections 2024-09-21T06:32:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:10.016+0000 7f4e71a00640 1 -- 172.21.15.37:0/2360598460 wait complete. 2024-09-21T06:32:10.043 INFO:tasks.cephfs.filesystem:scrub status for tag:1b29e3ab-8472-4965-b8aa-1954b7e38d82 - {'path': '/d1/d2/d3', 'tag': '1b29e3ab-8472-4965-b8aa-1954b7e38d82', 'options': 'recursive,force'} 2024-09-21T06:32:14.567 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:32:14.566+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:32:20.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:32:20.124 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:32:22.004 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:32:22.002+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:32:40.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:40.044 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 -- 172.21.15.37:0/964976839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0280ccfc0 msgr2=0x7ff0280b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 --2- 172.21.15.37:0/964976839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0280ccfc0 0x7ff0280b4f90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff010009830 tx=0x7ff01002a510 comp rx=0 tx=0).stop 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 -- 172.21.15.37:0/964976839 shutdown_connections 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 --2- 172.21.15.37:0/964976839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff028156c60 0x7ff0280cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 --2- 172.21.15.37:0/964976839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0280ccfc0 0x7ff0280b4f90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 -- 172.21.15.37:0/964976839 >> 172.21.15.37:0/964976839 conn(0x7ff02806f920 msgr2=0x7ff0280b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 -- 172.21.15.37:0/964976839 shutdown_connections 2024-09-21T06:32:40.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.283+0000 7ff027e00640 1 -- 172.21.15.37:0/964976839 wait complete. 2024-09-21T06:32:40.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.286+0000 7ff027e00640 1 Processor -- start 2024-09-21T06:32:40.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.286+0000 7ff027e00640 1 -- start start 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff02806e370 0x7ff0280c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff028156c60 0x7ff028145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff02806d610 con 0x7ff028147a80 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff02806d490 con 0x7ff028156c60 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff02806d790 con 0x7ff02806e370 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.287+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55492/0 (socket says 172.21.15.37:55492) 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 -- 172.21.15.37:0/3265185580 learned_addr learned my addr 172.21.15.37:0/3265185580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff026a00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff02806e370 0x7ff0280c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff01fe00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff028156c60 0x7ff028145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 -- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff02806e370 msgr2=0x7ff0280c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff02806e370 0x7ff0280c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 -- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff028156c60 msgr2=0x7ff028145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff028156c60 0x7ff028145230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 -- 172.21.15.37:0/3265185580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028148740 con 0x7ff028147a80 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.288+0000 7ff027400640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff02000e960 tx=0x7ff02000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.289+0000 7ff01ea00640 1 -- 172.21.15.37:0/3265185580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004d20 con 0x7ff028147a80 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.289+0000 7ff01ea00640 1 -- 172.21.15.37:0/3265185580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff02000cb20 con 0x7ff028147a80 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.289+0000 7ff01ea00640 1 -- 172.21.15.37:0/3265185580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020005020 con 0x7ff028147a80 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.289+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0280c93f0 con 0x7ff028147a80 2024-09-21T06:32:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.289+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0280c99b0 con 0x7ff028147a80 2024-09-21T06:32:40.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.291+0000 7ff01ea00640 1 -- 172.21.15.37:0/3265185580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0200051c0 con 0x7ff028147a80 2024-09-21T06:32:40.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.292+0000 7ff01ea00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 0x7ff00407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.292+0000 7ff01ea00640 1 -- 172.21.15.37:0/3265185580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7ff02009b8d0 con 0x7ff028147a80 2024-09-21T06:32:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.293+0000 7ff026a00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 0x7ff00407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.293+0000 7ff034852740 1 Processor -- start 2024-09-21T06:32:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.293+0000 7ff034852740 1 -- start start 2024-09-21T06:32:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.293+0000 7ff026a00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 0x7ff00407b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0180046b0 tx=0x7ff018007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c109ea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c109201c0 con 0x557c10619390 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c10933870 con 0x557c105f86b0 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c10922ca0 con 0x557c105f04d0 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55502/0 (socket says 172.21.15.37:55502) 2024-09-21T06:32:40.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 -- 172.21.15.37:0/1780112802 learned_addr learned my addr 172.21.15.37:0/1780112802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff027400640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff01fe00640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c109ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 -- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 msgr2=0x557c109ea400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c109ea400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 -- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 msgr2=0x557c109ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff026a00640 1 -- 172.21.15.37:0/1780112802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c106fb7d0 con 0x557c10619390 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.295+0000 7ff027400640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.296+0000 7ff01fe00640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c109ea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.296+0000 7ff026a00640 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff01800fba0 tx=0x7ff018012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396f51ef911bfbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.296+0000 7ff017400640 1 -- 172.21.15.37:0/1780112802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01801b050 con 0x557c10619390 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.296+0000 7ff017400640 1 -- 172.21.15.37:0/1780112802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff018025070 con 0x557c10619390 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.296+0000 7ff017400640 1 -- 172.21.15.37:0/1780112802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018020440 con 0x557c10619390 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 -- 172.21.15.37:0/1780112802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 msgr2=0x557c10a07730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff01800fba0 tx=0x7ff018012000 comp rx=0 tx=0).stop 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 -- 172.21.15.37:0/1780112802 shutdown_connections 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c109ea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 --2- 172.21.15.37:0/1780112802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a07730 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 -- 172.21.15.37:0/1780112802 >> 172.21.15.37:0/1780112802 conn(0x557c1060d4e0 msgr2=0x557c109c0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 -- 172.21.15.37:0/1780112802 shutdown_connections 2024-09-21T06:32:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.297+0000 7ff034852740 1 -- 172.21.15.37:0/1780112802 wait complete. 2024-09-21T06:32:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.298+0000 7ff034852740 1 -- start start 2024-09-21T06:32:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.298+0000 7ff034852740 1 Processor -- start 2024-09-21T06:32:40.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a05d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 0x557c109e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c108c9410 con 0x557c105f86b0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c10926ab0 con 0x557c10619390 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c10693740 con 0x557c105f04d0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff034852740 10 client.0 _ll_register_callbacks cb 0x557c108a6360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55512/0 (socket says 172.21.15.37:55512) 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff01fe00640 1 -- 172.21.15.37:0/3742334213 learned_addr learned my addr 172.21.15.37:0/3742334213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff027400640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 0x557c109e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.300+0000 7ff026a00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a05d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 -- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 msgr2=0x557c10a05d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a05d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 -- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 msgr2=0x557c109e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 0x557c109e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 -- 172.21.15.37:0/3742334213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c10684970 con 0x557c105f86b0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff027400640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 0x557c109e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff01fe00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff010009950 tx=0x7ff010003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55e15c49259a61f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff00be00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010003e20 con 0x557c105f86b0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7ff010005020 con 0x557c105f86b0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.301+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018025070 con 0x557c105f86b0 2024-09-21T06:32:40.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff01004c050 con 0x557c105f86b0 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 10 client.6630 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x557c106626a0 con 0x557c105f86b0 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff018025070 con 0x557c105f86b0 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7ff01004f020 con 0x557c105f86b0 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 10 client.6630 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 10 client.6630 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 20 client.6630 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c105d7f60 0x557c10a182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 4 client.6630 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c109357c0 con 0x557c105d7f60 2024-09-21T06:32:40.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.302+0000 7ff026a00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c105d7f60 0x557c10a182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.303+0000 7ff026a00640 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c105d7f60 0x557c10a182c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f53e2ca5b887a9 server_cookie=e4a17c67b95aa912 in_seq=0 out_seq=0 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.303+0000 7ff00be00640 10 client.6630 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff00be00640 1 -- 172.21.15.37:0/3742334213 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c109357c0 con 0x557c105d7f60 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff00be00640 10 client.6630 handle_command_reply: tid=0 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 client.6630 shutdown 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 msgr2=0x557c109e37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff010009950 tx=0x7ff010003820 comp rx=0 tx=0).stop 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 shutdown_connections 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c105d7f60 0x557c10a182c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c105f86b0 0x557c109e37d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10619390 0x557c109e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 --2- 172.21.15.37:0/3742334213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a05d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff00be00640 0 client.6630 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 >> 172.21.15.37:0/3742334213 conn(0x557c1060d4e0 msgr2=0x557c109ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 shutdown_connections 2024-09-21T06:32:40.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.304+0000 7ff034852740 1 -- 172.21.15.37:0/3742334213 wait complete. 2024-09-21T06:32:40.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.305+0000 7ff034852740 20 client.6630 trim_cache size 0 max 16384 2024-09-21T06:32:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.323+0000 7ff034852740 1 Processor -- start 2024-09-21T06:32:40.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.323+0000 7ff034852740 1 -- start start 2024-09-21T06:32:40.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.324+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f04d0 0x557c109baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f86b0 0x557c109bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff034852740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff034852740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c109bbeb0 con 0x557c10619390 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0282027f0 con 0x557c105f04d0 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff028201990 con 0x557c105f86b0 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55528/0 (socket says 172.21.15.37:55528) 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 -- 172.21.15.37:0/1092092383 learned_addr learned my addr 172.21.15.37:0/1092092383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff01fe00640 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f86b0 0x557c109bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff026a00640 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f04d0 0x557c109baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 -- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f86b0 msgr2=0x557c109bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f86b0 0x557c109bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 -- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f04d0 msgr2=0x557c109baa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f04d0 0x557c109baa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.325+0000 7ff027400640 1 -- 172.21.15.37:0/1092092383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c109ffa60 con 0x557c10619390 2024-09-21T06:32:40.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.326+0000 7ff027400640 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff020009450 tx=0x7ff02009b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6976a636335b6208 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.326+0000 7ff016a00640 1 -- 172.21.15.37:0/1092092383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02001f070 con 0x557c10619390 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.326+0000 7ff016a00640 1 -- 172.21.15.37:0/1092092383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff020067e20 con 0x557c10619390 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.326+0000 7ff016a00640 1 -- 172.21.15.37:0/1092092383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020085d60 con 0x557c10619390 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 -- 172.21.15.37:0/1092092383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 msgr2=0x557c10a28260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff020009450 tx=0x7ff02009b040 comp rx=0 tx=0).stop 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 -- 172.21.15.37:0/1092092383 shutdown_connections 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c10a28260 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f86b0 0x557c109bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 --2- 172.21.15.37:0/1092092383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f04d0 0x557c109baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 -- 172.21.15.37:0/1092092383 >> 172.21.15.37:0/1092092383 conn(0x557c1060d4e0 msgr2=0x557c10a056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 -- 172.21.15.37:0/1092092383 shutdown_connections 2024-09-21T06:32:40.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.327+0000 7ff034852740 1 -- 172.21.15.37:0/1092092383 wait complete. 2024-09-21T06:32:40.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.328+0000 7ff034852740 1 -- start start 2024-09-21T06:32:40.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.328+0000 7ff034852740 1 Processor -- start 2024-09-21T06:32:40.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281f9710 con 0x557c10619390 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0282020c0 con 0x557c105f86b0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.329+0000 7ff034852740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c10a88770 con 0x557c105f04d0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff034852740 10 client.0 _ll_register_callbacks cb 0x7ff0100044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55534/0 (socket says 172.21.15.37:55534) 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 -- 172.21.15.37:0/3413160687 learned_addr learned my addr 172.21.15.37:0/3413160687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff01fe00640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff026a00640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 -- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 msgr2=0x557c10a82050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a82050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 -- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 msgr2=0x557c109baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 -- 172.21.15.37:0/3413160687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c109ea4a0 con 0x557c10619390 2024-09-21T06:32:40.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff027400640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff02001d040 tx=0x7ff02005ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b3d6ba4edf9160 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.330+0000 7ff00be00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02001f070 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7ff020085b40 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff020067e20 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff02006cd60 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 10 client.6636 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x557c109d9c20 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff020067e20 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7ff02000f040 con 0x557c10619390 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 10 client.6636 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 10 client.6636 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 20 client.6636 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c109fd6c0 0x557c109fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 4 client.6636 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.331+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c10931270 con 0x557c109fd6c0 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff026a00640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c109fd6c0 0x557c109fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff026a00640 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c109fd6c0 0x557c109fdad0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf056c4cfe68ab server_cookie=b27e55423e29360 in_seq=0 out_seq=0 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff00be00640 10 client.6636 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff00be00640 1 -- 172.21.15.37:0/3413160687 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+382 (crc 0 0 0) 0x557c10931270 con 0x557c109fd6c0 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff00be00640 10 client.6636 handle_command_reply: tid=0 2024-09-21T06:32:40.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.332+0000 7ff034852740 1 client.6636 shutdown 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 msgr2=0x557c109baf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff02001d040 tx=0x7ff02005ef70 comp rx=0 tx=0).stop 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 shutdown_connections 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x557c109fd6c0 0x557c109fdad0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c10619390 0x557c109baf90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c105f86b0 0x557c109baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 --2- 172.21.15.37:0/3413160687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c105f04d0 0x557c10a82050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff00be00640 0 client.6636 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 >> 172.21.15.37:0/3413160687 conn(0x557c1060d4e0 msgr2=0x557c109d9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 shutdown_connections 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 1 -- 172.21.15.37:0/3413160687 wait complete. 2024-09-21T06:32:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.333+0000 7ff034852740 20 client.6636 trim_cache size 0 max 16384 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {}, 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "1b29e3ab-8472-4965-b8aa-1954b7e38d82": { 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:32:09.709354+0000", 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:32:40.339 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:40.340 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 msgr2=0x7ff00407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 0x7ff00407b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0180046b0 tx=0x7ff018007eb0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 msgr2=0x7ff028147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff02000e960 tx=0x7ff02000ee20 comp rx=0 tx=0).stop 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 shutdown_connections 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff004078b90 0x7ff00407b0a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff028147a80 0x7ff028147e90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff028156c60 0x7ff028145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 --2- 172.21.15.37:0/3265185580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff02806e370 0x7ff0280c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 >> 172.21.15.37:0/3265185580 conn(0x7ff02806f920 msgr2=0x7ff0280d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 shutdown_connections 2024-09-21T06:32:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.334+0000 7ff027e00640 1 -- 172.21.15.37:0/3265185580 wait complete. 2024-09-21T06:32:40.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:32:40.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 -- 172.21.15.37:0/3236086315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c406e370 msgr2=0x7fd3c40b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3236086315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c406e370 0x7fd3c40b4f90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005fd0 tx=0x7fd3b802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 -- 172.21.15.37:0/3236086315 shutdown_connections 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3236086315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c4156c60 0x7fd3c40cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3236086315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c406e370 0x7fd3c40b4f90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 -- 172.21.15.37:0/3236086315 >> 172.21.15.37:0/3236086315 conn(0x7fd3c406f920 msgr2=0x7fd3c40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 -- 172.21.15.37:0/3236086315 shutdown_connections 2024-09-21T06:32:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.589+0000 7fd3cb200640 1 -- 172.21.15.37:0/3236086315 wait complete. 2024-09-21T06:32:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.592+0000 7fd3cb200640 1 Processor -- start 2024-09-21T06:32:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.592+0000 7fd3cb200640 1 -- start start 2024-09-21T06:32:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 0x7fd3c4152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c40cc8b0 0x7fd3c4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 0x7fd3c4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c406d510 con 0x7fd3c4156c60 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c406d390 con 0x7fd3c40cc8b0 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.593+0000 7fd3cb200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c406d690 con 0x7fd3c406e370 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 0x7fd3c4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 0x7fd3c4152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 0x7fd3c4152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:52684/0 (socket says 172.21.15.37:52684) 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3c9e00640 1 -- 172.21.15.37:0/3319152950 learned_addr learned my addr 172.21.15.37:0/3319152950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3c9400640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c40cc8b0 0x7fd3c4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 -- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 msgr2=0x7fd3c4152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 0x7fd3c4152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 -- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c40cc8b0 msgr2=0x7fd3c4157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c40cc8b0 0x7fd3c4157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 -- 172.21.15.37:0/3319152950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c41583f0 con 0x7fd3c4156c60 2024-09-21T06:32:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.594+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 0x7fd3c4157e40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c760 tx=0x7fd3c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.595+0000 7fd3be000640 1 -- 172.21.15.37:0/3319152950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0003b70 con 0x7fd3c4156c60 2024-09-21T06:32:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.595+0000 7fd3be000640 1 -- 172.21.15.37:0/3319152950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3c0003d10 con 0x7fd3c4156c60 2024-09-21T06:32:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.595+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c41f54c0 con 0x7fd3c4156c60 2024-09-21T06:32:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.595+0000 7fd3be000640 1 -- 172.21.15.37:0/3319152950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0005330 con 0x7fd3c4156c60 2024-09-21T06:32:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.595+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c41f5a40 con 0x7fd3c4156c60 2024-09-21T06:32:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.597+0000 7fd3be000640 1 -- 172.21.15.37:0/3319152950 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd3c00054d0 con 0x7fd3c4156c60 2024-09-21T06:32:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.598+0000 7fd3be000640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 0x7fd3a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.598+0000 7fd3be000640 1 -- 172.21.15.37:0/3319152950 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fd3c009ab50 con 0x7fd3c4156c60 2024-09-21T06:32:40.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.599+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 0x7fd3a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.599+0000 7fd3d3816740 1 Processor -- start 2024-09-21T06:32:40.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.599+0000 7fd3d3816740 1 -- start start 2024-09-21T06:32:40.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.599+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 0x7fd3a807b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009230 tx=0x7fd3b8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45154c390 0x55f4519384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45190d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f4518666f0 con 0x55f4515234d0 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f451855b20 con 0x55f45152b6b0 2024-09-21T06:32:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.600+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f451859930 con 0x55f45154c390 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55552/0 (socket says 172.21.15.37:55552) 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 -- 172.21.15.37:0/2130941477 learned_addr learned my addr 172.21.15.37:0/2130941477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45154c390 0x55f4519384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3ca800640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45190d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 -- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45154c390 msgr2=0x55f4519384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45154c390 0x55f4519384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 -- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 msgr2=0x55f45190d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45190d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 -- 172.21.15.37:0/2130941477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4515b7970 con 0x55f4515234d0 2024-09-21T06:32:40.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3ca800640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45190d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:40.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.601+0000 7fd3c9400640 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400bbf0 tx=0x7fd3b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b070ce2ebbb619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.602+0000 7fd3b2600640 1 -- 172.21.15.37:0/2130941477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4004020 con 0x55f4515234d0 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.602+0000 7fd3b2600640 1 -- 172.21.15.37:0/2130941477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b40042b0 con 0x55f4515234d0 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.602+0000 7fd3b2600640 1 -- 172.21.15.37:0/2130941477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40044a0 con 0x55f4515234d0 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.602+0000 7fd3d3816740 1 -- 172.21.15.37:0/2130941477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 msgr2=0x55f45190d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.602+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400bbf0 tx=0x7fd3b40107a0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 -- 172.21.15.37:0/2130941477 shutdown_connections 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f4515234d0 0x55f45190d310 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45190d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2130941477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45154c390 0x55f4519384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 -- 172.21.15.37:0/2130941477 >> 172.21.15.37:0/2130941477 conn(0x55f4515404e0 msgr2=0x55f4518f37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 -- 172.21.15.37:0/2130941477 shutdown_connections 2024-09-21T06:32:40.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.603+0000 7fd3d3816740 1 -- 172.21.15.37:0/2130941477 wait complete. 2024-09-21T06:32:40.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.604+0000 7fd3d3816740 1 -- start start 2024-09-21T06:32:40.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.604+0000 7fd3d3816740 1 Processor -- start 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f45190a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45193ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f4515c6740 con 0x55f45154c390 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4517fc290 con 0x55f45152b6b0 2024-09-21T06:32:40.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f45190bcc0 con 0x55f4515234d0 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3d3816740 10 client.0 _ll_register_callbacks cb 0x55f4517d91e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55556/0 (socket says 172.21.15.37:55556) 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 -- 172.21.15.37:0/2090071014 learned_addr learned my addr 172.21.15.37:0/2090071014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f45190a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3c9400640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45193ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 -- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 msgr2=0x55f45190a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f45190a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 -- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 msgr2=0x55f45193ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45193ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.606+0000 7fd3ca800640 1 -- 172.21.15.37:0/2090071014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f45162e7d0 con 0x55f45154c390 2024-09-21T06:32:40.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3ca800640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c730 tx=0x7fd3c0062210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac523f28e61afacd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0088020 con 0x55f45154c390 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fd3c0061b60 con 0x55f45154c390 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b40042b0 con 0x55f45154c390 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fd3c0083050 con 0x55f45154c390 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3d3816740 10 client.6648 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x55f45194ba60 con 0x55f45154c390 2024-09-21T06:32:40.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b40042b0 con 0x55f45154c390 2024-09-21T06:32:40.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.607+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7fd3c0069340 con 0x55f45154c390 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 10 client.6648 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 10 client.6648 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 20 client.6648 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f45150af60 0x55f451933a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 4 client.6648 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f451868640 con 0x55f45150af60 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f45150af60 0x55f451933a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f45150af60 0x55f451933a50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fade51c0481cf69c server_cookie=2abb98b8d9617f9c in_seq=0 out_seq=0 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.608+0000 7fd3a6a00640 10 client.6648 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3a6a00640 1 -- 172.21.15.37:0/2090071014 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f451868640 con 0x55f45150af60 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3a6a00640 10 client.6648 handle_command_reply: tid=0 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3d3816740 1 client.6648 shutdown 2024-09-21T06:32:40.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 msgr2=0x55f45193b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c730 tx=0x7fd3c0062210 comp rx=0 tx=0).stop 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.609+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 shutdown_connections 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f45150af60 0x55f451933a50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45193b220 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45193ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 --2- 172.21.15.37:0/2090071014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f45190a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3a6a00640 0 client.6648 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 >> 172.21.15.37:0/2090071014 conn(0x55f4515404e0 msgr2=0x55f4518ee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 shutdown_connections 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 1 -- 172.21.15.37:0/2090071014 wait complete. 2024-09-21T06:32:40.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.610+0000 7fd3d3816740 20 client.6648 trim_cache size 0 max 16384 2024-09-21T06:32:40.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.624+0000 7fd3d3816740 1 Processor -- start 2024-09-21T06:32:40.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.624+0000 7fd3d3816740 1 -- start start 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 0x55f451931ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45152b6b0 0x55f45191c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4202ac0 con 0x55f45154c390 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4201cb0 con 0x55f4515234d0 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c4202390 con 0x55f45152b6b0 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55566/0 (socket says 172.21.15.37:55566) 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 -- 172.21.15.37:0/3505361737 learned_addr learned my addr 172.21.15.37:0/3505361737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 0x55f451931ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3c9400640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45152b6b0 0x55f45191c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 -- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45152b6b0 msgr2=0x55f45191c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45152b6b0 0x55f45191c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 -- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 msgr2=0x55f451931ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 0x55f451931ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3ca800640 1 -- 172.21.15.37:0/3505361737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4518f0310 con 0x55f45154c390 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.626+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 0x55f451931ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.627+0000 7fd3ca800640 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00628c0 tx=0x7fd3c0067200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740fdeaeab8b693c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.627+0000 7fd3b1c00640 1 -- 172.21.15.37:0/3505361737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0088040 con 0x55f45154c390 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.627+0000 7fd3b1c00640 1 -- 172.21.15.37:0/3505361737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3c0061960 con 0x55f45154c390 2024-09-21T06:32:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.627+0000 7fd3b1c00640 1 -- 172.21.15.37:0/3505361737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c006c030 con 0x55f45154c390 2024-09-21T06:32:40.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 -- 172.21.15.37:0/3505361737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 msgr2=0x55f45191c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00628c0 tx=0x7fd3c0067200 comp rx=0 tx=0).stop 2024-09-21T06:32:40.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 -- 172.21.15.37:0/3505361737 shutdown_connections 2024-09-21T06:32:40.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45191c7d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f45152b6b0 0x55f45191c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 --2- 172.21.15.37:0/3505361737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4515234d0 0x55f451931ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 -- 172.21.15.37:0/3505361737 >> 172.21.15.37:0/3505361737 conn(0x55f4515404e0 msgr2=0x55f4518f3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 -- 172.21.15.37:0/3505361737 shutdown_connections 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.628+0000 7fd3d3816740 1 -- 172.21.15.37:0/3505361737 wait complete. 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.629+0000 7fd3d3816740 1 -- start start 2024-09-21T06:32:40.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.629+0000 7fd3d3816740 1 Processor -- start 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f4519ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45195b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c41f9460 con 0x55f45154c390 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f45193ae20 con 0x55f45152b6b0 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f45191a290 con 0x55f4515234d0 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.630+0000 7fd3d3816740 10 client.0 _ll_register_callbacks cb 0x7fd3c0001790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f4519ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45195b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55574/0 (socket says 172.21.15.37:55574) 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 -- 172.21.15.37:0/396888656 learned_addr learned my addr 172.21.15.37:0/396888656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 -- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 msgr2=0x55f4519ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f4519ba1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 -- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 msgr2=0x55f45195b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45195b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 -- 172.21.15.37:0/396888656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f451938a30 con 0x55f45154c390 2024-09-21T06:32:40.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3ca800640 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c730 tx=0x7fd3c006b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd98641709482770 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.631+0000 7fd3a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0088040 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fd3c007cd20 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c0061960 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fd3c008f050 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 10 client.6654 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x55f451938a30 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3c0061960 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7fd3c0092020 con 0x55f45154c390 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 10 client.6654 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 10 client.6654 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 20 client.6654 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f451949970 0x55f4519baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 4 client.6654 mds_command: new command op to 6106 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.632+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 --> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4518640f0 con 0x55f451949970 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f451949970 0x55f4519baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3c9e00640 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f451949970 0x55f4519baf30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e48fc727a607f3 server_cookie=10e1721dbe90a9a4 in_seq=0 out_seq=0 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3a6a00640 10 client.6654 ms_handle_connect on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3a6a00640 1 -- 172.21.15.37:0/396888656 <== mds.0 v2:172.21.15.37:6833/1569973003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+271 (crc 0 0 0) 0x55f4518640f0 con 0x55f451949970 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3a6a00640 10 client.6654 handle_command_reply: tid=0 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3d3816740 1 client.6654 shutdown 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 msgr2=0x55f45195bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.633+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c730 tx=0x7fd3c006b2d0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 shutdown_connections 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:6833/1569973003,v1:172.21.15.37:6835/1569973003] conn(0x55f451949970 0x55f4519baf30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f45154c390 0x55f45195bc80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f45152b6b0 0x55f45195b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 --2- 172.21.15.37:0/396888656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4515234d0 0x55f4519ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3a6a00640 0 client.6654 ms_handle_reset on v2:172.21.15.37:6833/1569973003 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 >> 172.21.15.37:0/396888656 conn(0x55f4515404e0 msgr2=0x55f4518f3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 shutdown_connections 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 1 -- 172.21.15.37:0/396888656 wait complete. 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3d3816740 20 client.6654 trim_cache size 0 max 16384 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 668462740, 2024-09-21T06:32:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627778, 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3" 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 3423535304, 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627779, 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3/d4" 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:40.640 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 msgr2=0x7fd3a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 0x7fd3a807b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009230 tx=0x7fd3b8004730 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 msgr2=0x7fd3c4157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.634+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 0x7fd3c4157e40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c760 tx=0x7fd3c000cc20 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 shutdown_connections 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd3a8078bc0 0x7fd3a807b0d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3c4156c60 0x7fd3c4157e40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c40cc8b0 0x7fd3c4157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 --2- 172.21.15.37:0/3319152950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3c406e370 0x7fd3c4152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 >> 172.21.15.37:0/3319152950 conn(0x7fd3c406f920 msgr2=0x7fd3c4160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 shutdown_connections 2024-09-21T06:32:40.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.635+0000 7fd3cb200640 1 -- 172.21.15.37:0/3319152950 wait complete. 2024-09-21T06:32:40.662 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2024-09-21T06:32:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 -- 172.21.15.37:0/3295765199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00ccfe0 msgr2=0x7f81b01556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 --2- 172.21.15.37:0/3295765199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00ccfe0 0x7f81b01556f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8198009830 tx=0x7f819802a540 comp rx=0 tx=0).stop 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 -- 172.21.15.37:0/3295765199 shutdown_connections 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 --2- 172.21.15.37:0/3295765199 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b015f670 0x7f81b0162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 --2- 172.21.15.37:0/3295765199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00ccfe0 0x7f81b01556f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 -- 172.21.15.37:0/3295765199 >> 172.21.15.37:0/3295765199 conn(0x7f81b006f920 msgr2=0x7f81b00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 -- 172.21.15.37:0/3295765199 shutdown_connections 2024-09-21T06:32:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.888+0000 7f81afe00640 1 -- 172.21.15.37:0/3295765199 wait complete. 2024-09-21T06:32:40.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.890+0000 7f81afe00640 1 Processor -- start 2024-09-21T06:32:40.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.890+0000 7f81afe00640 1 -- start start 2024-09-21T06:32:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b015f670 0x7f81b0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b00b0f60 0x7f81b00b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b006e290 con 0x7f81b00cf850 2024-09-21T06:32:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f81b006e110 con 0x7f81b00b0f60 2024-09-21T06:32:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81afe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f81b006e410 con 0x7f81b015f670 2024-09-21T06:32:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81aea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81aea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55596/0 (socket says 172.21.15.37:55596) 2024-09-21T06:32:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.892+0000 7f81aea00640 1 -- 172.21.15.37:0/3346557250 learned_addr learned my addr 172.21.15.37:0/3346557250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 -- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b015f670 msgr2=0x7f81b0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b015f670 0x7f81b0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 -- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b00b0f60 msgr2=0x7f81b00b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b00b0f60 0x7f81b00b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 -- 172.21.15.37:0/3346557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00b18b0 con 0x7f81b00cf850 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81aea00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c3c0 tx=0x7f81a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81a7400640 1 -- 172.21.15.37:0/3346557250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0017070 con 0x7f81b00cf850 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81a7400640 1 -- 172.21.15.37:0/3346557250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a00035f0 con 0x7f81b00cf850 2024-09-21T06:32:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b0159760 con 0x7f81b00cf850 2024-09-21T06:32:40.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81a7400640 1 -- 172.21.15.37:0/3346557250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003a20 con 0x7f81b00cf850 2024-09-21T06:32:40.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.893+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b0159f50 con 0x7f81b00cf850 2024-09-21T06:32:40.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.896+0000 7f81a7400640 1 -- 172.21.15.37:0/3346557250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81a001a070 con 0x7f81b00cf850 2024-09-21T06:32:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.897+0000 7f81a7400640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 0x7f818c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.897+0000 7f81a7400640 1 -- 172.21.15.37:0/3346557250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f81a009a2a0 con 0x7f81b00cf850 2024-09-21T06:32:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.897+0000 7f81a6000640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 0x7f818c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.897+0000 7f81bca0a740 1 Processor -- start 2024-09-21T06:32:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.897+0000 7f81bca0a740 1 -- start start 2024-09-21T06:32:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.898+0000 7f81a6000640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 0x7f818c07b0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8198034000 tx=0x7f81980046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7427c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e741d56f0 con 0x560e73e9a6b0 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e741c4b20 con 0x560e73ebb390 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560e741c8930 con 0x560e73e924d0 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55606/0 (socket says 172.21.15.37:55606) 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 -- 172.21.15.37:0/3631100548 learned_addr learned my addr 172.21.15.37:0/3631100548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81a6000640 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7427c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81aea00640 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 -- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 msgr2=0x560e7427c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7427c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 -- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 msgr2=0x560e742a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.899+0000 7f81af400640 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.900+0000 7f81af400640 1 -- 172.21.15.37:0/3631100548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e73f26970 con 0x560e73e9a6b0 2024-09-21T06:32:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.900+0000 7f81af400640 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009d20 tx=0x7f81a800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdbe1abb20bb4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.900+0000 7f819f400640 1 -- 172.21.15.37:0/3631100548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018070 con 0x560e73e9a6b0 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.900+0000 7f819f400640 1 -- 172.21.15.37:0/3631100548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a8011520 con 0x560e73e9a6b0 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.900+0000 7f819f400640 1 -- 172.21.15.37:0/3631100548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011990 con 0x560e73e9a6b0 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 -- 172.21.15.37:0/3631100548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 msgr2=0x560e7427c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009d20 tx=0x7f81a800b810 comp rx=0 tx=0).stop 2024-09-21T06:32:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 -- 172.21.15.37:0/3631100548 shutdown_connections 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e7427c850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7427c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3631100548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 -- 172.21.15.37:0/3631100548 >> 172.21.15.37:0/3631100548 conn(0x560e73eaf4e0 msgr2=0x560e742627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 -- 172.21.15.37:0/3631100548 shutdown_connections 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.901+0000 7f81bca0a740 1 -- 172.21.15.37:0/3631100548 wait complete. 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.902+0000 7f81bca0a740 1 -- start start 2024-09-21T06:32:40.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.902+0000 7f81bca0a740 1 Processor -- start 2024-09-21T06:32:40.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7428a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e9a6b0 0x560e7428ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e73f35740 con 0x560e73ebb390 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e7416b290 con 0x560e73e9a6b0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560e7428c170 con 0x560e73e924d0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81bca0a740 10 client.0 _ll_register_callbacks cb 0x560e741481e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55614/0 (socket says 172.21.15.37:55614) 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81af400640 1 -- 172.21.15.37:0/916627312 learned_addr learned my addr 172.21.15.37:0/916627312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81a6000640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e9a6b0 0x560e7428ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.904+0000 7f81aea00640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7428a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 -- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 msgr2=0x560e7428a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7428a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 -- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e9a6b0 msgr2=0x560e7428ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e9a6b0 0x560e7428ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 -- 172.21.15.37:0/916627312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e73f9d7d0 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81af400640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f81a8016040 tx=0x7f81a8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba85bf4c5ba68d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f8193e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018040 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f81a8011a70 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.905+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a8011520 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f81a802a050 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 10 client.6669 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x560e742a4420 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a8011520 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7f81a802d020 con 0x560e73ebb390 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 10 client.6669 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 10 client.6669 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.d 2024-09-21T06:32:40.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 20 client.6669 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e73e79f60 0x560e742a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 4 client.6669 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:40.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.906+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e741d7640 con 0x560e73e79f60 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.907+0000 7f81aea00640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e73e79f60 0x560e742a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.907+0000 7f81aea00640 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e73e79f60 0x560e742a0f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b70a7875c4adfd0e server_cookie=c394017ec58f832b in_seq=0 out_seq=0 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.907+0000 7f8193e00640 10 client.6669 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f8193e00640 1 -- 172.21.15.37:0/916627312 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e741d7640 con 0x560e73e79f60 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f8193e00640 10 client.6669 handle_command_reply: tid=0 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f81bca0a740 1 client.6669 shutdown 2024-09-21T06:32:40.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 msgr2=0x560e7428b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f81a8016040 tx=0x7f81a8011900 comp rx=0 tx=0).stop 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 shutdown_connections 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.908+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e73e79f60 0x560e742a0f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428b370 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e9a6b0 0x560e7428ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f8193e00640 0 client.6669 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 --2- 172.21.15.37:0/916627312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e7428a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 >> 172.21.15.37:0/916627312 conn(0x560e73eaf4e0 msgr2=0x560e7425d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 shutdown_connections 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 1 -- 172.21.15.37:0/916627312 wait complete. 2024-09-21T06:32:40.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.909+0000 7f81bca0a740 20 client.6669 trim_cache size 0 max 16384 2024-09-21T06:32:40.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.923+0000 7f81bca0a740 1 Processor -- start 2024-09-21T06:32:40.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.923+0000 7f81bca0a740 1 -- start start 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.924+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 0x560e7428ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.924+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 0x560e7428b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.924+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81bca0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0202b80 con 0x560e73ebb390 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0201d20 con 0x560e73e924d0 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f81b0202450 con 0x560e73e9a6b0 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55624/0 (socket says 172.21.15.37:55624) 2024-09-21T06:32:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 -- 172.21.15.37:0/3947188209 learned_addr learned my addr 172.21.15.37:0/3947188209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81aea00640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 0x560e7428ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81a6000640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 0x560e7428b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 -- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 msgr2=0x560e7428b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 0x560e7428b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 -- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 msgr2=0x560e7428ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 0x560e7428ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 -- 172.21.15.37:0/3947188209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e74279740 con 0x560e73ebb390 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81a6000640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 0x560e7428b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81aea00640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 0x560e7428ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.925+0000 7f81af400640 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f81a800be40 tx=0x7f81a800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434faf763bd365c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f819ea00640 1 -- 172.21.15.37:0/3947188209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018050 con 0x560e73ebb390 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f819ea00640 1 -- 172.21.15.37:0/3947188209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a8011520 con 0x560e73ebb390 2024-09-21T06:32:40.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f819ea00640 1 -- 172.21.15.37:0/3947188209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a801e450 con 0x560e73ebb390 2024-09-21T06:32:40.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 -- 172.21.15.37:0/3947188209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 msgr2=0x560e7428c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f81a800be40 tx=0x7f81a800be70 comp rx=0 tx=0).stop 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 -- 172.21.15.37:0/3947188209 shutdown_connections 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73ebb390 0x560e7428c300 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e9a6b0 0x560e7428b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 --2- 172.21.15.37:0/3947188209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73e924d0 0x560e7428ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.926+0000 7f81bca0a740 1 -- 172.21.15.37:0/3947188209 >> 172.21.15.37:0/3947188209 conn(0x560e73eaf4e0 msgr2=0x560e74262500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.927+0000 7f81bca0a740 1 -- 172.21.15.37:0/3947188209 shutdown_connections 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.927+0000 7f81bca0a740 1 -- 172.21.15.37:0/3947188209 wait complete. 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.928+0000 7f81bca0a740 1 -- start start 2024-09-21T06:32:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.928+0000 7f81bca0a740 1 Processor -- start 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e742b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0157aa0 con 0x560e73e9a6b0 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e742a4210 con 0x560e73ebb390 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560e742b5830 con 0x560e73e924d0 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.929+0000 7f81bca0a740 10 client.0 _ll_register_callbacks cb 0x7f81a8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81aea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e742b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55632/0 (socket says 172.21.15.37:55632) 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 -- 172.21.15.37:0/2869459057 learned_addr learned my addr 172.21.15.37:0/2869459057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81af400640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 -- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 msgr2=0x560e742b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e742b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 -- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 msgr2=0x560e742b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 -- 172.21.15.37:0/2869459057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e74326fd0 con 0x560e73e9a6b0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81aea00640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e742b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f81a6000640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f819802ab30 tx=0x7f8198005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599c1d0849fd2079 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.930+0000 7f8193e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198003ea0 con 0x560e73e9a6b0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f8198054d40 con 0x560e73e9a6b0 2024-09-21T06:32:40.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a8011520 con 0x560e73e9a6b0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8198049840 con 0x560e73e9a6b0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 10 client.6675 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x560e742b5610 con 0x560e73e9a6b0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a8011520 con 0x560e73e9a6b0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7f8198051070 con 0x560e73e9a6b0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 10 client.6675 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 10 client.6675 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.d 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 20 client.6675 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e74278d10 0x560e74279120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 4 client.6675 mds_command: new command op to 6100 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.931+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 --> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e741d30f0 con 0x560e74278d10 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.932+0000 7f81aea00640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e74278d10 0x560e74279120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.932+0000 7f81aea00640 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e74278d10 0x560e74279120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3db61b0306771a77 server_cookie=c37d260a0ca775e7 in_seq=0 out_seq=0 2024-09-21T06:32:40.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.932+0000 7f8193e00640 10 client.6675 ms_handle_connect on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f8193e00640 1 -- 172.21.15.37:0/2869459057 <== mds.1 v2:172.21.15.150:6835/2893186945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+283 (crc 0 0 0) 0x560e741d30f0 con 0x560e74278d10 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f8193e00640 10 client.6675 handle_command_reply: tid=0 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 client.6675 shutdown 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 msgr2=0x560e743266c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f819802ab30 tx=0x7f8198005f90 comp rx=0 tx=0).stop 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 shutdown_connections 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:6835/2893186945,v1:172.21.15.150:6837/2893186945] conn(0x560e74278d10 0x560e74279120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e73e9a6b0 0x560e743266c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e73ebb390 0x560e742b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 --2- 172.21.15.37:0/2869459057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e73e924d0 0x560e742b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 >> 172.21.15.37:0/2869459057 conn(0x560e73eaf4e0 msgr2=0x560e7427c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f8193e00640 0 client.6675 ms_handle_reset on v2:172.21.15.150:6835/2893186945 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 shutdown_connections 2024-09-21T06:32:40.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 1 -- 172.21.15.37:0/2869459057 wait complete. 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.933+0000 7f81bca0a740 20 client.6675 trim_cache size 0 max 16384 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 845553855, 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627781, 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:40.940 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 1741864753, 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627780, 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3/d4/d5" 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:32:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 msgr2=0x7f818c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 0x7f818c07b0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8198034000 tx=0x7f81980046c0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 msgr2=0x7f81b01479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c3c0 tx=0x7f81a000c880 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 shutdown_connections 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f818c078bc0 0x7f818c07b0d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b00b0f60 0x7f81b00b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b015f670 0x7f81b0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 --2- 172.21.15.37:0/3346557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b00cf850 0x7f81b01479d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 >> 172.21.15.37:0/3346557250 conn(0x7f81b006f920 msgr2=0x7f81b015ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 shutdown_connections 2024-09-21T06:32:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:40.934+0000 7f81afe00640 1 -- 172.21.15.37:0/3346557250 wait complete. 2024-09-21T06:32:40.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2024-09-21T06:32:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 -- 172.21.15.37:0/899845660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb80d0660 msgr2=0x7f9cb8152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 --2- 172.21.15.37:0/899845660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb80d0660 0x7f9cb8152c00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80744e0 tx=0x7f9cb002a900 comp rx=0 tx=0).stop 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 -- 172.21.15.37:0/899845660 shutdown_connections 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 --2- 172.21.15.37:0/899845660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb80d0660 0x7f9cb8152c00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 --2- 172.21.15.37:0/899845660 >> v2:172.21.15.150:3300/0 conn(0x7f9cb80d1840 0x7f9cb80d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 -- 172.21.15.37:0/899845660 >> 172.21.15.37:0/899845660 conn(0x7f9cb80b3b30 msgr2=0x7f9cb80b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 -- 172.21.15.37:0/899845660 shutdown_connections 2024-09-21T06:32:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.188+0000 7f9cbd800640 1 -- 172.21.15.37:0/899845660 wait complete. 2024-09-21T06:32:41.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.190+0000 7f9cbd800640 1 Processor -- start 2024-09-21T06:32:41.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.191+0000 7f9cbd800640 1 -- start start 2024-09-21T06:32:41.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cb80d0660 0x7f9cb81f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 0x7f9cb81f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81534d0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8153350 con 0x7f9cb80d1840 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbd800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb8153650 con 0x7f9cb80d0660 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55652/0 (socket says 172.21.15.37:55652) 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cb7e00640 1 -- 172.21.15.37:0/202930766 learned_addr learned my addr 172.21.15.37:0/202930766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cbce00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 0x7f9cb81f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.192+0000 7f9cafe00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cb80d0660 0x7f9cb81f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 -- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cb80d0660 msgr2=0x7f9cb81f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cb80d0660 0x7f9cb81f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 -- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 msgr2=0x7f9cb81f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 0x7f9cb81f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 -- 172.21.15.37:0/202930766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb80c7cd0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cbce00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 0x7f9cb81f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c3f0 tx=0x7f9ca800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb4c00640 1 -- 172.21.15.37:0/202930766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8017070 con 0x7f9cb8071cf0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb4c00640 1 -- 172.21.15.37:0/202930766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca80035f0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb80c7fb0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cb4c00640 1 -- 172.21.15.37:0/202930766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8003a40 con 0x7f9cb8071cf0 2024-09-21T06:32:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.193+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81f59d0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.196+0000 7f9cb4c00640 1 -- 172.21.15.37:0/202930766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ca801a070 con 0x7f9cb8071cf0 2024-09-21T06:32:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.197+0000 7f9cb4c00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 0x7f9c9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.197+0000 7f9cb4c00640 1 -- 172.21.15.37:0/202930766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f9ca809a2e0 con 0x7f9cb8071cf0 2024-09-21T06:32:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.198+0000 7f9cc6049740 1 Processor -- start 2024-09-21T06:32:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.198+0000 7f9cafe00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 0x7f9c9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.198+0000 7f9cc6049740 1 -- start start 2024-09-21T06:32:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.198+0000 7f9cafe00640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 0x7f9c9407b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005fd0 tx=0x7f9ca0005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628c94006f0 con 0x5628c90bd4d0 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5628c93efb20 con 0x5628c90c56b0 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5628c93f3930 con 0x5628c90e6390 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.199+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55668/0 (socket says 172.21.15.37:55668) 2024-09-21T06:32:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 -- 172.21.15.37:0/3795282564 learned_addr learned my addr 172.21.15.37:0/3795282564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cbce00640 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 -- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 msgr2=0x5628c94d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 -- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 msgr2=0x5628c94abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 -- 172.21.15.37:0/3795282564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c9151970 con 0x5628c90bd4d0 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cafe00640 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000fe00 tx=0x7f9ca0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7107d96db0b5b9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.200+0000 7f9cacc00640 1 -- 172.21.15.37:0/3795282564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0022070 con 0x5628c90bd4d0 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cacc00640 1 -- 172.21.15.37:0/3795282564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca001e450 con 0x5628c90bd4d0 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cacc00640 1 -- 172.21.15.37:0/3795282564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca001e860 con 0x5628c90bd4d0 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 -- 172.21.15.37:0/3795282564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 msgr2=0x5628c94ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000fe00 tx=0x7f9ca0012000 comp rx=0 tx=0).stop 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 -- 172.21.15.37:0/3795282564 shutdown_connections 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94ab720 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3795282564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 -- 172.21.15.37:0/3795282564 >> 172.21.15.37:0/3795282564 conn(0x5628c90da4e0 msgr2=0x5628c9491bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.201+0000 7f9cc6049740 1 -- 172.21.15.37:0/3795282564 shutdown_connections 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.202+0000 7f9cc6049740 1 -- 172.21.15.37:0/3795282564 wait complete. 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.202+0000 7f9cc6049740 1 -- start start 2024-09-21T06:32:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.203+0000 7f9cc6049740 1 Processor -- start 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628c9160740 con 0x5628c90bd4d0 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5628c9396290 con 0x5628c90c56b0 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5628c94d29a0 con 0x5628c90e6390 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cc6049740 10 client.0 _ll_register_callbacks cb 0x5628c93731e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.204+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55672/0 (socket says 172.21.15.37:55672) 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 -- 172.21.15.37:0/2062474893 learned_addr learned my addr 172.21.15.37:0/2062474893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cafe00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cbce00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 -- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 msgr2=0x5628c94d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 -- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 msgr2=0x5628c94bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 -- 172.21.15.37:0/2062474893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c91c87d0 con 0x5628c90bd4d0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8000fb0 tx=0x7f9ca80618e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2346cf5b6ebd309 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9ca5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca806b4e0 con 0x5628c90bd4d0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f9ca8069bb0 con 0x5628c90bd4d0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.205+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca001e450 con 0x5628c90bd4d0 2024-09-21T06:32:41.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9ca8084050 con 0x5628c90bd4d0 2024-09-21T06:32:41.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 10 client.6690 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:41.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x5628c94d26e0 con 0x5628c90bd4d0 2024-09-21T06:32:41.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca001e450 con 0x5628c90bd4d0 2024-09-21T06:32:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7f9ca807e060 con 0x5628c90bd4d0 2024-09-21T06:32:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 10 client.6690 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 10 client.6690 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-21T06:32:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 20 client.6690 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c90a4f60 0x5628c94e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 4 client.6690 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.206+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c9402640 con 0x5628c90a4f60 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.207+0000 7f9cafe00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c90a4f60 0x5628c94e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.207+0000 7f9cafe00640 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c90a4f60 0x5628c94e5dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d8f883b7e353a44 server_cookie=b69b096c9ea6075c in_seq=0 out_seq=0 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.207+0000 7f9ca5600640 10 client.6690 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9ca5600640 1 -- 172.21.15.37:0/2062474893 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628c9402640 con 0x5628c90a4f60 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9ca5600640 10 client.6690 handle_command_reply: tid=0 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 client.6690 shutdown 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 msgr2=0x5628c94a8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8000fb0 tx=0x7f9ca80618e0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 shutdown_connections 2024-09-21T06:32:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c90a4f60 0x5628c94e5dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90e6390 0x5628c94d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 --2- 172.21.15.37:0/2062474893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90bd4d0 0x5628c94a8320 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 >> 172.21.15.37:0/2062474893 conn(0x5628c90da4e0 msgr2=0x5628c948c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.208+0000 7f9ca5600640 0 client.6690 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.209+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 shutdown_connections 2024-09-21T06:32:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.209+0000 7f9cc6049740 1 -- 172.21.15.37:0/2062474893 wait complete. 2024-09-21T06:32:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.209+0000 7f9cc6049740 20 client.6690 trim_cache size 0 max 16384 2024-09-21T06:32:41.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.224+0000 7f9cc6049740 1 Processor -- start 2024-09-21T06:32:41.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.224+0000 7f9cc6049740 1 -- start start 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90bd4d0 0x5628c94aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81fe400 con 0x5628c90e6390 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb806fe60 con 0x5628c90c56b0 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb81fdcd0 con 0x5628c90bd4d0 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55684/0 (socket says 172.21.15.37:55684) 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cbce00640 1 -- 172.21.15.37:0/1669917039 learned_addr learned my addr 172.21.15.37:0/1669917039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cafe00640 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.225+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90bd4d0 0x5628c94aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 -- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90bd4d0 msgr2=0x5628c94aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90bd4d0 0x5628c94aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 -- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 msgr2=0x5628c94aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 -- 172.21.15.37:0/1669917039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c948c6c0 con 0x5628c90e6390 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9cbce00640 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009cd0 tx=0x7f9cb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b91d053b5c2a938 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.226+0000 7f9ca7e00640 1 -- 172.21.15.37:0/1669917039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004040 con 0x5628c90e6390 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9ca7e00640 1 -- 172.21.15.37:0/1669917039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cb004b070 con 0x5628c90e6390 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9ca7e00640 1 -- 172.21.15.37:0/1669917039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004480 con 0x5628c90e6390 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 -- 172.21.15.37:0/1669917039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 msgr2=0x5628c94f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009cd0 tx=0x7f9cb0004a40 comp rx=0 tx=0).stop 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 -- 172.21.15.37:0/1669917039 shutdown_connections 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c94f9430 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90c56b0 0x5628c94aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 --2- 172.21.15.37:0/1669917039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90bd4d0 0x5628c94aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 -- 172.21.15.37:0/1669917039 >> 172.21.15.37:0/1669917039 conn(0x5628c90da4e0 msgr2=0x5628c94e5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.227+0000 7f9cc6049740 1 -- 172.21.15.37:0/1669917039 shutdown_connections 2024-09-21T06:32:41.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.228+0000 7f9cc6049740 1 -- 172.21.15.37:0/1669917039 wait complete. 2024-09-21T06:32:41.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.228+0000 7f9cc6049740 1 -- start start 2024-09-21T06:32:41.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.229+0000 7f9cc6049740 1 Processor -- start 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90bd4d0 0x5628c94cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90c56b0 0x5628c94d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628c9492d40 con 0x5628c90e6390 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81f7570 con 0x5628c90bd4d0 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5628c948cbc0 con 0x5628c90c56b0 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cc6049740 10 client.0 _ll_register_callbacks cb 0x7f9ca80614c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.230+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55698/0 (socket says 172.21.15.37:55698) 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 -- 172.21.15.37:0/3729143249 learned_addr learned my addr 172.21.15.37:0/3729143249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cafe00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90c56b0 0x5628c94d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90bd4d0 0x5628c94cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 -- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90c56b0 msgr2=0x5628c94d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90c56b0 0x5628c94d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 -- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90bd4d0 msgr2=0x5628c94cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:32:41.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90bd4d0 0x5628c94cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 -- 172.21.15.37:0/3729143249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c94d1930 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9cbce00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x5628c94d2670 tx=0x7f9cb00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e5a772a3135e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.231+0000 7f9ca5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003c420 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f9cb003c5c0 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb004b070 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9cb005f050 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 10 client.6696 fetch_fsmap learned FSMap version 63 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x5628c9556040 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cb004b070 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) ==== 3558+0+0 (secure 0 0 0) 0x7f9cb0062020 con 0x5628c90e6390 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 10 client.6696 fetch_fsmap finished waiting for FSMap version 63 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 10 client.6696 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-21T06:32:41.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 20 client.6696 populate_metadata read hostname 'smithi037' 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c94e60f0 0x5628c9556810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 4 client.6696 mds_command: new command op to 6097 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.232+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 --> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628c93fe0f0 con 0x5628c94e60f0 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.233+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c94e60f0 0x5628c9556810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.233+0000 7f9cb7e00640 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c94e60f0 0x5628c9556810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be386febdda438 server_cookie=2d2865ae39b61fea in_seq=0 out_seq=0 2024-09-21T06:32:41.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.233+0000 7f9ca5600640 10 client.6696 ms_handle_connect on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9ca5600640 1 -- 172.21.15.37:0/3729143249 <== mds.2 v2:172.21.15.150:6834/4023630261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+295 (crc 0 0 0) 0x5628c93fe0f0 con 0x5628c94e60f0 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9ca5600640 10 client.6696 handle_command_reply: tid=0 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 client.6696 shutdown 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 msgr2=0x5628c95595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x5628c94d2670 tx=0x7f9cb00319f0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 shutdown_connections 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:6834/4023630261,v1:172.21.15.150:6836/4023630261] conn(0x5628c94e60f0 0x5628c9556810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628c90e6390 0x5628c95595d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628c90c56b0 0x5628c94d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 --2- 172.21.15.37:0/3729143249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628c90bd4d0 0x5628c94cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9ca5600640 0 client.6696 ms_handle_reset on v2:172.21.15.150:6834/4023630261 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 >> 172.21.15.37:0/3729143249 conn(0x5628c90da4e0 msgr2=0x5628c9483410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 shutdown_connections 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 1 -- 172.21.15.37:0/3729143249 wait complete. 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.234+0000 7f9cc6049740 20 client.6696 trim_cache size 0 max 16384 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:41.239 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 647237833, 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627783, 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 1714424678, 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627782, 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:41.240 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 msgr2=0x7f9c9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 0x7f9c9407b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005fd0 tx=0x7f9ca0005850 comp rx=0 tx=0).stop 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 msgr2=0x7f9cb80c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c3f0 tx=0x7f9ca800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 shutdown_connections 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c94078b90 0x7f9c9407b0a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb80d1840 0x7f9cb81f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cb80d0660 0x7f9cb81f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 --2- 172.21.15.37:0/202930766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8071cf0 0x7f9cb80c6fd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 >> 172.21.15.37:0/202930766 conn(0x7f9cb80b3b30 msgr2=0x7f9cb80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 shutdown_connections 2024-09-21T06:32:41.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.235+0000 7f9cbd800640 1 -- 172.21.15.37:0/202930766 wait complete. 2024-09-21T06:32:41.262 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:32:41.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:32:41.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:32:41.458 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:32:41.458 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:32:41.458 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:32:41.458 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:41.495 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:32:41.495 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:41.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 -- 172.21.15.37:0/4255214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee0006e500 msgr2=0x7fee000ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 --2- 172.21.15.37:0/4255214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee0006e500 0x7fee000ce510 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fedf8005fd0 tx=0x7fedf802aa90 comp rx=0 tx=0).stop 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 -- 172.21.15.37:0/4255214941 shutdown_connections 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 --2- 172.21.15.37:0/4255214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee0006e500 0x7fee000ce510 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 --2- 172.21.15.37:0/4255214941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 0x7fee0015ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 -- 172.21.15.37:0/4255214941 >> 172.21.15.37:0/4255214941 conn(0x7fee000b1b90 msgr2=0x7fee000b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.835+0000 7fee06200640 1 -- 172.21.15.37:0/4255214941 shutdown_connections 2024-09-21T06:32:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.836+0000 7fee06200640 1 -- 172.21.15.37:0/4255214941 wait complete. 2024-09-21T06:32:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.838+0000 7fee06200640 1 Processor -- start 2024-09-21T06:32:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.838+0000 7fee06200640 1 -- start start 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 0x7fee000a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee000c3bd0 0x7fee000c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee000cede0 con 0x7fee000af5b0 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee000cec60 con 0x7fee000c3bd0 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fee06200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fee000cef60 con 0x7fee000cc530 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.839+0000 7fedff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55714/0 (socket says 172.21.15.37:55714) 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 -- 172.21.15.37:0/1032946116 learned_addr learned my addr 172.21.15.37:0/1032946116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedffe00640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee000c3bd0 0x7fee000c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedfea00640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 0x7fee000a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 -- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 msgr2=0x7fee000a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 0x7fee000a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 -- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee000c3bd0 msgr2=0x7fee000c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee000c3bd0 0x7fee000c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 -- 172.21.15.37:0/1032946116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000c4520 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.840+0000 7fedff400640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fedf000d750 tx=0x7fedf000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.841+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0017070 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.841+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee000ca950 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.841+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedf0004310 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.841+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf000de60 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.841+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee001f5b90 con 0x7fee000af5b0 2024-09-21T06:32:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.842+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee000cf410 con 0x7fee000af5b0 2024-09-21T06:32:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.843+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fedf0009c30 con 0x7fee000af5b0 2024-09-21T06:32:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.844+0000 7fedf7400640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 0x7fede007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.844+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fedf009b2f0 con 0x7fee000af5b0 2024-09-21T06:32:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.844+0000 7fedfea00640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 0x7fede007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.845+0000 7fedfea00640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 0x7fede007b0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee000afc00 tx=0x7fedec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:41.849+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedf0020020 con 0x7fee000af5b0 2024-09-21T06:32:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.060+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee0015c4f0 con 0x7fee000af5b0 2024-09-21T06:32:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.061+0000 7fedf7400640 1 -- 172.21.15.37:0/1032946116 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7fee000c4520 con 0x7fee000af5b0 2024-09-21T06:32:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:32:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 msgr2=0x7fede007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 0x7fede007b0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee000afc00 tx=0x7fedec009040 comp rx=0 tx=0).stop 2024-09-21T06:32:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 msgr2=0x7fee000ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fedf000d750 tx=0x7fedf000dc10 comp rx=0 tx=0).stop 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 shutdown_connections 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fede0078bc0 0x7fede007b0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee000c3bd0 0x7fee000c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee000cc530 0x7fee000a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 --2- 172.21.15.37:0/1032946116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee000af5b0 0x7fee000ca410 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 >> 172.21.15.37:0/1032946116 conn(0x7fee000b1b90 msgr2=0x7fee000b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 shutdown_connections 2024-09-21T06:32:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.063+0000 7fee06200640 1 -- 172.21.15.37:0/1032946116 wait complete. 2024-09-21T06:32:42.088 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:32:42.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:42.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:32:42.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:32:42.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:32:42.154 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:32:42.190 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:32:42.190 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:32:42.190 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:42.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:32:42.231 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:32:42.231 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:32:42.231 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:42.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:32:42.267 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:32:42.267 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:32:42.267 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:42.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:32:42.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.537+0000 7f5fb0e00640 1 -- 172.21.15.37:0/2273580708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f90005300 msgr2=0x7f5f9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.537+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/2273580708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f90005300 0x7f5f9000b9c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4005fd0 tx=0x7f5f90005830 comp rx=0 tx=0).stop 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.538+0000 7f5fb0e00640 1 -- 172.21.15.37:0/2273580708 shutdown_connections 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.538+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/2273580708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f90005300 0x7f5f9000b9c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.538+0000 7f5fb0e00640 1 -- 172.21.15.37:0/2273580708 >> 172.21.15.37:0/2273580708 conn(0x7f5fac0d4360 msgr2=0x7f5fac0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.538+0000 7f5fb0e00640 1 -- 172.21.15.37:0/2273580708 shutdown_connections 2024-09-21T06:32:42.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.538+0000 7f5fb0e00640 1 -- 172.21.15.37:0/2273580708 wait complete. 2024-09-21T06:32:42.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.540+0000 7f5fb0e00640 1 Processor -- start 2024-09-21T06:32:42.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.540+0000 7f5fb0e00640 1 -- start start 2024-09-21T06:32:42.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f90005300 0x7f5fac15f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:42.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:42.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fac1f5650 0x7f5fac1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:42.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0cf4b0 con 0x7f5fac0cc590 2024-09-21T06:32:42.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15eac0 con 0x7f5f90005300 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fb0e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac0cf630 con 0x7f5fac1f5650 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5faaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5faaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55726/0 (socket says 172.21.15.37:55726) 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5faaa00640 1 -- 172.21.15.37:0/248065517 learned_addr learned my addr 172.21.15.37:0/248065517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fab400640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f90005300 0x7f5fac15f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.542+0000 7f5fabe00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fac1f5650 0x7f5fac1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 -- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fac1f5650 msgr2=0x7f5fac1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fac1f5650 0x7f5fac1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 -- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f90005300 msgr2=0x7f5fac15f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f90005300 0x7f5fac15f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 -- 172.21.15.37:0/248065517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1f5fa0 con 0x7f5fac0cc590 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5faaa00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006340 tx=0x7f5f980039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98015070 con 0x7f5fac0cc590 2024-09-21T06:32:42.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f98011880 con 0x7f5fac0cc590 2024-09-21T06:32:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98011cd0 con 0x7f5fac0cc590 2024-09-21T06:32:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1f8c10 con 0x7f5fac0cc590 2024-09-21T06:32:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.543+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1f91d0 con 0x7f5fac0cc590 2024-09-21T06:32:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.544+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac0cfb90 con 0x7f5fac0cc590 2024-09-21T06:32:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.546+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f98018070 con 0x7f5fac0cc590 2024-09-21T06:32:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.547+0000 7f5fa3400640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 0x7f5f8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.547+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f5f9809bb30 con 0x7f5fac0cc590 2024-09-21T06:32:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.548+0000 7f5fab400640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 0x7f5f8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.549+0000 7f5fab400640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 0x7f5f8c07b0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006570 tx=0x7f5f9c006cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.552+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f980636f0 con 0x7f5fac0cc590 2024-09-21T06:32:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.788+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fac0b9060 con 0x7f5fac0cc590 2024-09-21T06:32:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.789+0000 7f5fa3400640 1 -- 172.21.15.37:0/248065517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5277 (secure 0 0 0) 0x7f5f9809b030 con 0x7f5fac0cc590 2024-09-21T06:32:42.791 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:42.791 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":63,"btime":"2024-09-21T06:32:16:626541+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:32:16.554630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:32:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 63 2024-09-21T06:32:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 msgr2=0x7f5f8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 0x7f5f8c07b0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006570 tx=0x7f5f9c006cc0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 msgr2=0x7f5fac154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006340 tx=0x7f5f980039c0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 shutdown_connections 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5f8c078bc0 0x7f5f8c07b0d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fac0cc590 0x7f5fac154890 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fac1f5650 0x7f5fac1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 --2- 172.21.15.37:0/248065517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f90005300 0x7f5fac15f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 >> 172.21.15.37:0/248065517 conn(0x7f5fac0d4360 msgr2=0x7f5fac0b9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 shutdown_connections 2024-09-21T06:32:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:42.791+0000 7f5fb0e00640 1 -- 172.21.15.37:0/248065517 wait complete. 2024-09-21T06:32:42.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 -- 172.21.15.37:0/2326670444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06840051a0 msgr2=0x7f068400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 --2- 172.21.15.37:0/2326670444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06840051a0 0x7f068400b9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0698005fd0 tx=0x7f0684005830 comp rx=0 tx=0).stop 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f069f200640 1 -- 172.21.15.37:0/2326670444 reap_dead start 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 -- 172.21.15.37:0/2326670444 shutdown_connections 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 -- 172.21.15.37:0/2326670444 >> 172.21.15.37:0/2326670444 conn(0x7f06a006f940 msgr2=0x7f06a00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 -- 172.21.15.37:0/2326670444 shutdown_connections 2024-09-21T06:32:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.042+0000 7f06a4a00640 1 -- 172.21.15.37:0/2326670444 wait complete. 2024-09-21T06:32:43.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.045+0000 7f06a4a00640 1 Processor -- start 2024-09-21T06:32:43.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.045+0000 7f06a4a00640 1 -- start start 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 0x7f06a0157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 0x7f06a015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06a006e990 con 0x7f06a013f400 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06a006e810 con 0x7f06840051a0 2024-09-21T06:32:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f06a4a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f06a0165250 con 0x7f06a00d0940 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.046+0000 7f0697e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55740/0 (socket says 172.21.15.37:55740) 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 -- 172.21.15.37:0/772415191 learned_addr learned my addr 172.21.15.37:0/772415191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f069e800640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 0x7f06a0157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f069f200640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 0x7f06a015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 -- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 msgr2=0x7f06a015ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 0x7f06a015ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 -- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 msgr2=0x7f06a0157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 0x7f06a0157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 -- 172.21.15.37:0/772415191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00d0d50 con 0x7f06a013f400 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f069e800640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 0x7f06a0157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f069f200640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 0x7f06a015ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.047+0000 7f0697e00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0688006420 tx=0x7f06880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.048+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688015070 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.048+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0688011880 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.048+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688011cd0 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.048+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a0158820 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.048+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a0158de0 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.049+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a00b20c0 con 0x7f06a013f400 2024-09-21T06:32:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.050+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0688018070 con 0x7f06a013f400 2024-09-21T06:32:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.051+0000 7f0696a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 0x7f067c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.051+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f068809bc40 con 0x7f06a013f400 2024-09-21T06:32:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.051+0000 7f069e800640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 0x7f067c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.052+0000 7f069e800640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 0x7f067c07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0690006510 tx=0x7f0690006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:43.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.055+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06a00d0d50 con 0x7f06a013f400 2024-09-21T06:32:43.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.295+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06a00cc0e0 con 0x7f06a013f400 2024-09-21T06:32:43.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.296+0000 7f0696a00640 1 -- 172.21.15.37:0/772415191 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5277 (secure 0 0 0) 0x7f0688063e50 con 0x7f06a013f400 2024-09-21T06:32:43.298 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:43.299 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":63,"btime":"2024-09-21T06:32:16:626541+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:32:16.554630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:32:43.299 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 63 2024-09-21T06:32:43.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 msgr2=0x7f067c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 0x7f067c07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0690006510 tx=0x7f0690006b10 comp rx=0 tx=0).stop 2024-09-21T06:32:43.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 msgr2=0x7f06a00d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0688006420 tx=0x7f06880039c0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 shutdown_connections 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f067c078b90 0x7f067c07b0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a013f400 0x7f06a00d0400 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a00d0940 0x7f06a015ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 --2- 172.21.15.37:0/772415191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06840051a0 0x7f06a0157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 >> 172.21.15.37:0/772415191 conn(0x7f06a006f940 msgr2=0x7f06a00b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.298+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 shutdown_connections 2024-09-21T06:32:43.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.299+0000 7f06a4a00640 1 -- 172.21.15.37:0/772415191 wait complete. 2024-09-21T06:32:43.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.552+0000 7f4291600640 1 -- 172.21.15.37:0/213976509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c06e500 msgr2=0x7f428c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.552+0000 7f4291600640 1 --2- 172.21.15.37:0/213976509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c06e500 0x7f428c06ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4284009be0 tx=0x7f428402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.552+0000 7f4291600640 1 -- 172.21.15.37:0/213976509 shutdown_connections 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.552+0000 7f4291600640 1 --2- 172.21.15.37:0/213976509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c06e500 0x7f428c06ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.552+0000 7f4291600640 1 -- 172.21.15.37:0/213976509 >> 172.21.15.37:0/213976509 conn(0x7f428c0cca40 msgr2=0x7f428c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.553+0000 7f4291600640 1 -- 172.21.15.37:0/213976509 shutdown_connections 2024-09-21T06:32:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.553+0000 7f4291600640 1 -- 172.21.15.37:0/213976509 wait complete. 2024-09-21T06:32:43.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.555+0000 7f4291600640 1 Processor -- start 2024-09-21T06:32:43.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.555+0000 7f4291600640 1 -- start start 2024-09-21T06:32:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c074cb0 0x7f428c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c157a50 0x7f428c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f428c06f6b0 con 0x7f428c073dd0 2024-09-21T06:32:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f428c06f530 con 0x7f428c074cb0 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.556+0000 7f4291600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f428c06f830 con 0x7f428c157a50 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55742/0 (socket says 172.21.15.37:55742) 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 -- 172.21.15.37:0/530614899 learned_addr learned my addr 172.21.15.37:0/530614899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f4283e00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c074cb0 0x7f428c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428b400640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c157a50 0x7f428c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 -- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c157a50 msgr2=0x7f428c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c157a50 0x7f428c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 -- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c074cb0 msgr2=0x7f428c1534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c074cb0 0x7f428c1534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 -- 172.21.15.37:0/530614899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0b1750 con 0x7f428c073dd0 2024-09-21T06:32:43.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.557+0000 7f428aa00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c4b0 tx=0x7f427c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:43.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.558+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c016020 con 0x7f428c073dd0 2024-09-21T06:32:43.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.558+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f427c012920 con 0x7f428c073dd0 2024-09-21T06:32:43.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.558+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c012cf0 con 0x7f428c073dd0 2024-09-21T06:32:43.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.558+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c0cedf0 con 0x7f428c073dd0 2024-09-21T06:32:43.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.558+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c0cf3b0 con 0x7f428c073dd0 2024-09-21T06:32:43.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.559+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c06ede0 con 0x7f428c073dd0 2024-09-21T06:32:43.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.560+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f427c019020 con 0x7f428c073dd0 2024-09-21T06:32:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.562+0000 7f4282a00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.562+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f427c09ab60 con 0x7f428c073dd0 2024-09-21T06:32:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.562+0000 7f4283e00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.563+0000 7f4283e00640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 0x7f426807b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f428c1e8b40 tx=0x7f4274009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:43.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.567+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f427c020030 con 0x7f428c073dd0 2024-09-21T06:32:43.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.765+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f428c0ca130 con 0x7f428c073dd0 2024-09-21T06:32:43.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.766+0000 7f4282a00640 1 -- 172.21.15.37:0/530614899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17341 (secure 0 0 0) 0x7f427c0627a0 con 0x7f428c073dd0 2024-09-21T06:32:43.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:43.769 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":69,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:32:39.938002+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:31:35.929166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T06:31:36.946257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":1}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2c","mappings":[{"from":2,"to":5}]},{"pgid":"10.37","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 msgr2=0x7f426807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 0x7f426807b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f428c1e8b40 tx=0x7f4274009040 comp rx=0 tx=0).stop 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 msgr2=0x7f428c0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c4b0 tx=0x7f427c00c970 comp rx=0 tx=0).stop 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 shutdown_connections 2024-09-21T06:32:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f428c157a50 0x7f428c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c074cb0 0x7f428c1534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 --2- 172.21.15.37:0/530614899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c073dd0 0x7f428c0b0d00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 >> 172.21.15.37:0/530614899 conn(0x7f428c0cca40 msgr2=0x7f428c1e8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 shutdown_connections 2024-09-21T06:32:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:43.769+0000 7f4291600640 1 -- 172.21.15.37:0/530614899 wait complete. 2024-09-21T06:32:43.794 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:32:43.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.019+0000 7fd632600640 1 -- 172.21.15.37:0/2774533110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 msgr2=0x7fd62c15e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.019+0000 7fd632600640 1 --2- 172.21.15.37:0/2774533110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c15e5e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd624009be0 tx=0x7fd62402a790 comp rx=0 tx=0).stop 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.019+0000 7fd632600640 1 -- 172.21.15.37:0/2774533110 shutdown_connections 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.019+0000 7fd632600640 1 --2- 172.21.15.37:0/2774533110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c15e5e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.019+0000 7fd632600640 1 -- 172.21.15.37:0/2774533110 >> 172.21.15.37:0/2774533110 conn(0x7fd62c0d4360 msgr2=0x7fd62c0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.020+0000 7fd632600640 1 -- 172.21.15.37:0/2774533110 shutdown_connections 2024-09-21T06:32:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.020+0000 7fd632600640 1 -- 172.21.15.37:0/2774533110 wait complete. 2024-09-21T06:32:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.022+0000 7fd632600640 1 Processor -- start 2024-09-21T06:32:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.022+0000 7fd632600640 1 -- start start 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd62c0ceff0 0x7fd62c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0b9060 con 0x7fd62c153b20 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0b8ee0 con 0x7fd62c0ceff0 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.023+0000 7fd632600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0b91e0 con 0x7fd62c0d2520 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45192/0 (socket says 172.21.15.37:45192) 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 -- 172.21.15.37:0/2551764057 learned_addr learned my addr 172.21.15.37:0/2551764057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631200640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd62c0ceff0 0x7fd62c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd623e00640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 -- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 msgr2=0x7fd62c0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 -- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd62c0ceff0 msgr2=0x7fd62c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd62c0ceff0 0x7fd62c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.024+0000 7fd631c00640 1 -- 172.21.15.37:0/2551764057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c0cbdd0 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd631c00640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd624034040 tx=0x7fd6240305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624041030 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd624003970 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624036040 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c155390 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.025+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c0cc050 con 0x7fd62c153b20 2024-09-21T06:32:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.026+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62c15e5e0 con 0x7fd62c153b20 2024-09-21T06:32:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.027+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd624044070 con 0x7fd62c153b20 2024-09-21T06:32:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.028+0000 7fd629600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 0x7fd60807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.028+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fd6240bbcf0 con 0x7fd62c153b20 2024-09-21T06:32:44.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.029+0000 7fd631200640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 0x7fd60807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.029+0000 7fd631200640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 0x7fd60807b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001120 tx=0x7fd61c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:44.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.033+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd62c0cbdd0 con 0x7fd62c153b20 2024-09-21T06:32:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.269+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62c050ff0 con 0x7fd62c153b20 2024-09-21T06:32:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.270+0000 7fd629600640 1 -- 172.21.15.37:0/2551764057 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5277 (secure 0 0 0) 0x7fd6240838a0 con 0x7fd62c153b20 2024-09-21T06:32:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:44.272 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":63,"btime":"2024-09-21T06:32:16:626541+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:32:16.554630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:32:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 63 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 msgr2=0x7fd60807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 0x7fd60807b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001120 tx=0x7fd61c009040 comp rx=0 tx=0).stop 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 msgr2=0x7fd62c14a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd624034040 tx=0x7fd6240305d0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 shutdown_connections 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd608078b90 0x7fd60807b0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd62c153b20 0x7fd62c14a580 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd62c0d2520 0x7fd62c0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 --2- 172.21.15.37:0/2551764057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd62c0ceff0 0x7fd62c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 >> 172.21.15.37:0/2551764057 conn(0x7fd62c0d4360 msgr2=0x7fd62c06d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 shutdown_connections 2024-09-21T06:32:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.272+0000 7fd632600640 1 -- 172.21.15.37:0/2551764057 wait complete. 2024-09-21T06:32:44.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:32:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 -- 172.21.15.37:0/2045195653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f580cc8d0 msgr2=0x7f1f580ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 --2- 172.21.15.37:0/2045195653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f580cc8d0 0x7f1f580ccce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1f58074f80 tx=0x7f1f5002a9a0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 -- 172.21.15.37:0/2045195653 shutdown_connections 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 --2- 172.21.15.37:0/2045195653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f580cc8d0 0x7f1f580ccce0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 --2- 172.21.15.37:0/2045195653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 0x7f1f5806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 --2- 172.21.15.37:0/2045195653 >> v2:172.21.15.150:3300/0 conn(0x7f1f5815dfe0 0x7f1f58160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 -- 172.21.15.37:0/2045195653 >> 172.21.15.37:0/2045195653 conn(0x7f1f5806f940 msgr2=0x7f1f580b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 -- 172.21.15.37:0/2045195653 shutdown_connections 2024-09-21T06:32:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.520+0000 7f1f57200640 1 -- 172.21.15.37:0/2045195653 wait complete. 2024-09-21T06:32:44.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.522+0000 7f1f57200640 1 Processor -- start 2024-09-21T06:32:44.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.522+0000 7f1f57200640 1 -- start start 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 0x7f1f580cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 0x7f1f581f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5806d460 con 0x7f1f5815dfe0 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5806d2e0 con 0x7f1f580cc8d0 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f57200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5806d5e0 con 0x7f1f5806e160 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45208/0 (socket says 172.21.15.37:45208) 2024-09-21T06:32:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 -- 172.21.15.37:0/54305203 learned_addr learned my addr 172.21.15.37:0/54305203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f55e00640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 0x7f1f580cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f4fe00640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 0x7f1f581f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 -- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 msgr2=0x7f1f580cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 0x7f1f580cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 -- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 msgr2=0x7f1f581f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 0x7f1f581f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f56800640 1 -- 172.21.15.37:0/54305203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f581f43b0 con 0x7f1f5815dfe0 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.524+0000 7f1f4fe00640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 0x7f1f581f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f56800640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1f5002af90 tx=0x7f1f50003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50033050 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f50004450 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50038450 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581f4690 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.525+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581f4c50 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.526+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f581f4120 con 0x7f1f5815dfe0 2024-09-21T06:32:44.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.527+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f50036070 con 0x7f1f5815dfe0 2024-09-21T06:32:44.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.528+0000 7f1f4e000640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 0x7f1f3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:44.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.528+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f1f500bc420 con 0x7f1f5815dfe0 2024-09-21T06:32:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.529+0000 7f1f55e00640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 0x7f1f3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.530+0000 7f1f55e00640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 0x7f1f3407b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800bb00 tx=0x7f1f48005e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.533+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f50084590 con 0x7f1f5815dfe0 2024-09-21T06:32:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.771+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f5806db00 con 0x7f1f5815dfe0 2024-09-21T06:32:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.771+0000 7f1f4e000640 1 -- 172.21.15.37:0/54305203 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5277 (secure 0 0 0) 0x7f1f50083e90 con 0x7f1f5815dfe0 2024-09-21T06:32:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:44.774 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":63,"btime":"2024-09-21T06:32:16:626541+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:31:37.988979+0000","modified":"2024-09-21T06:32:16.554630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6106,"mds_1":6100,"mds_2":6097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6097":{"gid":6097,"name":"b","rank":2,"incarnation":56,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6836/4023630261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4023630261},{"type":"v1","addr":"172.21.15.150:6836","nonce":4023630261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6100":{"gid":6100,"name":"d","rank":1,"incarnation":54,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/2893186945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":2893186945},{"type":"v1","addr":"172.21.15.150:6837","nonce":2893186945}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6106":{"gid":6106,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.37:6835/1569973003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1569973003},{"type":"v1","addr":"172.21.15.37:6835","nonce":1569973003}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6097,6100,6106]},"id":4}]} 2024-09-21T06:32:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 63 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 msgr2=0x7f1f3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 0x7f1f3407b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800bb00 tx=0x7f1f48005e30 comp rx=0 tx=0).stop 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 msgr2=0x7f1f581e9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1f5002af90 tx=0x7f1f50003f90 comp rx=0 tx=0).stop 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 shutdown_connections 2024-09-21T06:32:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1f34078bc0 0x7f1f3407b0d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f5815dfe0 0x7f1f581e9020 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f580cc8d0 0x7f1f581f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 --2- 172.21.15.37:0/54305203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f5806e160 0x7f1f580cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 >> 172.21.15.37:0/54305203 conn(0x7f1f5806f940 msgr2=0x7f1f580b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 shutdown_connections 2024-09-21T06:32:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:44.774+0000 7f1f57200640 1 -- 172.21.15.37:0/54305203 wait complete. 2024-09-21T06:32:44.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:32:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 -- 172.21.15.37:0/3739506695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea7806e330 msgr2=0x7fea780cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 --2- 172.21.15.37:0/3739506695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea7806e330 0x7fea780cdb90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fea70005fd0 tx=0x7fea7002a8a0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 -- 172.21.15.37:0/3739506695 shutdown_connections 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 --2- 172.21.15.37:0/3739506695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea7806e330 0x7fea780cdb90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 --2- 172.21.15.37:0/3739506695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78070030 0x7fea7806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 -- 172.21.15.37:0/3739506695 >> 172.21.15.37:0/3739506695 conn(0x7fea780cca40 msgr2=0x7fea780cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 -- 172.21.15.37:0/3739506695 shutdown_connections 2024-09-21T06:32:45.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.027+0000 7fea7d200640 1 -- 172.21.15.37:0/3739506695 wait complete. 2024-09-21T06:32:45.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.029+0000 7fea7d200640 1 Processor -- start 2024-09-21T06:32:45.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.029+0000 7fea7d200640 1 -- start start 2024-09-21T06:32:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea7806e330 0x7fea78152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78074d90 0x7fea780a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea78071000 con 0x7fea78070030 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea78070e80 con 0x7fea7806e330 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea7d200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fea780ce0d0 con 0x7fea78074d90 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45228/0 (socket says 172.21.15.37:45228) 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 -- 172.21.15.37:0/2321679516 learned_addr learned my addr 172.21.15.37:0/2321679516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea75e00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea7806e330 0x7fea78152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea76800640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78074d90 0x7fea780a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 -- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78074d90 msgr2=0x7fea780a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78074d90 0x7fea780a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 -- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea7806e330 msgr2=0x7fea78152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea7806e330 0x7fea78152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.031+0000 7fea6fe00640 1 -- 172.21.15.37:0/2321679516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea78156120 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea6fe00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fea6000e790 tx=0x7fea6000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000cac0 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea6000cc60 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60005280 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea78159430 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.032+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea781596d0 con 0x7fea78070030 2024-09-21T06:32:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.033+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea78159e30 con 0x7fea78070030 2024-09-21T06:32:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.034+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea60005420 con 0x7fea78070030 2024-09-21T06:32:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.035+0000 7fea6e000640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 0x7fea5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.036+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fea6009b820 con 0x7fea78070030 2024-09-21T06:32:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.036+0000 7fea75e00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 0x7fea5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.037+0000 7fea75e00640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 0x7fea5407b0d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fea680046b0 tx=0x7fea68009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:45.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.040+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea78156120 con 0x7fea78070030 2024-09-21T06:32:45.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.236+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea780ce610 con 0x7fea78070030 2024-09-21T06:32:45.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.237+0000 7fea6e000640 1 -- 172.21.15.37:0/2321679516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17341 (secure 0 0 0) 0x7fea60063ab0 con 0x7fea78070030 2024-09-21T06:32:45.240 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:45.240 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":69,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:32:39.938002+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:31:35.929166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T06:31:36.946257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":1}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2c","mappings":[{"from":2,"to":5}]},{"pgid":"10.37","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:32:45.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 msgr2=0x7fea5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 0x7fea5407b0d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fea680046b0 tx=0x7fea68009040 comp rx=0 tx=0).stop 2024-09-21T06:32:45.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 msgr2=0x7fea780a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fea6000e790 tx=0x7fea6000ec50 comp rx=0 tx=0).stop 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 shutdown_connections 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fea54078bc0 0x7fea5407b0d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea78070030 0x7fea780a05f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea78074d90 0x7fea780a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 --2- 172.21.15.37:0/2321679516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea7806e330 0x7fea78152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 >> 172.21.15.37:0/2321679516 conn(0x7fea780cca40 msgr2=0x7fea780742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 shutdown_connections 2024-09-21T06:32:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.240+0000 7fea7d200640 1 -- 172.21.15.37:0/2321679516 wait complete. 2024-09-21T06:32:45.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 -- 172.21.15.37:0/1036996183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf800702a0 msgr2=0x7fcf800cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 --2- 172.21.15.37:0/1036996183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf800702a0 0x7fcf800cf6d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf78005fd0 tx=0x7fcf7802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 -- 172.21.15.37:0/1036996183 shutdown_connections 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 --2- 172.21.15.37:0/1036996183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf800702a0 0x7fcf800cf6d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 -- 172.21.15.37:0/1036996183 >> 172.21.15.37:0/1036996183 conn(0x7fcf800d05c0 msgr2=0x7fcf8006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.492+0000 7fcf7f200640 1 -- 172.21.15.37:0/1036996183 shutdown_connections 2024-09-21T06:32:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.493+0000 7fcf7f200640 1 -- 172.21.15.37:0/1036996183 wait complete. 2024-09-21T06:32:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.495+0000 7fcf7f200640 1 Processor -- start 2024-09-21T06:32:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.495+0000 7fcf7f200640 1 -- start start 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf800cc0e0 0x7fcf80141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8013cb50 0x7fcf80141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800b3e50 con 0x7fcf800702a0 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800b3cd0 con 0x7fcf8013cb50 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcf800b3fd0 con 0x7fcf800cc0e0 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.496+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45230/0 (socket says 172.21.15.37:45230) 2024-09-21T06:32:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 -- 172.21.15.37:0/2343134533 learned_addr learned my addr 172.21.15.37:0/2343134533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7e800640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8013cb50 0x7fcf80141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf75400640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf800cc0e0 0x7fcf80141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 -- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf800cc0e0 msgr2=0x7fcf80141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf800cc0e0 0x7fcf80141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 -- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8013cb50 msgr2=0x7fcf80141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8013cb50 0x7fcf80141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 -- 172.21.15.37:0/2343134533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf80156fa0 con 0x7fcf800702a0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.497+0000 7fcf7de00640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c400 tx=0x7fcf7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.498+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70017070 con 0x7fcf800702a0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.498+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf700053c0 con 0x7fcf800702a0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.498+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70003850 con 0x7fcf800702a0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.498+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf80157280 con 0x7fcf800702a0 2024-09-21T06:32:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.498+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf801f3040 con 0x7fcf800702a0 2024-09-21T06:32:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.499+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf800cf6d0 con 0x7fcf800702a0 2024-09-21T06:32:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.500+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf7001a070 con 0x7fcf800702a0 2024-09-21T06:32:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.501+0000 7fcf76a00640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 0x7fcf5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.501+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fcf7009a330 con 0x7fcf800702a0 2024-09-21T06:32:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.502+0000 7fcf75400640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 0x7fcf5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.502+0000 7fcf75400640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 0x7fcf5c07b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf8013e900 tx=0x7fcf68007cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.505+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf70061ee0 con 0x7fcf800702a0 2024-09-21T06:32:45.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.741+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcf80142780 con 0x7fcf800702a0 2024-09-21T06:32:45.744 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:32:45.742+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:32:45.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.745+0000 7fcf76a00640 1 -- 172.21.15.37:0/2343134533 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) ==== 161+0+0 (secure 0 0 0) 0x7fcf70066910 con 0x7fcf800702a0 2024-09-21T06:32:45.747 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.748+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 msgr2=0x7fcf5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.748+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 0x7fcf5c07b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf8013e900 tx=0x7fcf68007cc0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.748+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 msgr2=0x7fcf8013bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.748+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c400 tx=0x7fcf7000c8c0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 shutdown_connections 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcf5c078b90 0x7fcf5c07b0a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8013cb50 0x7fcf80141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf800cc0e0 0x7fcf80141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 --2- 172.21.15.37:0/2343134533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf800702a0 0x7fcf8013bff0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:45.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 >> 172.21.15.37:0/2343134533 conn(0x7fcf800d05c0 msgr2=0x7fcf8006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:45.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 shutdown_connections 2024-09-21T06:32:45.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:45.749+0000 7fcf7f200640 1 -- 172.21.15.37:0/2343134533 wait complete. 2024-09-21T06:32:45.765 INFO:tasks.ceph.mds.b.smithi150.stderr: -1629> 2024-09-21T06:32:09.711+0000 7fa37dc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1) n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560c7684e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.838586+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.765 INFO:tasks.ceph.mds.b.smithi150.stderr: -1602> 2024-09-21T06:32:09.711+0000 7fa37dc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/AsLsXsFsx@2},l=6220 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560c7684e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.768 INFO:tasks.ceph.mds.d.smithi150.stderr: -1742> 2024-09-21T06:32:09.710+0000 7efc40600640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1) n(v2 rc2024-09-21T06:31:52.126583+0000 4=0+4) (inest lock) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56113f526100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:31:52.126583+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.768 INFO:tasks.ceph.mds.d.smithi150.stderr: -1590> 2024-09-21T06:32:09.712+0000 7efc48800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1) n(v1 rc2024-09-21T06:31:52.126583+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6220=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56113f526680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:51.838586+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.837586+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:31:52.126583+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.769 INFO:tasks.ceph.mds.a.smithi037.stderr: -3863> 2024-09-21T06:32:09.709+0000 7f34e7400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1) n(v3 rc2024-09-21T06:31:52.126583+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6220=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6220 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5644cb46a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.835587+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:31:52.126583+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.769 INFO:tasks.ceph.mds.a.smithi037.stderr: -3723> 2024-09-21T06:32:09.711+0000 7f34ef600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1) n(v4 rc2024-09-21T06:31:52.126583+0000 5=0+5) (inest sync->mix g=1,2) caps={6220=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5644cb46d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:31:52.126583+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:31:51.836586+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:31:52.126583+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:32:45.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:32:45.887 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:32:45.888 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:32:45.892 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:32:46.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 -- 172.21.15.37:0/1560206216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf806e500 msgr2=0x7f9cf80cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:46.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 --2- 172.21.15.37:0/1560206216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf806e500 0x7f9cf80cef00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b2c70 tx=0x7f9cf002aac0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 -- 172.21.15.37:0/1560206216 shutdown_connections 2024-09-21T06:32:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 --2- 172.21.15.37:0/1560206216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf806e500 0x7f9cf80cef00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 --2- 172.21.15.37:0/1560206216 >> v2:172.21.15.150:3300/0 conn(0x7f9cf8156a50 0x7f9cf8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 -- 172.21.15.37:0/1560206216 >> 172.21.15.37:0/1560206216 conn(0x7f9cf80b1b90 msgr2=0x7f9cf80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 -- 172.21.15.37:0/1560206216 shutdown_connections 2024-09-21T06:32:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.012+0000 7f9cfee00640 1 -- 172.21.15.37:0/1560206216 wait complete. 2024-09-21T06:32:46.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.014+0000 7f9cfee00640 1 Processor -- start 2024-09-21T06:32:46.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.014+0000 7f9cfee00640 1 -- start start 2024-09-21T06:32:46.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cf8074b60 0x7f9cf80cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:46.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:46.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cf81f9060 0x7f9cf815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:46.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf80cf7d0 con 0x7f9cf8156a50 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf80cf650 con 0x7f9cf81f9060 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf80cf950 con 0x7f9cf8074b60 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cefe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cefe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45246/0 (socket says 172.21.15.37:45246) 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cefe00640 1 -- 172.21.15.37:0/2667467503 learned_addr learned my addr 172.21.15.37:0/2667467503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cf7e00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cf8074b60 0x7f9cf80cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cfce00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cf81f9060 0x7f9cf815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.016+0000 7f9cefe00640 1 -- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cf8074b60 msgr2=0x7f9cf80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cefe00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cf8074b60 0x7f9cf80cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cefe00640 1 -- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cf81f9060 msgr2=0x7f9cf815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cefe00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cf81f9060 0x7f9cf815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cefe00640 1 -- 172.21.15.37:0/2667467503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf81f5130 con 0x7f9cf8156a50 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cefe00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3f0 tx=0x7f9ce000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0017070 con 0x7f9cf8156a50 2024-09-21T06:32:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce00035f0 con 0x7f9cf8156a50 2024-09-21T06:32:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0003a40 con 0x7f9cf8156a50 2024-09-21T06:32:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf81f5410 con 0x7f9cf8156a50 2024-09-21T06:32:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.017+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf80cc150 con 0x7f9cf8156a50 2024-09-21T06:32:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.018+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf80cef00 con 0x7f9cf8156a50 2024-09-21T06:32:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.020+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ce001a070 con 0x7f9cf8156a50 2024-09-21T06:32:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.021+0000 7f9cf4c00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 0x7f9cd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.021+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6562+0+0 (secure 0 0 0) 0x7f9ce009a340 con 0x7f9cf8156a50 2024-09-21T06:32:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.021+0000 7f9cf7e00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 0x7f9cd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.022+0000 7f9cf7e00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 0x7f9cd407b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009b30 tx=0x7f9ce8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.025+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ce0061df0 con 0x7f9cf8156a50 2024-09-21T06:32:46.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.262+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9cf80cc460 con 0x7f9cf8156a50 2024-09-21T06:32:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.747+0000 7f9cf4c00640 1 -- 172.21.15.37:0/2667467503 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) ==== 104+0+0 (secure 0 0 0) 0x7f9ce0066820 con 0x7f9cf8156a50 2024-09-21T06:32:46.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.756+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 msgr2=0x7f9cd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:46.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.756+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 0x7f9cd407b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009b30 tx=0x7f9ce8005f90 comp rx=0 tx=0).stop 2024-09-21T06:32:46.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.756+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 msgr2=0x7f9cf81f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:46.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.756+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3f0 tx=0x7f9ce000c8b0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 shutdown_connections 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cd4078b90 0x7f9cd407b0a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf8156a50 0x7f9cf81f8b20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cf81f9060 0x7f9cf815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 --2- 172.21.15.37:0/2667467503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cf8074b60 0x7f9cf80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 >> 172.21.15.37:0/2667467503 conn(0x7f9cf80b1b90 msgr2=0x7f9cf80b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 shutdown_connections 2024-09-21T06:32:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:46.759+0000 7f9cfee00640 1 -- 172.21.15.37:0/2667467503 wait complete. 2024-09-21T06:32:46.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:32:47.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 -- 172.21.15.37:0/2366342939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00b4a20 msgr2=0x7f3ec00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:47.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 --2- 172.21.15.37:0/2366342939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00b4a20 0x7f3ec00b4e30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00b6d50 tx=0x7f3eac02a560 comp rx=0 tx=0).stop 2024-09-21T06:32:47.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 -- 172.21.15.37:0/2366342939 shutdown_connections 2024-09-21T06:32:47.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 --2- 172.21.15.37:0/2366342939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00b4a20 0x7f3ec00b4e30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 --2- 172.21.15.37:0/2366342939 >> v2:172.21.15.150:3300/0 conn(0x7f3ec00ced00 0x7f3ec00b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 -- 172.21.15.37:0/2366342939 >> 172.21.15.37:0/2366342939 conn(0x7f3ec00d18c0 msgr2=0x7f3ec00d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:47.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 -- 172.21.15.37:0/2366342939 shutdown_connections 2024-09-21T06:32:47.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.019+0000 7f3ec5400640 1 -- 172.21.15.37:0/2366342939 wait complete. 2024-09-21T06:32:47.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.021+0000 7f3ec5400640 1 Processor -- start 2024-09-21T06:32:47.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.021+0000 7f3ec5400640 1 -- start start 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00b4a20 0x7f3ec0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec00b5370 0x7f3ec0156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec00d33c0 con 0x7f3ec00ced00 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec00b3420 con 0x7f3ec00b4a20 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec5400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec00d3540 con 0x7f3ec00b5370 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:47.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45258/0 (socket says 172.21.15.37:45258) 2024-09-21T06:32:47.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 -- 172.21.15.37:0/673031953 learned_addr learned my addr 172.21.15.37:0/673031953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:47.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ebfe00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00b4a20 0x7f3ec0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:47.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ebf400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec00b5370 0x7f3ec0156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 -- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec00b5370 msgr2=0x7f3ec0156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec00b5370 0x7f3ec0156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 -- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00b4a20 msgr2=0x7f3ec0151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00b4a20 0x7f3ec0151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.023+0000 7f3ec4a00640 1 -- 172.21.15.37:0/673031953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec0156a70 con 0x7f3ec00ced00 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3ec4a00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8002230 tx=0x7f3eb800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8003b70 con 0x7f3ec00ced00 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3eb8003d10 con 0x7f3ec00ced00 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec0153c20 con 0x7f3ec00ced00 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8005330 con 0x7f3ec00ced00 2024-09-21T06:32:47.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.024+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec01541e0 con 0x7f3ec00ced00 2024-09-21T06:32:47.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.025+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec00d3a10 con 0x7f3ec00ced00 2024-09-21T06:32:47.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.026+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3eb80056e0 con 0x7f3ec00ced00 2024-09-21T06:32:47.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.028+0000 7f3eb7e00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 0x7f3ea007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:47.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.028+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6562+0+0 (secure 0 0 0) 0x7f3eb809af20 con 0x7f3ec00ced00 2024-09-21T06:32:47.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.028+0000 7f3ebfe00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 0x7f3ea007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:47.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.029+0000 7f3ebfe00640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 0x7f3ea007b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000bb50 tx=0x7f3eb000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.033+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3eb80629a0 con 0x7f3ec00ced00 2024-09-21T06:32:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.234+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ec0156800 con 0x7f3ec00ced00 2024-09-21T06:32:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.750+0000 7f3eb7e00640 1 -- 172.21.15.37:0/673031953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) ==== 181+0+0 (secure 0 0 0) 0x7f3eb80673d0 con 0x7f3ec00ced00 2024-09-21T06:32:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:32:47.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.752+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 msgr2=0x7f3ea007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:47.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.752+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 0x7f3ea007b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000bb50 tx=0x7f3eb000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.752+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 msgr2=0x7f3ec0149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.752+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8002230 tx=0x7f3eb800cb90 comp rx=0 tx=0).stop 2024-09-21T06:32:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 shutdown_connections 2024-09-21T06:32:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078bc0 0x7f3ea007b0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec00ced00 0x7f3ec0149470 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec00b5370 0x7f3ec0156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 --2- 172.21.15.37:0/673031953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00b4a20 0x7f3ec0151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 >> 172.21.15.37:0/673031953 conn(0x7f3ec00d18c0 msgr2=0x7f3ec015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 shutdown_connections 2024-09-21T06:32:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:47.753+0000 7f3ec5400640 1 -- 172.21.15.37:0/673031953 wait complete. 2024-09-21T06:32:47.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:32:48.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 -- 172.21.15.37:0/220233043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b80041e0 msgr2=0x7fc3b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:48.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 --2- 172.21.15.37:0/220233043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b80041e0 0x7fc3b8004800 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc001830 tx=0x7fc3b8005830 comp rx=0 tx=0).stop 2024-09-21T06:32:48.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d5400640 1 -- 172.21.15.37:0/220233043 reap_dead start 2024-09-21T06:32:48.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 -- 172.21.15.37:0/220233043 shutdown_connections 2024-09-21T06:32:48.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 -- 172.21.15.37:0/220233043 >> 172.21.15.37:0/220233043 conn(0x7fc3d00ccac0 msgr2=0x7fc3d00cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:48.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 -- 172.21.15.37:0/220233043 shutdown_connections 2024-09-21T06:32:48.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.022+0000 7fc3d7400640 1 -- 172.21.15.37:0/220233043 wait complete. 2024-09-21T06:32:48.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.024+0000 7fc3d7400640 1 Processor -- start 2024-09-21T06:32:48.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.024+0000 7fc3d7400640 1 -- start start 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 0x7fc3d00c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3d01535b0 0x7fc3d00c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d006dd80 con 0x7fc3d009c140 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d006dc00 con 0x7fc3d01535b0 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d7400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d006df00 con 0x7fc3b80041e0 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d4a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 0x7fc3d00c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:48.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3cfe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3d01535b0 0x7fc3d00c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45276/0 (socket says 172.21.15.37:45276) 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.026+0000 7fc3d5400640 1 -- 172.21.15.37:0/1842443019 learned_addr learned my addr 172.21.15.37:0/1842443019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 -- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 msgr2=0x7fc3d00c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 0x7fc3d00c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 -- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3d01535b0 msgr2=0x7fc3d00c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3d01535b0 0x7fc3d00c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 -- 172.21.15.37:0/1842443019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d009cec0 con 0x7fc3d009c140 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d4a00640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 0x7fc3d00c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:32:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d5400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005fd0 tx=0x7fc3c800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8003970 con 0x7fc3d009c140 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c8003b10 con 0x7fc3d009c140 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8004030 con 0x7fc3d009c140 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.027+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d009d1a0 con 0x7fc3d009c140 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.028+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d00c8990 con 0x7fc3d009c140 2024-09-21T06:32:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.029+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d009d3e0 con 0x7fc3d009c140 2024-09-21T06:32:48.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.030+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc3c80041d0 con 0x7fc3d009c140 2024-09-21T06:32:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.031+0000 7fc3ccc00640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 0x7fc3b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.031+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6124+0+0 (secure 0 0 0) 0x7fc3c809b410 con 0x7fc3d009c140 2024-09-21T06:32:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.031+0000 7fc3d4a00640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 0x7fc3b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:48.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.032+0000 7fc3d4a00640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 0x7fc3b007b0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4001220 tx=0x7fc3c4001110 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.035+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3d009d3e0 con 0x7fc3d009c140 2024-09-21T06:32:48.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.233+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3d006e460 con 0x7fc3d009c140 2024-09-21T06:32:48.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.761+0000 7fc3ccc00640 1 -- 172.21.15.37:0/1842443019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) ==== 169+0+0 (secure 0 0 0) 0x7fc3c8062f40 con 0x7fc3d009c140 2024-09-21T06:32:48.764 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.764+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 msgr2=0x7fc3b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.764+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 0x7fc3b007b0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4001220 tx=0x7fc3c4001110 comp rx=0 tx=0).stop 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.764+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 msgr2=0x7fc3d009c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.764+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005fd0 tx=0x7fc3c800f3c0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 shutdown_connections 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc3b0078b90 0x7fc3b007b0a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d009c140 0x7fc3d009c550 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3d01535b0 0x7fc3d00c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 --2- 172.21.15.37:0/1842443019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3b80041e0 0x7fc3d00c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 >> 172.21.15.37:0/1842443019 conn(0x7fc3d00ccac0 msgr2=0x7fc3d00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 shutdown_connections 2024-09-21T06:32:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:48.765+0000 7fc3d7400640 1 -- 172.21.15.37:0/1842443019 wait complete. 2024-09-21T06:32:48.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:32:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 -- 172.21.15.37:0/4035714457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd8406ddf0 msgr2=0x7fdd8406e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 --2- 172.21.15.37:0/4035714457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd8406ddf0 0x7fdd8406e200 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fdd8406fe40 tx=0x7fdd7002a5f0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 -- 172.21.15.37:0/4035714457 shutdown_connections 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 --2- 172.21.15.37:0/4035714457 >> v2:172.21.15.150:3300/0 conn(0x7fdd840cef00 0x7fdd840cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 --2- 172.21.15.37:0/4035714457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd8406ddf0 0x7fdd8406e200 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 -- 172.21.15.37:0/4035714457 >> 172.21.15.37:0/4035714457 conn(0x7fdd840b1b90 msgr2=0x7fdd840b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 -- 172.21.15.37:0/4035714457 shutdown_connections 2024-09-21T06:32:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.028+0000 7fdd8b800640 1 -- 172.21.15.37:0/4035714457 wait complete. 2024-09-21T06:32:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.030+0000 7fdd8b800640 1 Processor -- start 2024-09-21T06:32:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.030+0000 7fdd8b800640 1 -- start start 2024-09-21T06:32:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 0x7fdd84154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd840cef00 0x7fdd8409edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd840cf850 con 0x7fdd84156a50 2024-09-21T06:32:49.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84161060 con 0x7fdd8406ddf0 2024-09-21T06:32:49.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd8b800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdd840cf9d0 con 0x7fdd840cef00 2024-09-21T06:32:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd89800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd89800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45300/0 (socket says 172.21.15.37:45300) 2024-09-21T06:32:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd89800640 1 -- 172.21.15.37:0/2126361137 learned_addr learned my addr 172.21.15.37:0/2126361137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd88e00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 0x7fdd84154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.032+0000 7fdd83e00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd840cef00 0x7fdd8409edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 -- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd840cef00 msgr2=0x7fdd8409edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd840cef00 0x7fdd8409edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 -- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 msgr2=0x7fdd84154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 0x7fdd84154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 -- 172.21.15.37:0/2126361137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd840b5480 con 0x7fdd84156a50 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd88e00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 0x7fdd84154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd89800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e990 tx=0x7fdd7c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c004d20 con 0x7fdd84156a50 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd7c00ca40 con 0x7fdd84156a50 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd840b3070 con 0x7fdd84156a50 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c005020 con 0x7fdd84156a50 2024-09-21T06:32:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.033+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd840b3350 con 0x7fdd84156a50 2024-09-21T06:32:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.034+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8406e200 con 0x7fdd84156a50 2024-09-21T06:32:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.036+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd7c0051c0 con 0x7fdd84156a50 2024-09-21T06:32:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.037+0000 7fdd80c00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 0x7fdd6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.037+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fdd7c09b560 con 0x7fdd84156a50 2024-09-21T06:32:49.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.037+0000 7fdd88e00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 0x7fdd6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.038+0000 7fdd88e00640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 0x7fdd6407b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007fd0 tx=0x7fdd78009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.041+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdd7c016030 con 0x7fdd84156a50 2024-09-21T06:32:49.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.239+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fdd840cf310 con 0x7fdd84156a50 2024-09-21T06:32:49.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.244+0000 7fdd80c00640 1 -- 172.21.15.37:0/2126361137 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7fdd7c063a50 con 0x7fdd84156a50 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 msgr2=0x7fdd6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 0x7fdd6407b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007fd0 tx=0x7fdd78009040 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 msgr2=0x7fdd8409f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e990 tx=0x7fdd7c00ee50 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 shutdown_connections 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd64078b90 0x7fdd6407b0a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84156a50 0x7fdd8409f330 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd840cef00 0x7fdd8409edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 --2- 172.21.15.37:0/2126361137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8406ddf0 0x7fdd84154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 >> 172.21.15.37:0/2126361137 conn(0x7fdd840b1b90 msgr2=0x7fdd840b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.246+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 shutdown_connections 2024-09-21T06:32:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.247+0000 7fdd8b800640 1 -- 172.21.15.37:0/2126361137 wait complete. 2024-09-21T06:32:49.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-21T06:32:49.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 -- 172.21.15.37:0/374803472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500039e0 msgr2=0x7fae50003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 --2- 172.21.15.37:0/374803472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500039e0 0x7fae50003e10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009b30 tx=0x7fae50005830 comp rx=0 tx=0).stop 2024-09-21T06:32:49.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6b400640 1 -- 172.21.15.37:0/374803472 reap_dead start 2024-09-21T06:32:49.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 -- 172.21.15.37:0/374803472 shutdown_connections 2024-09-21T06:32:49.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 -- 172.21.15.37:0/374803472 >> 172.21.15.37:0/374803472 conn(0x7fae6c0d4360 msgr2=0x7fae6c0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:49.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 -- 172.21.15.37:0/374803472 shutdown_connections 2024-09-21T06:32:49.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.502+0000 7fae6be00640 1 -- 172.21.15.37:0/374803472 wait complete. 2024-09-21T06:32:49.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.504+0000 7fae6be00640 1 Processor -- start 2024-09-21T06:32:49.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.504+0000 7fae6be00640 1 -- start start 2024-09-21T06:32:49.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae500039e0 0x7fae6c1f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c0cb410 0x7fae6c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0b8a00 con 0x7fae6c155d30 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0b8b80 con 0x7fae6c0cb410 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae6be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0b8d00 con 0x7fae500039e0 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45324/0 (socket says 172.21.15.37:45324) 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 -- 172.21.15.37:0/1913811653 learned_addr learned my addr 172.21.15.37:0/1913811653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 -- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae500039e0 msgr2=0x7fae6c1f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae500039e0 0x7fae6c1f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 -- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c0cb410 msgr2=0x7fae6c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c0cb410 0x7fae6c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.506+0000 7fae63e00640 1 -- 172.21.15.37:0/1913811653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0cbd60 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae63e00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fae54003f50 tx=0x7fae540017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54015070 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae54011840 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c151600 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54011c70 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.507+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c155250 con 0x7fae6c155d30 2024-09-21T06:32:49.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.508+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c0ce070 con 0x7fae6c155d30 2024-09-21T06:32:49.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.510+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae54018070 con 0x7fae6c155d30 2024-09-21T06:32:49.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.510+0000 7fae62a00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 0x7fae4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.511+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fae5409b9c0 con 0x7fae6c155d30 2024-09-21T06:32:49.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.511+0000 7fae6aa00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 0x7fae4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.512+0000 7fae6aa00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 0x7fae4807b0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.515+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae54063eb0 con 0x7fae6c155d30 2024-09-21T06:32:49.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:49.789+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7fae6c0b9a20 con 0x7fae6c155d30 2024-09-21T06:32:50.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:32:50.220 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:32:50.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.769+0000 7fae62a00640 1 -- 172.21.15.37:0/1913811653 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v262) ==== 142+0+0 (secure 0 0 0) 0x7fae540637b0 con 0x7fae6c155d30 2024-09-21T06:32:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 msgr2=0x7fae4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 0x7fae4807b0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c004730 comp rx=0 tx=0).stop 2024-09-21T06:32:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 msgr2=0x7fae6c1510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fae54003f50 tx=0x7fae540017d0 comp rx=0 tx=0).stop 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 shutdown_connections 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fae48078b90 0x7fae4807b0a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c155d30 0x7fae6c1510c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c0cb410 0x7fae6c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 --2- 172.21.15.37:0/1913811653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae500039e0 0x7fae6c1f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 >> 172.21.15.37:0/1913811653 conn(0x7fae6c0d4360 msgr2=0x7fae6c06de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 shutdown_connections 2024-09-21T06:32:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:50.773+0000 7fae6be00640 1 -- 172.21.15.37:0/1913811653 wait complete. 2024-09-21T06:32:50.803 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:32:50.803 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:32:50.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:32:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 -- 172.21.15.37:0/3640467280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 msgr2=0x7f169c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 --2- 172.21.15.37:0/3640467280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0cc810 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f169c0b3ff0 tx=0x7f169002a560 comp rx=0 tx=0).stop 2024-09-21T06:32:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 -- 172.21.15.37:0/3640467280 shutdown_connections 2024-09-21T06:32:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 --2- 172.21.15.37:0/3640467280 >> v2:172.21.15.150:3300/0 conn(0x7f169c0ccd50 0x7f169c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 --2- 172.21.15.37:0/3640467280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0cc810 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 -- 172.21.15.37:0/3640467280 >> 172.21.15.37:0/3640467280 conn(0x7f169c06f940 msgr2=0x7f169c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 -- 172.21.15.37:0/3640467280 shutdown_connections 2024-09-21T06:32:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.035+0000 7f16a3000640 1 -- 172.21.15.37:0/3640467280 wait complete. 2024-09-21T06:32:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.037+0000 7f16a3000640 1 Processor -- start 2024-09-21T06:32:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.037+0000 7f16a3000640 1 -- start start 2024-09-21T06:32:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c0b4ed0 0x7f169c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c0ccd50 0x7f169c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f169c06e560 con 0x7f169c0cddc0 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f169c06e3e0 con 0x7f169c0b4ed0 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a3000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f169c15d630 con 0x7f169c0ccd50 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45348/0 (socket says 172.21.15.37:45348) 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 -- 172.21.15.37:0/48221659 learned_addr learned my addr 172.21.15.37:0/48221659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a1200640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c0ccd50 0x7f169c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a1c00640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c0b4ed0 0x7f169c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 -- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c0ccd50 msgr2=0x7f169c1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c0ccd50 0x7f169c1590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 -- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c0b4ed0 msgr2=0x7f169c14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c0b4ed0 0x7f169c14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.039+0000 7f16a2600640 1 -- 172.21.15.37:0/48221659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c1f6a90 con 0x7f169c0cddc0 2024-09-21T06:32:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f16a2600640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f169400c760 tx=0x7f169400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694016020 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f169400ce10 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c1f6d70 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694006e60 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.040+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c1f7330 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.041+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c0b4990 con 0x7f169c0cddc0 2024-09-21T06:32:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.043+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1694003bf0 con 0x7f169c0cddc0 2024-09-21T06:32:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.043+0000 7f1699600640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 0x7f167c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.044+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f169409a6c0 con 0x7f169c0cddc0 2024-09-21T06:32:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.044+0000 7f16a1c00640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 0x7f167c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.045+0000 7f16a1c00640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 0x7f167c07b0a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1690033000 tx=0x7f1690004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.048+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1694020020 con 0x7f169c0cddc0 2024-09-21T06:32:51.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.247+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f169c0cc4e0 con 0x7f169c0cddc0 2024-09-21T06:32:51.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.248+0000 7f1699600640 1 -- 172.21.15.37:0/48221659 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+13582 (secure 0 0 0) 0x7f1694069070 con 0x7f169c0cddc0 2024-09-21T06:32:51.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:51.251 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":72,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:32:48.760669+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6836/4023630261":"2024-09-22T06:32:45.743237+0000","172.21.15.150:6834/4023630261":"2024-09-22T06:32:45.743237+0000","172.21.15.150:6837/2893186945":"2024-09-22T06:32:45.743255+0000","172.21.15.150:6835/2893186945":"2024-09-22T06:32:45.743255+0000","172.21.15.37:6835/1569973003":"2024-09-22T06:32:45.743260+0000","172.21.15.37:6833/1569973003":"2024-09-22T06:32:45.743260+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:32:51.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 msgr2=0x7f167c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 0x7f167c07b0a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1690033000 tx=0x7f1690004730 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 msgr2=0x7f169c0b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f169400c760 tx=0x7f169400cc20 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 shutdown_connections 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f167c078b90 0x7f167c07b0a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f169c0cddc0 0x7f169c0b0df0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c0ccd50 0x7f169c1590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 --2- 172.21.15.37:0/48221659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c0b4ed0 0x7f169c14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 >> 172.21.15.37:0/48221659 conn(0x7f169c06f940 msgr2=0x7f169c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 shutdown_connections 2024-09-21T06:32:51.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.251+0000 7f16a3000640 1 -- 172.21.15.37:0/48221659 wait complete. 2024-09-21T06:32:51.276 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:32:51.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:32:51.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd2a00640 1 -- 172.21.15.37:0/79063738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc031bf0 con 0x7f8bd4072b90 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 -- 172.21.15.37:0/79063738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd4072b90 msgr2=0x7f8bd4072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 --2- 172.21.15.37:0/79063738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd4072b90 0x7f8bd4072fa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009830 tx=0x7f8bbc02a5c0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 -- 172.21.15.37:0/79063738 shutdown_connections 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 --2- 172.21.15.37:0/79063738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd4072b90 0x7f8bd4072fa0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 -- 172.21.15.37:0/79063738 >> 172.21.15.37:0/79063738 conn(0x7f8bd40ccd70 msgr2=0x7f8bd40cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 -- 172.21.15.37:0/79063738 shutdown_connections 2024-09-21T06:32:51.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.505+0000 7f8bd9200640 1 -- 172.21.15.37:0/79063738 wait complete. 2024-09-21T06:32:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.508+0000 7f8bd9200640 1 Processor -- start 2024-09-21T06:32:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.508+0000 7f8bd9200640 1 -- start start 2024-09-21T06:32:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 0x7f8bd4153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8bd40b5640 0x7f8bd4155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4070190 con 0x7f8bd41533b0 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4070010 con 0x7f8bd4072b90 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd9200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd4070310 con 0x7f8bd40b5640 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.509+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45366/0 (socket says 172.21.15.37:45366) 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 -- 172.21.15.37:0/1750195360 learned_addr learned my addr 172.21.15.37:0/1750195360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bcbe00640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8bd40b5640 0x7f8bd4155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3400640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 0x7f8bd4153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 -- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8bd40b5640 msgr2=0x7f8bd4155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8bd40b5640 0x7f8bd4155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 -- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 msgr2=0x7f8bd4153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 0x7f8bd4153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 -- 172.21.15.37:0/1750195360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd4154260 con 0x7f8bd41533b0 2024-09-21T06:32:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3400640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 0x7f8bd4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bd3e00640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00ca20 tx=0x7f8bcc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.510+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc016020 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.511+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bcc00ace0 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.511+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc012680 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.511+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd4154540 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.511+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd4154b00 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.512+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4072fa0 con 0x7f8bd41533b0 2024-09-21T06:32:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.513+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8bcc019020 con 0x7f8bd41533b0 2024-09-21T06:32:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.514+0000 7f8bcb400640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 0x7f8bb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.514+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f8bcc09b2b0 con 0x7f8bd41533b0 2024-09-21T06:32:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.515+0000 7f8bd3400640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 0x7f8bb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.515+0000 7f8bd3400640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 0x7f8bb007b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.519+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8bcc020020 con 0x7f8bd41533b0 2024-09-21T06:32:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.715+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bd4153ff0 con 0x7f8bd41533b0 2024-09-21T06:32:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.716+0000 7f8bcb400640 1 -- 172.21.15.37:0/1750195360 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8bcc063710 con 0x7f8bd41533b0 2024-09-21T06:32:51.718 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 msgr2=0x7f8bb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 0x7f8bb007b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 msgr2=0x7f8bd414dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00ca20 tx=0x7f8bcc00cee0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 shutdown_connections 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8bb0078b90 0x7f8bb007b0a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd41533b0 0x7f8bd414dc70 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8bd40b5640 0x7f8bd4155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 --2- 172.21.15.37:0/1750195360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bd4072b90 0x7f8bd4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 >> 172.21.15.37:0/1750195360 conn(0x7f8bd40ccd70 msgr2=0x7f8bd406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:51.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 shutdown_connections 2024-09-21T06:32:51.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.718+0000 7f8bd9200640 1 -- 172.21.15.37:0/1750195360 wait complete. 2024-09-21T06:32:51.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 -- 172.21.15.37:0/1849727451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a006ddf0 msgr2=0x7f42a006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 --2- 172.21.15.37:0/1849727451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a006ddf0 0x7f42a006e200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f428c005fd0 tx=0x7f428c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 -- 172.21.15.37:0/1849727451 shutdown_connections 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 --2- 172.21.15.37:0/1849727451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a006ddf0 0x7f42a006e200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 --2- 172.21.15.37:0/1849727451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a00ceb20 0x7f42a0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 -- 172.21.15.37:0/1849727451 >> 172.21.15.37:0/1849727451 conn(0x7f42a0073400 msgr2=0x7f42a0073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 -- 172.21.15.37:0/1849727451 shutdown_connections 2024-09-21T06:32:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.969+0000 7f42a7000640 1 -- 172.21.15.37:0/1849727451 wait complete. 2024-09-21T06:32:51.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.972+0000 7f42a7000640 1 Processor -- start 2024-09-21T06:32:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.972+0000 7f42a7000640 1 -- start start 2024-09-21T06:32:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a006ddf0 0x7f42a014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 0x7f42a00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42a00ae8d0 con 0x7f42a00addb0 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f42a00cc7e0 con 0x7f42a00ceb20 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a7000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f42a00aea50 con 0x7f42a006ddf0 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a5200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a5200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45378/0 (socket says 172.21.15.37:45378) 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a5200640 1 -- 172.21.15.37:0/160292247 learned_addr learned my addr 172.21.15.37:0/160292247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.973+0000 7f42a5c00640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a006ddf0 0x7f42a014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a6600640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 0x7f42a00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 -- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a006ddf0 msgr2=0x7f42a014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a006ddf0 0x7f42a014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 -- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 msgr2=0x7f42a00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 0x7f42a00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 -- 172.21.15.37:0/160292247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a015f6d0 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a6600640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 0x7f42a00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f42a5200640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f428c034040 tx=0x7f428c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.974+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c03e030 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.975+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f428c003ae0 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.975+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c035040 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.975+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a015f9b0 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.975+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a0154b50 con 0x7f42a00addb0 2024-09-21T06:32:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.976+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a006e200 con 0x7f42a00addb0 2024-09-21T06:32:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.977+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f428c041030 con 0x7f42a00addb0 2024-09-21T06:32:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.978+0000 7f429d600640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 0x7f428007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.978+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f428c0bb400 con 0x7f42a00addb0 2024-09-21T06:32:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.978+0000 7f42a5c00640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 0x7f428007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:51.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.979+0000 7f42a5c00640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 0x7f428007b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4294009b30 tx=0x7f4294005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:51.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:51.982+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f428c083330 con 0x7f42a00addb0 2024-09-21T06:32:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.254+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7f42a0154de0 con 0x7f42a00addb0 2024-09-21T06:32:52.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.778+0000 7f429d600640 1 -- 172.21.15.37:0/160292247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v264) ==== 144+0+0 (secure 0 0 0) 0x7f428c087d60 con 0x7f42a00addb0 2024-09-21T06:32:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 msgr2=0x7f428007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 0x7f428007b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4294009b30 tx=0x7f4294005f90 comp rx=0 tx=0).stop 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 msgr2=0x7f42a00b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f428c034040 tx=0x7f428c0035f0 comp rx=0 tx=0).stop 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 shutdown_connections 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4280078b90 0x7f428007b0a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42a00addb0 0x7f42a00b5750 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a00ceb20 0x7f42a00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 --2- 172.21.15.37:0/160292247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42a006ddf0 0x7f42a014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 >> 172.21.15.37:0/160292247 conn(0x7f42a0073400 msgr2=0x7f42a00cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 shutdown_connections 2024-09-21T06:32:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:52.781+0000 7f42a7000640 1 -- 172.21.15.37:0/160292247 wait complete. 2024-09-21T06:32:52.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.039+0000 7fcc5d600640 1 -- 172.21.15.37:0/363183749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58073dd0 msgr2=0x7fcc5806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.039+0000 7fcc5d600640 1 --2- 172.21.15.37:0/363183749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58073dd0 0x7fcc5806ddf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009860 tx=0x7fcc4002a5a0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.039+0000 7fcc5d600640 1 -- 172.21.15.37:0/363183749 shutdown_connections 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.039+0000 7fcc5d600640 1 --2- 172.21.15.37:0/363183749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58073dd0 0x7fcc5806ddf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.039+0000 7fcc5d600640 1 -- 172.21.15.37:0/363183749 >> 172.21.15.37:0/363183749 conn(0x7fcc580cca40 msgr2=0x7fcc580cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.040+0000 7fcc5d600640 1 -- 172.21.15.37:0/363183749 shutdown_connections 2024-09-21T06:32:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.040+0000 7fcc5d600640 1 -- 172.21.15.37:0/363183749 wait complete. 2024-09-21T06:32:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.042+0000 7fcc5d600640 1 Processor -- start 2024-09-21T06:32:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.042+0000 7fcc5d600640 1 -- start start 2024-09-21T06:32:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 0x7fcc580b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58074cb0 0x7fcc581f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5806f670 con 0x7fcc58073dd0 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5806f4f0 con 0x7fcc5806e330 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.043+0000 7fcc5d600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5806f7f0 con 0x7fcc58074cb0 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45392/0 (socket says 172.21.15.37:45392) 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 -- 172.21.15.37:0/644357414 learned_addr learned my addr 172.21.15.37:0/644357414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc57400640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58074cb0 0x7fcc581f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc56a00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 0x7fcc580b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 -- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58074cb0 msgr2=0x7fcc581f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58074cb0 0x7fcc581f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 -- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 msgr2=0x7fcc580b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 0x7fcc580b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 -- 172.21.15.37:0/644357414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58157d30 con 0x7fcc58073dd0 2024-09-21T06:32:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc56a00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 0x7fcc580b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.044+0000 7fcc4fe00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009980 tx=0x7fcc40033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:53.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.045+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40031b50 con 0x7fcc58073dd0 2024-09-21T06:32:53.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.045+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc40031de0 con 0x7fcc58073dd0 2024-09-21T06:32:53.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.045+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40033330 con 0x7fcc58073dd0 2024-09-21T06:32:53.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.045+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc58158010 con 0x7fcc58073dd0 2024-09-21T06:32:53.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.045+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc58156210 con 0x7fcc58073dd0 2024-09-21T06:32:53.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.046+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58155d40 con 0x7fcc58073dd0 2024-09-21T06:32:53.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.047+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc400334d0 con 0x7fcc58073dd0 2024-09-21T06:32:53.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.048+0000 7fcc4ea00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.048+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fcc400bbd20 con 0x7fcc58073dd0 2024-09-21T06:32:53.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.049+0000 7fcc56a00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.050+0000 7fcc56a00640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 0x7fcc3407b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc5815a730 tx=0x7fcc48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.053+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc4004a030 con 0x7fcc58073dd0 2024-09-21T06:32:53.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.253+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fcc5806ddf0 con 0x7fcc58073dd0 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.254+0000 7fcc4ea00640 1 -- 172.21.15.37:0/644357414 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7fcc4003f030 con 0x7fcc58073dd0 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:32:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:32:53.257 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:32:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:32:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 msgr2=0x7fcc3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 0x7fcc3407b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc5815a730 tx=0x7fcc48009040 comp rx=0 tx=0).stop 2024-09-21T06:32:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 msgr2=0x7fcc580b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009980 tx=0x7fcc40033ba0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 shutdown_connections 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58073dd0 0x7fcc580b0c90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc58074cb0 0x7fcc581f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 --2- 172.21.15.37:0/644357414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5806e330 0x7fcc580b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 >> 172.21.15.37:0/644357414 conn(0x7fcc580cca40 msgr2=0x7fcc58070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 shutdown_connections 2024-09-21T06:32:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.257+0000 7fcc5d600640 1 -- 172.21.15.37:0/644357414 wait complete. 2024-09-21T06:32:53.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 -- 172.21.15.37:0/3785295649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0ccfc0 msgr2=0x7f4bec0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 --2- 172.21.15.37:0/3785295649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0ccfc0 0x7f4bec0b4f90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005fd0 tx=0x7f4bd802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 -- 172.21.15.37:0/3785295649 shutdown_connections 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 --2- 172.21.15.37:0/3785295649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec156c60 0x7f4bec0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 --2- 172.21.15.37:0/3785295649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0ccfc0 0x7f4bec0b4f90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 -- 172.21.15.37:0/3785295649 >> 172.21.15.37:0/3785295649 conn(0x7f4bec06f920 msgr2=0x7f4bec0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 -- 172.21.15.37:0/3785295649 shutdown_connections 2024-09-21T06:32:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.510+0000 7f4bf2400640 1 -- 172.21.15.37:0/3785295649 wait complete. 2024-09-21T06:32:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.512+0000 7f4bf2400640 1 Processor -- start 2024-09-21T06:32:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.513+0000 7f4bf2400640 1 -- start start 2024-09-21T06:32:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 0x7f4bec0c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 0x7f4bec0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec06d610 con 0x7f4bec156c60 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec06d490 con 0x7f4bec0cb4d0 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bf2400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec06d790 con 0x7f4bec06e370 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45410/0 (socket says 172.21.15.37:45410) 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.514+0000 7f4bebe00640 1 -- 172.21.15.37:0/808349896 learned_addr learned my addr 172.21.15.37:0/808349896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf1a00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 0x7f4bec0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 -- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 msgr2=0x7f4bec0c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf1000640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 0x7f4bec0c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 0x7f4bec0c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 -- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 msgr2=0x7f4bec0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 0x7f4bec0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 -- 172.21.15.37:0/808349896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0cbe20 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf1a00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 0x7f4bec0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf1000640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 0x7f4bec0c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bebe00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8034040 tx=0x7f4bd80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd803e030 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd8003ca0 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8035040 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec0c9bc0 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.515+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec0ca180 con 0x7f4bec156c60 2024-09-21T06:32:53.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.516+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec06dcb0 con 0x7f4bec156c60 2024-09-21T06:32:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.518+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4bd8041030 con 0x7f4bec156c60 2024-09-21T06:32:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.519+0000 7f4be8c00640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 0x7f4bcc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.519+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f4bd80bb240 con 0x7f4bec156c60 2024-09-21T06:32:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.519+0000 7f4bf1000640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 0x7f4bcc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.520+0000 7f4bf1000640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 0x7f4bcc07b0a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06eb80 tx=0x7f4be0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.523+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4bec0cbe20 con 0x7f4bec156c60 2024-09-21T06:32:53.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.719+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4bec156270 con 0x7f4bec156c60 2024-09-21T06:32:53.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.723+0000 7f4be8c00640 1 -- 172.21.15.37:0/808349896 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f4bd80837c0 con 0x7f4bec156c60 2024-09-21T06:32:53.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 msgr2=0x7f4bcc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 0x7f4bcc07b0a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06eb80 tx=0x7f4be0009040 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 msgr2=0x7f4bec0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8034040 tx=0x7f4bd80038a0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 shutdown_connections 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bcc078b90 0x7f4bcc07b0a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec156c60 0x7f4bec0b1060 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec0cb4d0 0x7f4bec0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 --2- 172.21.15.37:0/808349896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec06e370 0x7f4bec0c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 >> 172.21.15.37:0/808349896 conn(0x7f4bec06f920 msgr2=0x7f4bec15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 shutdown_connections 2024-09-21T06:32:53.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:53.726+0000 7f4bf2400640 1 -- 172.21.15.37:0/808349896 wait complete. 2024-09-21T06:32:53.753 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:32:53.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:32:53.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:32:53.924 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:32:53.924 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:32:53.924 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:53.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:32:53.960 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:32:53.960 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:53.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:32:53.998 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:32:53.998 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:54.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:32:54.035 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:32:54.035 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:32:54.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:32:54.076 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:32:54.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 -- 172.21.15.37:0/3754352782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de015ca00 msgr2=0x7f1de00afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/3754352782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de015ca00 0x7f1de00afb80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8005fd0 tx=0x7f1dd802aac0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 -- 172.21.15.37:0/3754352782 shutdown_connections 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/3754352782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de015ca00 0x7f1de00afb80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 -- 172.21.15.37:0/3754352782 >> 172.21.15.37:0/3754352782 conn(0x7f1de0073400 msgr2=0x7f1de0073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.301+0000 7f1ddfe00640 1 -- 172.21.15.37:0/3754352782 shutdown_connections 2024-09-21T06:32:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.302+0000 7f1ddfe00640 1 -- 172.21.15.37:0/3754352782 wait complete. 2024-09-21T06:32:54.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.304+0000 7f1ddfe00640 1 Processor -- start 2024-09-21T06:32:54.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.304+0000 7f1ddfe00640 1 -- start start 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0073f20 0x7f1de0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1de015ca00 0x7f1de00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1de00cf640 con 0x7f1de00c8b10 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1de00cf4c0 con 0x7f1de0073f20 2024-09-21T06:32:54.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.305+0000 7f1ddfe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1de00cf7c0 con 0x7f1de015ca00 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42476/0 (socket says 172.21.15.37:42476) 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 -- 172.21.15.37:0/164051402 learned_addr learned my addr 172.21.15.37:0/164051402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddea00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0073f20 0x7f1de0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1dd7e00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1de015ca00 0x7f1de00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 -- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1de015ca00 msgr2=0x7f1de00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1de015ca00 0x7f1de00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 -- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0073f20 msgr2=0x7f1de0159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0073f20 0x7f1de0159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 -- 172.21.15.37:0/164051402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de00cbd80 con 0x7f1de00c8b10 2024-09-21T06:32:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.306+0000 7f1ddf400640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009d90 tx=0x7f1dd8030e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.307+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8046070 con 0x7f1de00c8b10 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.307+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd8004560 con 0x7f1de00c8b10 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.307+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8033070 con 0x7f1de00c8b10 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.307+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de00cc060 con 0x7f1de00c8b10 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.307+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de00cc620 con 0x7f1de00c8b10 2024-09-21T06:32:54.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.308+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de00afb80 con 0x7f1de00c8b10 2024-09-21T06:32:54.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.309+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1dd8049070 con 0x7f1de00c8b10 2024-09-21T06:32:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.310+0000 7f1dd6a00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 0x7f1dbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.311+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f1dd80bbb40 con 0x7f1de00c8b10 2024-09-21T06:32:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.311+0000 7f1ddea00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 0x7f1dbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.312+0000 7f1ddea00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 0x7f1dbc07b0a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0001120 tx=0x7f1dd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:54.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.315+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1dd803e070 con 0x7f1de00c8b10 2024-09-21T06:32:54.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.552+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de00b1f10 con 0x7f1de00c8b10 2024-09-21T06:32:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.553+0000 7f1dd6a00640 1 -- 172.21.15.37:0/164051402 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+3917 (secure 0 0 0) 0x7f1dd8083950 con 0x7f1de00c8b10 2024-09-21T06:32:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:54.555 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":65,"btime":"2024-09-21T06:32:46:746975+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3829852426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3829852426},{"type":"v1","addr":"172.21.15.37:6834","nonce":3829852426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1126548826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1126548826},{"type":"v1","addr":"172.21.15.150:6836","nonce":1126548826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3599963281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3599963281},{"type":"v1","addr":"172.21.15.150:6837","nonce":3599963281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2239924416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":2239924416},{"type":"v1","addr":"172.21.15.37:6835","nonce":2239924416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[]} 2024-09-21T06:32:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 65 2024-09-21T06:32:54.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 msgr2=0x7f1dbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 0x7f1dbc07b0a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0001120 tx=0x7f1dd0009040 comp rx=0 tx=0).stop 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 msgr2=0x7f1de00c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009d90 tx=0x7f1dd8030e40 comp rx=0 tx=0).stop 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 shutdown_connections 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1dbc078b90 0x7f1dbc07b0a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1de00c8b10 0x7f1de00c8f20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1de015ca00 0x7f1de00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 --2- 172.21.15.37:0/164051402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0073f20 0x7f1de0159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 >> 172.21.15.37:0/164051402 conn(0x7f1de0073400 msgr2=0x7f1de00b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 shutdown_connections 2024-09-21T06:32:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.555+0000 7f1ddfe00640 1 -- 172.21.15.37:0/164051402 wait complete. 2024-09-21T06:32:54.579 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:32:54.579 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:32:54.580 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:32:54.580 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:32:54.580 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:32:54.580 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:32:54.580 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:32:54.580 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:32:54.580 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:32:54.581 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:32:54.581 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:32:54.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:32:54.581 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:32:54.581 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:32:54.581 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:32:54.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:32:54.603 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:32:54.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:32:54.606 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:32:54.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:32:54.616 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:32:54.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:32:54.617 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:32:54.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:32:54.649 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:32:54.650 DEBUG:teuthology.parallel:result is None 2024-09-21T06:32:54.652 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:32:54.653 DEBUG:teuthology.parallel:result is None 2024-09-21T06:32:54.660 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:32:54.661 DEBUG:teuthology.parallel:result is None 2024-09-21T06:32:54.663 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:32:54.663 DEBUG:teuthology.parallel:result is None 2024-09-21T06:32:54.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:32:54.845 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:32:54.848 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:32:54.857 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:32:54.859 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.895+0000 7f771e200640 1 -- 172.21.15.37:0/4061461521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718162770 msgr2=0x7f7718154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.895+0000 7f771e200640 1 --2- 172.21.15.37:0/4061461521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718162770 0x7f7718154ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7710005fd0 tx=0x7f771002aa80 comp rx=0 tx=0).stop 2024-09-21T06:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 -- 172.21.15.37:0/4061461521 shutdown_connections 2024-09-21T06:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 --2- 172.21.15.37:0/4061461521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718162770 0x7f7718154ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 --2- 172.21.15.37:0/4061461521 >> v2:172.21.15.150:3300/0 conn(0x7f77180b9dd0 0x7f7718162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 -- 172.21.15.37:0/4061461521 >> 172.21.15.37:0/4061461521 conn(0x7f77180d42e0 msgr2=0x7f77180d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 -- 172.21.15.37:0/4061461521 shutdown_connections 2024-09-21T06:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.896+0000 7f771e200640 1 -- 172.21.15.37:0/4061461521 wait complete. 2024-09-21T06:32:54.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.898+0000 7f771e200640 1 Processor -- start 2024-09-21T06:32:54.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.898+0000 7f771e200640 1 -- start start 2024-09-21T06:32:54.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77180d2ba0 0x7f7718153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718162770 0x7f7718153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7718155770 con 0x7f77180b9dd0 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77181555f0 con 0x7f7718162770 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771e200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f77181558f0 con 0x7f77180d2ba0 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42516/0 (socket says 172.21.15.37:42516) 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 -- 172.21.15.37:0/146564815 learned_addr learned my addr 172.21.15.37:0/146564815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771d800640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718162770 0x7f7718153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f7717e00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77180d2ba0 0x7f7718153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 -- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77180d2ba0 msgr2=0x7f7718153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77180d2ba0 0x7f7718153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 -- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718162770 msgr2=0x7f7718153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718162770 0x7f7718153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:54.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.900+0000 7f771ce00640 1 -- 172.21.15.37:0/146564815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181f54b0 con 0x7f77180b9dd0 2024-09-21T06:32:54.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f771ce00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c760 tx=0x7f770c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:54.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0052d0 con 0x7f77180b9dd0 2024-09-21T06:32:54.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718154060 con 0x7f77180b9dd0 2024-09-21T06:32:54.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f770c005490 con 0x7f77180b9dd0 2024-09-21T06:32:54.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c003850 con 0x7f77180b9dd0 2024-09-21T06:32:54.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.901+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77181595e0 con 0x7f77180b9dd0 2024-09-21T06:32:54.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.902+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77180ce250 con 0x7f77180b9dd0 2024-09-21T06:32:54.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.904+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f770c0039f0 con 0x7f77180b9dd0 2024-09-21T06:32:54.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.905+0000 7f7714c00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 0x7f76f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:54.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.905+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f770c09a610 con 0x7f77180b9dd0 2024-09-21T06:32:54.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.905+0000 7f7717e00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 0x7f76f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:54.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.906+0000 7f7717e00640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 0x7f76f807b0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f771815bb50 tx=0x7f7704005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:54.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:54.909+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77180cfef0 con 0x7f77180b9dd0 2024-09-21T06:32:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.118+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f77180b9790 con 0x7f77180b9dd0 2024-09-21T06:32:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.801+0000 7f7714c00640 1 -- 172.21.15.37:0/146564815 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7f770c062430 con 0x7f77180b9dd0 2024-09-21T06:32:55.804 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.805+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 msgr2=0x7f76f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.805+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 0x7f76f807b0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f771815bb50 tx=0x7f7704005f90 comp rx=0 tx=0).stop 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 msgr2=0x7f77181f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c760 tx=0x7f770c00cc20 comp rx=0 tx=0).stop 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 shutdown_connections 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078bc0 0x7f76f807b0d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718162770 0x7f7718153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77180d2ba0 0x7f7718153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 --2- 172.21.15.37:0/146564815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77180b9dd0 0x7f77181f4f70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 >> 172.21.15.37:0/146564815 conn(0x7f77180d42e0 msgr2=0x7f7718154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 shutdown_connections 2024-09-21T06:32:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:55.806+0000 7f771e200640 1 -- 172.21.15.37:0/146564815 wait complete. 2024-09-21T06:32:55.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:32:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 -- 172.21.15.37:0/3003575159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cbc10 msgr2=0x7f4eec0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 --2- 172.21.15.37:0/3003575159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cbc10 0x7f4eec0cc020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4eec076360 tx=0x7f4ed402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 -- 172.21.15.37:0/3003575159 shutdown_connections 2024-09-21T06:32:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 --2- 172.21.15.37:0/3003575159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cbc10 0x7f4eec0cc020 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 --2- 172.21.15.37:0/3003575159 >> v2:172.21.15.150:3300/0 conn(0x7f4eec074cb0 0x7f4eec073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 -- 172.21.15.37:0/3003575159 >> 172.21.15.37:0/3003575159 conn(0x7f4eec0cca40 msgr2=0x7f4eec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 -- 172.21.15.37:0/3003575159 shutdown_connections 2024-09-21T06:32:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.065+0000 7f4ef2000640 1 -- 172.21.15.37:0/3003575159 wait complete. 2024-09-21T06:32:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.067+0000 7f4ef2000640 1 Processor -- start 2024-09-21T06:32:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.068+0000 7f4ef2000640 1 -- start start 2024-09-21T06:32:56.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4eec074cb0 0x7f4eec0a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eec0cbc10 0x7f4eec09fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec06f360 con 0x7f4eec06ddf0 2024-09-21T06:32:56.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec06f1e0 con 0x7f4eec0cbc10 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ef2000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec06f4e0 con 0x7f4eec074cb0 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4eeb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4eeb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42536/0 (socket says 172.21.15.37:42536) 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4eeb400640 1 -- 172.21.15.37:0/4124993765 learned_addr learned my addr 172.21.15.37:0/4124993765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4ee3e00640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4eec074cb0 0x7f4eec0a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.069+0000 7f4eebe00640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eec0cbc10 0x7f4eec09fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 -- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4eec074cb0 msgr2=0x7f4eec0a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4eec074cb0 0x7f4eec0a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 -- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eec0cbc10 msgr2=0x7f4eec09fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eec0cbc10 0x7f4eec09fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 -- 172.21.15.37:0/4124993765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0cdf50 con 0x7f4eec06ddf0 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4eeb400640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e790 tx=0x7f4edc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00eea0 con 0x7f4eec06ddf0 2024-09-21T06:32:56.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4edc00ccc0 con 0x7f4eec06ddf0 2024-09-21T06:32:56.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004280 con 0x7f4eec06ddf0 2024-09-21T06:32:56.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.070+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec0ce230 con 0x7f4eec06ddf0 2024-09-21T06:32:56.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.071+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec09f300 con 0x7f4eec06ddf0 2024-09-21T06:32:56.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.072+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec06f990 con 0x7f4eec06ddf0 2024-09-21T06:32:56.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.073+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4edc0035f0 con 0x7f4eec06ddf0 2024-09-21T06:32:56.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.074+0000 7f4ee3400640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 0x7f4ec807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.074+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f4edc09b450 con 0x7f4eec06ddf0 2024-09-21T06:32:56.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.075+0000 7f4ee3e00640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 0x7f4ec807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.076+0000 7f4ee3e00640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 0x7f4ec807b0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4033000 tx=0x7f4ed4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:56.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.079+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4edc063360 con 0x7f4eec06ddf0 2024-09-21T06:32:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.348+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4eec15be20 con 0x7f4eec06ddf0 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.349+0000 7f4ee3400640 1 -- 172.21.15.37:0/4124993765 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f4edc067d90 con 0x7f4eec06ddf0 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:32:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:32:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:32:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:32:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:32:56.364 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:32:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 msgr2=0x7f4ec807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 0x7f4ec807b0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4033000 tx=0x7f4ed4004730 comp rx=0 tx=0).stop 2024-09-21T06:32:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 msgr2=0x7f4eec155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e790 tx=0x7f4edc00ec50 comp rx=0 tx=0).stop 2024-09-21T06:32:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 shutdown_connections 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4ec8078bc0 0x7f4ec807b0d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eec0cbc10 0x7f4eec09fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4eec074cb0 0x7f4eec0a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 --2- 172.21.15.37:0/4124993765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec06ddf0 0x7f4eec155e00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 >> 172.21.15.37:0/4124993765 conn(0x7f4eec0cca40 msgr2=0x7f4eec0cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 shutdown_connections 2024-09-21T06:32:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.351+0000 7f4ef2000640 1 -- 172.21.15.37:0/4124993765 wait complete. 2024-09-21T06:32:56.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 -- 172.21.15.37:0/3462502398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc074cb0 msgr2=0x7fcfcc073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3462502398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc074cb0 0x7fcfcc073dd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc09e110 tx=0x7fcfbc02a5f0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 -- 172.21.15.37:0/3462502398 shutdown_connections 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3462502398 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc0cbc10 0x7fcfcc0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3462502398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc074cb0 0x7fcfcc073dd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 -- 172.21.15.37:0/3462502398 >> 172.21.15.37:0/3462502398 conn(0x7fcfcc0cca40 msgr2=0x7fcfcc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 -- 172.21.15.37:0/3462502398 shutdown_connections 2024-09-21T06:32:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.600+0000 7fcfd1600640 1 -- 172.21.15.37:0/3462502398 wait complete. 2024-09-21T06:32:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.603+0000 7fcfd1600640 1 Processor -- start 2024-09-21T06:32:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.603+0000 7fcfd1600640 1 -- start start 2024-09-21T06:32:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc074cb0 0x7fcfcc157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc0cbc10 0x7fcfcc1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc06f360 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc06f1e0 con 0x7fcfcc0cbc10 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfd1600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcfcc06f4e0 con 0x7fcfcc074cb0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.604+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42550/0 (socket says 172.21.15.37:42550) 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 -- 172.21.15.37:0/3811889155 learned_addr learned my addr 172.21.15.37:0/3811889155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfca000640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc074cb0 0x7fcfcc157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcb400640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc0cbc10 0x7fcfcc1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 -- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc074cb0 msgr2=0x7fcfcc157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc074cb0 0x7fcfcc157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 -- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc0cbc10 msgr2=0x7fcfcc1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc0cbc10 0x7fcfcc1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 -- 172.21.15.37:0/3811889155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc151a10 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.605+0000 7fcfcaa00640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc000c00 tx=0x7fcfbc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.606+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc03e030 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.606+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfbc003e00 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.606+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc151cf0 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.606+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc035040 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.606+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc1522b0 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.607+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfcc15c580 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.608+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcfbc041030 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.609+0000 7fcfc2a00640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 0x7fcfac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:56.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.609+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7fcfbc0bb360 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.609+0000 7fcfca000640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 0x7fcfac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.610+0000 7fcfca000640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 0x7fcfac07b0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc154d60 tx=0x7fcfb800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.614+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcfcc15c580 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.817+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfcc06f990 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.818+0000 7fcfc2a00640 1 -- 172.21.15.37:0/3811889155 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fcfbc083270 con 0x7fcfcc06ddf0 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:32:56.820 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 msgr2=0x7fcfac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 0x7fcfac07b0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc154d60 tx=0x7fcfb800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 msgr2=0x7fcfcc1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc000c00 tx=0x7fcfbc003910 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 shutdown_connections 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcfac078bc0 0x7fcfac07b0d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc0cbc10 0x7fcfcc1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc074cb0 0x7fcfcc157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 --2- 172.21.15.37:0/3811889155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfcc06ddf0 0x7fcfcc1f4600 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 >> 172.21.15.37:0/3811889155 conn(0x7fcfcc0cca40 msgr2=0x7fcfcc070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 shutdown_connections 2024-09-21T06:32:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:56.820+0000 7fcfd1600640 1 -- 172.21.15.37:0/3811889155 wait complete. 2024-09-21T06:32:56.844 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:32:56.844 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.074+0000 7f7af2200640 1 -- 172.21.15.37:0/1898667201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec15bcb0 msgr2=0x7f7aec15c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.074+0000 7f7af2200640 1 --2- 172.21.15.37:0/1898667201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec15bcb0 0x7f7aec15c0c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7aec071e00 tx=0x7f7ad802a560 comp rx=0 tx=0).stop 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 -- 172.21.15.37:0/1898667201 shutdown_connections 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 --2- 172.21.15.37:0/1898667201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec15c600 0x7f7aec1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 --2- 172.21.15.37:0/1898667201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec15bcb0 0x7f7aec15c0c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 --2- 172.21.15.37:0/1898667201 >> v2:172.21.15.150:3300/0 conn(0x7f7aec0adee0 0x7f7aec06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 -- 172.21.15.37:0/1898667201 >> 172.21.15.37:0/1898667201 conn(0x7f7aec0ccac0 msgr2=0x7f7aec0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 -- 172.21.15.37:0/1898667201 shutdown_connections 2024-09-21T06:32:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.075+0000 7f7af2200640 1 -- 172.21.15.37:0/1898667201 wait complete. 2024-09-21T06:32:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.077+0000 7f7af2200640 1 Processor -- start 2024-09-21T06:32:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.077+0000 7f7af2200640 1 -- start start 2024-09-21T06:32:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.078+0000 7f7af2200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.078+0000 7f7af2200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aec15bcb0 0x7f7aec159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7af2200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec09b3f0 0x7f7aec1f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7af2200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec162930 con 0x7f7aec0adee0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7af2200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec06e9c0 con 0x7f7aec15bcb0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7af2200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec162ab0 con 0x7f7aec09b3f0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42560/0 (socket says 172.21.15.37:42560) 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 -- 172.21.15.37:0/2674180641 learned_addr learned my addr 172.21.15.37:0/2674180641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aebe00640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec09b3f0 0x7f7aec1f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeaa00640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aec15bcb0 0x7f7aec159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 -- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec09b3f0 msgr2=0x7f7aec1f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec09b3f0 0x7f7aec1f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 -- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aec15bcb0 msgr2=0x7f7aec159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aec15bcb0 0x7f7aec159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.079+0000 7f7aeb400640 1 -- 172.21.15.37:0/2674180641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec09b030 con 0x7f7aec0adee0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7aeb400640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e740 tx=0x7f7adc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00ccc0 con 0x7f7aec0adee0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7adc00ee50 con 0x7f7aec0adee0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc010670 con 0x7f7aec0adee0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec09ad50 con 0x7f7aec0adee0 2024-09-21T06:32:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.080+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1f54e0 con 0x7f7aec0adee0 2024-09-21T06:32:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.081+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec162fd0 con 0x7f7aec0adee0 2024-09-21T06:32:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.082+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7adc010810 con 0x7f7aec0adee0 2024-09-21T06:32:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.084+0000 7f7ae3400640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 0x7f7acc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.084+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f7adc09b4d0 con 0x7f7aec0adee0 2024-09-21T06:32:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.085+0000 7f7aeaa00640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 0x7f7acc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.085+0000 7f7aeaa00640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 0x7f7acc07b0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8033000 tx=0x7f7ad8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.089+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7adc0633e0 con 0x7f7aec0adee0 2024-09-21T06:32:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.287+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7aec0b0d00 con 0x7f7aec0adee0 2024-09-21T06:32:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.808+0000 7f7ae3400640 1 -- 172.21.15.37:0/2674180641 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f7adc067e10 con 0x7f7aec0adee0 2024-09-21T06:32:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.810+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 msgr2=0x7f7acc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.810+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 0x7f7acc07b0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8033000 tx=0x7f7ad8004730 comp rx=0 tx=0).stop 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.810+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 msgr2=0x7f7aec1f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.810+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e740 tx=0x7f7adc00ec00 comp rx=0 tx=0).stop 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 shutdown_connections 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7acc078bc0 0x7f7acc07b0d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7aec09b3f0 0x7f7aec1f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aec15bcb0 0x7f7aec159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 --2- 172.21.15.37:0/2674180641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec0adee0 0x7f7aec1f5ce0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 >> 172.21.15.37:0/2674180641 conn(0x7f7aec0ccac0 msgr2=0x7f7aec0b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 shutdown_connections 2024-09-21T06:32:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:57.811+0000 7f7af2200640 1 -- 172.21.15.37:0/2674180641 wait complete. 2024-09-21T06:32:57.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 -- 172.21.15.37:0/800510633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 msgr2=0x7f18640cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 --2- 172.21.15.37:0/800510633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18640cddc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f18640b2c90 tx=0x7f185402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 -- 172.21.15.37:0/800510633 shutdown_connections 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 --2- 172.21.15.37:0/800510633 >> v2:172.21.15.150:3300/0 conn(0x7f18640b4a20 0x7f18640b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 --2- 172.21.15.37:0/800510633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18640cddc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 -- 172.21.15.37:0/800510633 >> 172.21.15.37:0/800510633 conn(0x7f186406f940 msgr2=0x7f18640b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 -- 172.21.15.37:0/800510633 shutdown_connections 2024-09-21T06:32:58.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.076+0000 7f1869600640 1 -- 172.21.15.37:0/800510633 wait complete. 2024-09-21T06:32:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.079+0000 7f1869600640 1 Processor -- start 2024-09-21T06:32:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.079+0000 7f1869600640 1 -- start start 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18640b4a20 0x7f1864152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18640cc8d0 0x7f18641e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f186406d6c0 con 0x7f186415cb20 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f186406d540 con 0x7f18640cc8d0 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1869600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f186406d840 con 0x7f18640b4a20 2024-09-21T06:32:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.080+0000 7f1868c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1863e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18640b4a20 0x7f1864152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42568/0 (socket says 172.21.15.37:42568) 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 -- 172.21.15.37:0/3176724769 learned_addr learned my addr 172.21.15.37:0/3176724769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1863400640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18640cc8d0 0x7f18641e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 -- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18640b4a20 msgr2=0x7f1864152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18640b4a20 0x7f1864152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 -- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18640cc8d0 msgr2=0x7f18641e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18640cc8d0 0x7f18641e71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 -- 172.21.15.37:0/3176724769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18641f6030 con 0x7f186415cb20 2024-09-21T06:32:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.081+0000 7f1868c00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c6b0 tx=0x7f185c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.082+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c003b70 con 0x7f186415cb20 2024-09-21T06:32:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.082+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f185c003d30 con 0x7f186415cb20 2024-09-21T06:32:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.082+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c005330 con 0x7f186415cb20 2024-09-21T06:32:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.082+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18641f6310 con 0x7f186415cb20 2024-09-21T06:32:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.082+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1864152ec0 con 0x7f186415cb20 2024-09-21T06:32:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.083+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18641f5da0 con 0x7f186415cb20 2024-09-21T06:32:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.085+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f185c0054d0 con 0x7f186415cb20 2024-09-21T06:32:58.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.085+0000 7f185be00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.086+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5619+0+0 (secure 0 0 0) 0x7f185c09b420 con 0x7f186415cb20 2024-09-21T06:32:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.086+0000 7f1863e00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.087+0000 7f1863e00640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 0x7f184407b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1854033000 tx=0x7f1854004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.089+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f185c063240 con 0x7f186415cb20 2024-09-21T06:32:58.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.285+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f18641e5ee0 con 0x7f186415cb20 2024-09-21T06:32:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.815+0000 7f185be00640 1 -- 172.21.15.37:0/3176724769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f185c067c70 con 0x7f186415cb20 2024-09-21T06:32:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.818+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 msgr2=0x7f184407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.818+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 0x7f184407b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1854033000 tx=0x7f1854004730 comp rx=0 tx=0).stop 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.818+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 msgr2=0x7f18641e68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c6b0 tx=0x7f185c00cb70 comp rx=0 tx=0).stop 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 shutdown_connections 2024-09-21T06:32:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f186415cb20 0x7f18641e68f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18640cc8d0 0x7f18641e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 --2- 172.21.15.37:0/3176724769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18640b4a20 0x7f1864152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:58.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.819+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 >> 172.21.15.37:0/3176724769 conn(0x7f186406f940 msgr2=0x7f186415dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.820+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 shutdown_connections 2024-09-21T06:32:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:58.822+0000 7f1869600640 1 -- 172.21.15.37:0/3176724769 wait complete. 2024-09-21T06:32:58.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:32:59.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 -- 172.21.15.37:0/2045091735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f071815ee80 msgr2=0x7f07180d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:59.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 --2- 172.21.15.37:0/2045091735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f071815ee80 0x7f07180d0170 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f07180d06b0 tx=0x7f070802a4d0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 -- 172.21.15.37:0/2045091735 shutdown_connections 2024-09-21T06:32:59.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 --2- 172.21.15.37:0/2045091735 >> v2:172.21.15.150:3300/0 conn(0x7f07180b9ad0 0x7f07180d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 --2- 172.21.15.37:0/2045091735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f071815ee80 0x7f07180d0170 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 -- 172.21.15.37:0/2045091735 >> 172.21.15.37:0/2045091735 conn(0x7f071809d830 msgr2=0x7f07180b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 -- 172.21.15.37:0/2045091735 shutdown_connections 2024-09-21T06:32:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.094+0000 7f071f000640 1 -- 172.21.15.37:0/2045091735 wait complete. 2024-09-21T06:32:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.096+0000 7f071f000640 1 Processor -- start 2024-09-21T06:32:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.096+0000 7f071f000640 1 -- start start 2024-09-21T06:32:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07180b9ad0 0x7f07180b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f071815ee80 0x7f07180d3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07180b6700 con 0x7f07180d1120 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07180b6580 con 0x7f07180b9ad0 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071f000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f07180b6880 con 0x7f071815ee80 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f070fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f070fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42576/0 (socket says 172.21.15.37:42576) 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f070fe00640 1 -- 172.21.15.37:0/163763982 learned_addr learned my addr 172.21.15.37:0/163763982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f0717e00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07180b9ad0 0x7f07180b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.098+0000 7f071d000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f071815ee80 0x7f07180d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 -- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f071815ee80 msgr2=0x7f07180d3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f071815ee80 0x7f07180d3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 -- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07180b9ad0 msgr2=0x7f07180b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07180b9ad0 0x7f07180b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 -- 172.21.15.37:0/163763982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07180b2160 con 0x7f07180d1120 2024-09-21T06:32:59.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f070fe00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0700005ca0 tx=0x7f070000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:59.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700017070 con 0x7f07180d1120 2024-09-21T06:32:59.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0700004310 con 0x7f07180d1120 2024-09-21T06:32:59.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07180b2440 con 0x7f07180d1120 2024-09-21T06:32:59.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700009a20 con 0x7f07180d1120 2024-09-21T06:32:59.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.099+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07180b2a00 con 0x7f07180d1120 2024-09-21T06:32:59.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.100+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07180b1ed0 con 0x7f07180d1120 2024-09-21T06:32:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.102+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0700009bc0 con 0x7f07180d1120 2024-09-21T06:32:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.103+0000 7f0714c00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 0x7f06f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.103+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6001+0+0 (secure 0 0 0) 0x7f070009af30 con 0x7f07180d1120 2024-09-21T06:32:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.103+0000 7f0717e00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 0x7f06f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.104+0000 7f0717e00640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 0x7f06f407b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0708033000 tx=0x7f0708004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:59.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.108+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0700062ad0 con 0x7f07180d1120 2024-09-21T06:32:59.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.343+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f071806a6d0 con 0x7f07180d1120 2024-09-21T06:32:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.830+0000 7f0714c00640 1 -- 172.21.15.37:0/163763982 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v68) ==== 174+0+0 (secure 0 0 0) 0x7f071806a6d0 con 0x7f07180d1120 2024-09-21T06:32:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 11 and data pool 12 2024-09-21T06:32:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.835+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 msgr2=0x7f06f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.835+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 0x7f06f407b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0708033000 tx=0x7f0708004730 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.835+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 msgr2=0x7f07180d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.835+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0700005ca0 tx=0x7f070000dbb0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 shutdown_connections 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f06f4078b90 0x7f06f407b0a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07180d1120 0x7f07180d4410 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f071815ee80 0x7f07180d3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 --2- 172.21.15.37:0/163763982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07180b9ad0 0x7f07180b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.836+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 >> 172.21.15.37:0/163763982 conn(0x7f071809d830 msgr2=0x7f071815e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.837+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 shutdown_connections 2024-09-21T06:32:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:32:59.837+0000 7f071f000640 1 -- 172.21.15.37:0/163763982 wait complete. 2024-09-21T06:32:59.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 -- 172.21.15.37:0/3755543051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 msgr2=0x7f1cbc0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3755543051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc0ce320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005fd0 tx=0x7f1ca802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 -- 172.21.15.37:0/3755543051 shutdown_connections 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3755543051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc0ce320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3755543051 >> v2:172.21.15.150:3300/0 conn(0x7f1cbc15b9f0 0x7f1cbc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 -- 172.21.15.37:0/3755543051 >> 172.21.15.37:0/3755543051 conn(0x7f1cbc0cca40 msgr2=0x7f1cbc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 -- 172.21.15.37:0/3755543051 shutdown_connections 2024-09-21T06:33:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.108+0000 7f1cc3600640 1 -- 172.21.15.37:0/3755543051 wait complete. 2024-09-21T06:33:00.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.110+0000 7f1cc3600640 1 Processor -- start 2024-09-21T06:33:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.111+0000 7f1cc3600640 1 -- start start 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cbc0ce860 0x7f1cbc1f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc1e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06df70 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06ddf0 con 0x7f1cbc0ce860 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.112+0000 7f1cc3600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc06e0f0 con 0x7f1cbc0cefa0 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42584/0 (socket says 172.21.15.37:42584) 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 -- 172.21.15.37:0/3723936951 learned_addr learned my addr 172.21.15.37:0/3723936951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cbbe00640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc1e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc0c00640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cbc0ce860 0x7f1cbc1f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 -- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 msgr2=0x7f1cbc1e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc1e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 -- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cbc0ce860 msgr2=0x7f1cbc1f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cbc0ce860 0x7f1cbc1f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 -- 172.21.15.37:0/3723936951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc158c00 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.113+0000 7f1cc1600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c730 tx=0x7f1cb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.114+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4003b90 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.114+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb4003d50 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.114+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4005330 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.114+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc158ea0 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.114+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc0b5c40 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.115+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc06e5c0 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.116+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1cb40054d0 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.117+0000 7f1cb8c00640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 0x7f1c9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.117+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f1cb409a9d0 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.118+0000 7f1cc0c00640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 0x7f1c9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.118+0000 7f1cc0c00640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 0x7f1c9c07b0d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00095f0 tx=0x7f1cb0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.122+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1cb4062750 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.323+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cbc075050 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.324+0000 7f1cb8c00640 1 -- 172.21.15.37:0/3723936951 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16970 (secure 0 0 0) 0x7f1cb4067180 con 0x7f1cbc15b9f0 2024-09-21T06:33:00.327 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:00.327 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:32:59.345659+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:32:57.301028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T06:32:58.299171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:00.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 msgr2=0x7f1c9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 0x7f1c9c07b0d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00095f0 tx=0x7f1cb0005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:00.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 msgr2=0x7f1cbc09ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c730 tx=0x7f1cb400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 shutdown_connections 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1c9c078bc0 0x7f1c9c07b0d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cbc15b9f0 0x7f1cbc09ff60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cbc0cefa0 0x7f1cbc1e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 --2- 172.21.15.37:0/3723936951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cbc0ce860 0x7f1cbc1f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 >> 172.21.15.37:0/3723936951 conn(0x7f1cbc0cca40 msgr2=0x7f1cbc0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 shutdown_connections 2024-09-21T06:33:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.327+0000 7f1cc3600640 1 -- 172.21.15.37:0/3723936951 wait complete. 2024-09-21T06:33:00.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:00.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 -- 172.21.15.37:0/4181029642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf80003a00 msgr2=0x7fbf80003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 --2- 172.21.15.37:0/4181029642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf80003a00 0x7fbf80003e70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c4a0 tx=0x7fbf80005830 comp rx=0 tx=0).stop 2024-09-21T06:33:00.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa0c00640 1 -- 172.21.15.37:0/4181029642 reap_dead start 2024-09-21T06:33:00.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 -- 172.21.15.37:0/4181029642 shutdown_connections 2024-09-21T06:33:00.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 -- 172.21.15.37:0/4181029642 >> 172.21.15.37:0/4181029642 conn(0x7fbf9c0d0d40 msgr2=0x7fbf9c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:00.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 -- 172.21.15.37:0/4181029642 shutdown_connections 2024-09-21T06:33:00.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.580+0000 7fbfa1600640 1 -- 172.21.15.37:0/4181029642 wait complete. 2024-09-21T06:33:00.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.582+0000 7fbfa1600640 1 Processor -- start 2024-09-21T06:33:00.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.582+0000 7fbfa1600640 1 -- start start 2024-09-21T06:33:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf80003a00 0x7fbf9c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf9c0ccf60 0x7fbf9c1f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c162920 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1627a0 con 0x7fbf80003a00 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa1600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c0b7760 con 0x7fbf9c0ccf60 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42604/0 (socket says 172.21.15.37:42604) 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 -- 172.21.15.37:0/3710794073 learned_addr learned my addr 172.21.15.37:0/3710794073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbf9be00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf80003a00 0x7fbf9c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbf93400640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf9c0ccf60 0x7fbf9c1f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 -- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf9c0ccf60 msgr2=0x7fbf9c1f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf9c0ccf60 0x7fbf9c1f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 -- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf80003a00 msgr2=0x7fbf9c0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf80003a00 0x7fbf9c0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.584+0000 7fbfa0c00640 1 -- 172.21.15.37:0/3710794073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c1f8e30 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbfa0c00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c163160 tx=0x7fbf9400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94004b20 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf94004cc0 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94003bd0 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c0cd790 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.585+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c0cdd50 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.586+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9c0cd500 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.588+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf94004030 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.589+0000 7fbf98c00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 0x7fbf7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.589+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fbf9409b420 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.590+0000 7fbf9be00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 0x7fbf7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.590+0000 7fbf9be00640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 0x7fbf7807b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c033000 tx=0x7fbf8c038040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.594+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf940636c0 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.792+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf9c06ec60 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.793+0000 7fbf98c00640 1 -- 172.21.15.37:0/3710794073 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16970 (secure 0 0 0) 0x7fbf94062fc0 con 0x7fbf9c1f8a20 2024-09-21T06:33:00.796 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:00.796 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:32:59.345659+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:32:57.301028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T06:32:58.299171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 msgr2=0x7fbf7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 0x7fbf7807b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c033000 tx=0x7fbf8c038040 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 msgr2=0x7fbf9c0b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c163160 tx=0x7fbf9400f3c0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 shutdown_connections 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf78078b90 0x7fbf7807b0a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf9c1f8a20 0x7fbf9c0b5d70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf9c0ccf60 0x7fbf9c1f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 --2- 172.21.15.37:0/3710794073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf80003a00 0x7fbf9c0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 >> 172.21.15.37:0/3710794073 conn(0x7fbf9c0d0d40 msgr2=0x7fbf9c15fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 shutdown_connections 2024-09-21T06:33:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:00.796+0000 7fbfa1600640 1 -- 172.21.15.37:0/3710794073 wait complete. 2024-09-21T06:33:00.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.049+0000 7fbcd0c00640 1 -- 172.21.15.37:0/640266978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 msgr2=0x7fbccc06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.049+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/640266978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc06ddf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0b2c70 tx=0x7fbcb802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.049+0000 7fbcd0c00640 1 -- 172.21.15.37:0/640266978 shutdown_connections 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.049+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/640266978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc06ddf0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.049+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/640266978 >> v2:172.21.15.150:3300/0 conn(0x7fbccc156a50 0x7fbccc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.050+0000 7fbcd0c00640 1 -- 172.21.15.37:0/640266978 >> 172.21.15.37:0/640266978 conn(0x7fbccc0b1b90 msgr2=0x7fbccc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.050+0000 7fbcd0c00640 1 -- 172.21.15.37:0/640266978 shutdown_connections 2024-09-21T06:33:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.050+0000 7fbcd0c00640 1 -- 172.21.15.37:0/640266978 wait complete. 2024-09-21T06:33:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.052+0000 7fbcd0c00640 1 Processor -- start 2024-09-21T06:33:01.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.052+0000 7fbcd0c00640 1 -- start start 2024-09-21T06:33:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.053+0000 7fbcd0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbccc06e330 0x7fbccc0c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.053+0000 7fbcd0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.053+0000 7fbcd0c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbccc156a50 0x7fbccc0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcd0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0ce500 con 0x7fbccc074b60 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcd0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0ce380 con 0x7fbccc06e330 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcd0c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc0ce680 con 0x7fbccc156a50 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42622/0 (socket says 172.21.15.37:42622) 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 -- 172.21.15.37:0/4012466721 learned_addr learned my addr 172.21.15.37:0/4012466721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcca800640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbccc156a50 0x7fbccc0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9e00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbccc06e330 0x7fbccc0c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 -- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbccc156a50 msgr2=0x7fbccc0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbccc156a50 0x7fbccc0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 -- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbccc06e330 msgr2=0x7fbccc0c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbccc06e330 0x7fbccc0c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.054+0000 7fbcc9400640 1 -- 172.21.15.37:0/4012466721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc0c90b0 con 0x7fbccc074b60 2024-09-21T06:33:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcc9400640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8034040 tx=0x7fbcb8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb803e030 con 0x7fbccc074b60 2024-09-21T06:33:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb8003e00 con 0x7fbccc074b60 2024-09-21T06:33:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8035040 con 0x7fbccc074b60 2024-09-21T06:33:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc0c9390 con 0x7fbccc074b60 2024-09-21T06:33:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.055+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc0c9950 con 0x7fbccc074b60 2024-09-21T06:33:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.056+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc0b4b10 con 0x7fbccc074b60 2024-09-21T06:33:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.057+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcb8041030 con 0x7fbccc074b60 2024-09-21T06:33:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.059+0000 7fbcc2000640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.059+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fbcb80bb7a0 con 0x7fbccc074b60 2024-09-21T06:33:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.059+0000 7fbcc9e00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.060+0000 7fbcc9e00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 0x7fbcac07b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:01.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.064+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcb80839b0 con 0x7fbccc074b60 2024-09-21T06:33:01.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.300+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbccc06ddf0 con 0x7fbccc074b60 2024-09-21T06:33:01.810 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:33:01.808+0000 7fd256000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:33:01.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.846+0000 7fbcc2000640 1 -- 172.21.15.37:0/4012466721 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) ==== 118+0+0 (secure 0 0 0) 0x7fbcb80832b0 con 0x7fbccc074b60 2024-09-21T06:33:01.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.848+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 msgr2=0x7fbcac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.848+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 0x7fbcac07b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc005850 comp rx=0 tx=0).stop 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.848+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 msgr2=0x7fbccc0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.848+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8034040 tx=0x7fbcb8003910 comp rx=0 tx=0).stop 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 shutdown_connections 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbccc074b60 0x7fbccc0c8510 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbccc156a50 0x7fbccc0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 --2- 172.21.15.37:0/4012466721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbccc06e330 0x7fbccc0c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:01.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 >> 172.21.15.37:0/4012466721 conn(0x7fbccc0b1b90 msgr2=0x7fbccc0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:01.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 shutdown_connections 2024-09-21T06:33:01.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:01.849+0000 7fbcd0c00640 1 -- 172.21.15.37:0/4012466721 wait complete. 2024-09-21T06:33:01.878 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:33:01.878 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:33:02.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.111+0000 7fd0d3200640 1 -- 172.21.15.37:0/1877689713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b8004800 msgr2=0x7fd0b8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:02.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.111+0000 7fd0d3200640 1 --2- 172.21.15.37:0/1877689713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b8004800 0x7fd0b8004c10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009b50 tx=0x7fd0b8005830 comp rx=0 tx=0).stop 2024-09-21T06:33:02.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.112+0000 7fd0d2800640 1 -- 172.21.15.37:0/1877689713 reap_dead start 2024-09-21T06:33:02.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.112+0000 7fd0d3200640 1 -- 172.21.15.37:0/1877689713 shutdown_connections 2024-09-21T06:33:02.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.112+0000 7fd0d3200640 1 -- 172.21.15.37:0/1877689713 >> 172.21.15.37:0/1877689713 conn(0x7fd0d406f920 msgr2=0x7fd0d40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:02.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.112+0000 7fd0d3200640 1 -- 172.21.15.37:0/1877689713 shutdown_connections 2024-09-21T06:33:02.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.112+0000 7fd0d3200640 1 -- 172.21.15.37:0/1877689713 wait complete. 2024-09-21T06:33:02.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.114+0000 7fd0d3200640 1 Processor -- start 2024-09-21T06:33:02.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.114+0000 7fd0d3200640 1 -- start start 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b8004800 0x7fd0d40b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0d40b1750 0x7fd0d41f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b5110 con 0x7fd0d41f5a30 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b4f90 con 0x7fd0b8004800 2024-09-21T06:33:02.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d3200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d40b5290 con 0x7fd0d40b1750 2024-09-21T06:33:02.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:02.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42644/0 (socket says 172.21.15.37:42644) 2024-09-21T06:33:02.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 -- 172.21.15.37:0/2137154175 learned_addr learned my addr 172.21.15.37:0/2137154175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:02.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d1e00640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b8004800 0x7fd0d40b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:02.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0cbe00640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0d40b1750 0x7fd0d41f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 -- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0d40b1750 msgr2=0x7fd0d41f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0d40b1750 0x7fd0d41f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 -- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b8004800 msgr2=0x7fd0d40b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b8004800 0x7fd0d40b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.116+0000 7fd0d2800640 1 -- 172.21.15.37:0/2137154175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d40a95e0 con 0x7fd0d41f5a30 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0d2800640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40b36f0 tx=0x7fd0cc001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc011840 con 0x7fd0d41f5a30 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0cc018070 con 0x7fd0d41f5a30 2024-09-21T06:33:02.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc011c60 con 0x7fd0d41f5a30 2024-09-21T06:33:02.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d40a98c0 con 0x7fd0d41f5a30 2024-09-21T06:33:02.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.117+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d40a9e80 con 0x7fd0d41f5a30 2024-09-21T06:33:02.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.118+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d41f5fd0 con 0x7fd0d41f5a30 2024-09-21T06:33:02.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.120+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0cc01d460 con 0x7fd0d41f5a30 2024-09-21T06:33:02.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.120+0000 7fd0ca000640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 0x7fd0b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:02.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.121+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd0cc09b6b0 con 0x7fd0d41f5a30 2024-09-21T06:33:02.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.121+0000 7fd0d1e00640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 0x7fd0b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:02.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.122+0000 7fd0d1e00640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 0x7fd0b007b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006bd0 tx=0x7fd0c40067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:02.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.125+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0cc063270 con 0x7fd0d41f5a30 2024-09-21T06:33:02.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.361+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd0d4069450 con 0x7fd0d41f5a30 2024-09-21T06:33:02.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.856+0000 7fd0ca000640 1 -- 172.21.15.37:0/2137154175 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) ==== 122+0+0 (secure 0 0 0) 0x7fd0d4069450 con 0x7fd0d41f5a30 2024-09-21T06:33:02.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 msgr2=0x7fd0b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:02.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 0x7fd0b007b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006bd0 tx=0x7fd0c40067a0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 msgr2=0x7fd0d4146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:02.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40b36f0 tx=0x7fd0cc001bb0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 shutdown_connections 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0b0078b90 0x7fd0b007b0a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d41f5a30 0x7fd0d4146320 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0d40b1750 0x7fd0d41f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 --2- 172.21.15.37:0/2137154175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b8004800 0x7fd0d40b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 >> 172.21.15.37:0/2137154175 conn(0x7fd0d406f920 msgr2=0x7fd0d40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 shutdown_connections 2024-09-21T06:33:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:02.859+0000 7fd0d3200640 1 -- 172.21.15.37:0/2137154175 wait complete. 2024-09-21T06:33:02.888 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:33:02.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 -- 172.21.15.37:0/2527501627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede806e500 msgr2=0x7fede80cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 --2- 172.21.15.37:0/2527501627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede806e500 0x7fede80cc020 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fede8076360 tx=0x7fede002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 -- 172.21.15.37:0/2527501627 shutdown_connections 2024-09-21T06:33:03.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 --2- 172.21.15.37:0/2527501627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede806e500 0x7fede80cc020 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 --2- 172.21.15.37:0/2527501627 >> v2:172.21.15.150:3300/0 conn(0x7fede8074cb0 0x7fede8073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 -- 172.21.15.37:0/2527501627 >> 172.21.15.37:0/2527501627 conn(0x7fede80cca40 msgr2=0x7fede80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:03.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 -- 172.21.15.37:0/2527501627 shutdown_connections 2024-09-21T06:33:03.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.115+0000 7fedede00640 1 -- 172.21.15.37:0/2527501627 wait complete. 2024-09-21T06:33:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.117+0000 7fedede00640 1 Processor -- start 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.118+0000 7fedede00640 1 -- start start 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fede8074cb0 0x7fede81f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 0x7fede80c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fede806f360 con 0x7fede80cbc10 2024-09-21T06:33:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fede806f1e0 con 0x7fede81e5700 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedede00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fede806f4e0 con 0x7fede8074cb0 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedde000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedde000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42658/0 (socket says 172.21.15.37:42658) 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.119+0000 7fedde000640 1 -- 172.21.15.37:0/4093419842 learned_addr learned my addr 172.21.15.37:0/4093419842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fede6a00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fede8074cb0 0x7fede81f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fede7400640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 0x7fede80c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 -- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fede8074cb0 msgr2=0x7fede81f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fede8074cb0 0x7fede81f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 -- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 msgr2=0x7fede80c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 0x7fede80c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 -- 172.21.15.37:0/4093419842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede806d7b0 con 0x7fede80cbc10 2024-09-21T06:33:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fede7400640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 0x7fede80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedde000640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c370 tx=0x7fedd000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0011790 con 0x7fede80cbc10 2024-09-21T06:33:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedd0017070 con 0x7fede80cbc10 2024-09-21T06:33:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0011cc0 con 0x7fede80cbc10 2024-09-21T06:33:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.120+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81f9a90 con 0x7fede80cbc10 2024-09-21T06:33:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.121+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81f8940 con 0x7fede80cbc10 2024-09-21T06:33:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.122+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede806f990 con 0x7fede80cbc10 2024-09-21T06:33:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.123+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fedd001c460 con 0x7fede80cbc10 2024-09-21T06:33:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.124+0000 7feddfe00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 0x7fedc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.124+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fedd009abc0 con 0x7fede80cbc10 2024-09-21T06:33:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.124+0000 7fede6a00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 0x7fedc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.125+0000 7fede6a00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 0x7fedc407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fede815a730 tx=0x7fedd8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.129+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedd0062f00 con 0x7fede80cbc10 2024-09-21T06:33:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.364+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede806ddb0 con 0x7fede80cbc10 2024-09-21T06:33:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.365+0000 7feddfe00640 1 -- 172.21.15.37:0/4093419842 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5231 (secure 0 0 0) 0x7fedd0062800 con 0x7fede80cbc10 2024-09-21T06:33:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:03.368 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":72,"btime":"2024-09-21T06:33:02:854262+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6852,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:02.363468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870]},"id":5}]} 2024-09-21T06:33:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 72 2024-09-21T06:33:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 msgr2=0x7fedc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 0x7fedc407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fede815a730 tx=0x7fedd8005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 msgr2=0x7fede80c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c370 tx=0x7fedd000c830 comp rx=0 tx=0).stop 2024-09-21T06:33:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 shutdown_connections 2024-09-21T06:33:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fedc4078b90 0x7fedc407b0a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede80cbc10 0x7fede80c8d70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fede81e5700 0x7fede80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 --2- 172.21.15.37:0/4093419842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fede8074cb0 0x7fede81f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 >> 172.21.15.37:0/4093419842 conn(0x7fede80cca40 msgr2=0x7fede815ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 shutdown_connections 2024-09-21T06:33:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.367+0000 7fedede00640 1 -- 172.21.15.37:0/4093419842 wait complete. 2024-09-21T06:33:03.394 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:03.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 -- 172.21.15.37:0/1403525006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c073dd0 msgr2=0x7f439c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 --2- 172.21.15.37:0/1403525006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c073dd0 0x7f439c06ddf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4384009830 tx=0x7f438402a5c0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 -- 172.21.15.37:0/1403525006 shutdown_connections 2024-09-21T06:33:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 --2- 172.21.15.37:0/1403525006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c073dd0 0x7f439c06ddf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 -- 172.21.15.37:0/1403525006 >> 172.21.15.37:0/1403525006 conn(0x7f439c0cca40 msgr2=0x7f439c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 -- 172.21.15.37:0/1403525006 shutdown_connections 2024-09-21T06:33:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.619+0000 7f43a0c00640 1 -- 172.21.15.37:0/1403525006 wait complete. 2024-09-21T06:33:03.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.621+0000 7f43a0c00640 1 Processor -- start 2024-09-21T06:33:03.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.621+0000 7f43a0c00640 1 -- start start 2024-09-21T06:33:03.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f439c06e330 0x7f439c1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f439c073dd0 0x7f439c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f439c06f670 con 0x7f439c074cb0 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f439c06f4f0 con 0x7f439c06e330 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f43a0c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f439c06f7f0 con 0x7f439c073dd0 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42678/0 (socket says 172.21.15.37:42678) 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 -- 172.21.15.37:0/3938021185 learned_addr learned my addr 172.21.15.37:0/3938021185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f4399e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f439c06e330 0x7f439c1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f4391400640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f439c073dd0 0x7f439c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 -- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f439c073dd0 msgr2=0x7f439c158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f439c073dd0 0x7f439c158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 -- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f439c06e330 msgr2=0x7f439c1f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f439c06e330 0x7f439c1f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.623+0000 7f439a800640 1 -- 172.21.15.37:0/3938021185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c1f3290 con 0x7f439c074cb0 2024-09-21T06:33:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f439a800640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f439400c700 tx=0x7f439400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394003b70 con 0x7f439c074cb0 2024-09-21T06:33:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4394003d10 con 0x7f439c074cb0 2024-09-21T06:33:03.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400aab0 con 0x7f439c074cb0 2024-09-21T06:33:03.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c0b2f90 con 0x7f439c074cb0 2024-09-21T06:33:03.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.624+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c0b4c40 con 0x7f439c074cb0 2024-09-21T06:33:03.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.625+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c1f6630 con 0x7f439c074cb0 2024-09-21T06:33:03.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.627+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f439400ac50 con 0x7f439c074cb0 2024-09-21T06:33:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.628+0000 7f4392a00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.628+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f439409abe0 con 0x7f439c074cb0 2024-09-21T06:33:03.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.629+0000 7f4399e00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:03.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.629+0000 7f4399e00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 0x7f437807b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f438c000f60 tx=0x7f438c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:03.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.633+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f439c06e740 con 0x7f439c074cb0 2024-09-21T06:33:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.833+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f439c0b4a30 con 0x7f439c074cb0 2024-09-21T06:33:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.834+0000 7f4392a00640 1 -- 172.21.15.37:0/3938021185 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16957 (secure 0 0 0) 0x7f4394062840 con 0x7f439c074cb0 2024-09-21T06:33:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:03.837 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:00.819168+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:32:57.301028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T06:32:58.299171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.836+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 msgr2=0x7f437807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.836+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 0x7f437807b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f438c000f60 tx=0x7f438c009040 comp rx=0 tx=0).stop 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.836+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 msgr2=0x7f439c158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.836+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f439400c700 tx=0x7f439400cbc0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 shutdown_connections 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f439c074cb0 0x7f439c158c50 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f439c073dd0 0x7f439c158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 --2- 172.21.15.37:0/3938021185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f439c06e330 0x7f439c1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 >> 172.21.15.37:0/3938021185 conn(0x7f439c0cca40 msgr2=0x7f439c070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 shutdown_connections 2024-09-21T06:33:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:03.837+0000 7f43a0c00640 1 -- 172.21.15.37:0/3938021185 wait complete. 2024-09-21T06:33:03.861 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:33:03.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:33:04.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3517917648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415cb20 msgr2=0x7fd9d40cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3517917648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415cb20 0x7fd9d40cddc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009830 tx=0x7fd9c402a4a0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3517917648 shutdown_connections 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3517917648 >> v2:172.21.15.150:3300/0 conn(0x7fd9d40ccfe0 0x7fd9d40b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3517917648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415cb20 0x7fd9d40cddc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3517917648 >> 172.21.15.37:0/3517917648 conn(0x7fd9d406f940 msgr2=0x7fd9d40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3517917648 shutdown_connections 2024-09-21T06:33:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.088+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3517917648 wait complete. 2024-09-21T06:33:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.090+0000 7fd9d8e00640 1 Processor -- start 2024-09-21T06:33:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.090+0000 7fd9d8e00640 1 -- start start 2024-09-21T06:33:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d40b4a20 0x7fd9d41f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 0x7fd9d41f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d406d4b0 con 0x7fd9d415cb20 2024-09-21T06:33:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d406d330 con 0x7fd9d4155310 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d8e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d406d630 con 0x7fd9d40b4a20 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54510/0 (socket says 172.21.15.37:54510) 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9caa00640 1 -- 172.21.15.37:0/3400317524 learned_addr learned my addr 172.21.15.37:0/3400317524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d3400640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d40b4a20 0x7fd9d41f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.092+0000 7fd9d3e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 0x7fd9d41f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 -- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d40b4a20 msgr2=0x7fd9d41f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d40b4a20 0x7fd9d41f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 -- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 msgr2=0x7fd9d41f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 0x7fd9d41f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 -- 172.21.15.37:0/3400317524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d41f49c0 con 0x7fd9d415cb20 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9d3e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 0x7fd9d41f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9caa00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00c500 tx=0x7fd9bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc016020 con 0x7fd9d415cb20 2024-09-21T06:33:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9bc012940 con 0x7fd9d415cb20 2024-09-21T06:33:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc012d90 con 0x7fd9d415cb20 2024-09-21T06:33:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d4155af0 con 0x7fd9d415cb20 2024-09-21T06:33:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.093+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d41560b0 con 0x7fd9d415cb20 2024-09-21T06:33:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.094+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d406db80 con 0x7fd9d415cb20 2024-09-21T06:33:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.096+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9bc019020 con 0x7fd9d415cb20 2024-09-21T06:33:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.097+0000 7fd9cbe00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 0x7fd9b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.097+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd9bc09ac20 con 0x7fd9d415cb20 2024-09-21T06:33:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.097+0000 7fd9d3400640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 0x7fd9b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.098+0000 7fd9d3400640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 0x7fd9b007b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4034000 tx=0x7fd9c4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.101+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd9bc020030 con 0x7fd9d415cb20 2024-09-21T06:33:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.295+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9d4144450 con 0x7fd9b0078bc0 2024-09-21T06:33:04.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.300+0000 7fd9cbe00640 1 -- 172.21.15.37:0/3400317524 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7fd9c4004210 con 0x7fd9b0078bc0 2024-09-21T06:33:04.306 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 msgr2=0x7fd9b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 0x7fd9b007b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4034000 tx=0x7fd9c4004730 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 msgr2=0x7fd9d41f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00c500 tx=0x7fd9bc00c9c0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 shutdown_connections 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd9b0078bc0 0x7fd9b007b0d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415cb20 0x7fd9d41f4480 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9d4155310 0x7fd9d41f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 --2- 172.21.15.37:0/3400317524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d40b4a20 0x7fd9d41f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 >> 172.21.15.37:0/3400317524 conn(0x7fd9d406f940 msgr2=0x7fd9d40ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 shutdown_connections 2024-09-21T06:33:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.305+0000 7fd9d8e00640 1 -- 172.21.15.37:0/3400317524 wait complete. 2024-09-21T06:33:04.329 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-21T06:33:03.171920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258312,"kb_used_data":8780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716408,"statfs":{"total":773094113280,"available":772829601792,"internally_reserved":0,"allocated":8990720,"data_stored":4728735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.357267"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824683+0000","last_change":"2024-09-21T06:32:59.838263+0000","last_active":"2024-09-21T06:33:00.824683+0000","last_peered":"2024-09-21T06:33:00.824683+0000","last_clean":"2024-09-21T06:33:00.824683+0000","last_became_active":"2024-09-21T06:32:59.837943+0000","last_became_peered":"2024-09-21T06:32:59.837943+0000","last_unstale":"2024-09-21T06:33:00.824683+0000","last_undegraded":"2024-09-21T06:33:00.824683+0000","last_fullsized":"2024-09-21T06:33:00.824683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:34.295415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850302+0000","last_change":"2024-09-21T06:32:59.853583+0000","last_active":"2024-09-21T06:33:00.850302+0000","last_peered":"2024-09-21T06:33:00.850302+0000","last_clean":"2024-09-21T06:33:00.850302+0000","last_became_active":"2024-09-21T06:32:59.853204+0000","last_became_peered":"2024-09-21T06:32:59.853204+0000","last_unstale":"2024-09-21T06:33:00.850302+0000","last_undegraded":"2024-09-21T06:33:00.850302+0000","last_fullsized":"2024-09-21T06:33:00.850302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:34.568193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850092+0000","last_change":"2024-09-21T06:32:59.848913+0000","last_active":"2024-09-21T06:33:00.850092+0000","last_peered":"2024-09-21T06:33:00.850092+0000","last_clean":"2024-09-21T06:33:00.850092+0000","last_became_active":"2024-09-21T06:32:59.848556+0000","last_became_peered":"2024-09-21T06:32:59.848556+0000","last_unstale":"2024-09-21T06:33:00.850092+0000","last_undegraded":"2024-09-21T06:33:00.850092+0000","last_fullsized":"2024-09-21T06:33:00.850092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:38.985928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967391+0000","last_change":"2024-09-21T06:32:59.839766+0000","last_active":"2024-09-21T06:33:01.967391+0000","last_peered":"2024-09-21T06:33:01.967391+0000","last_clean":"2024-09-21T06:33:01.967391+0000","last_became_active":"2024-09-21T06:32:59.839187+0000","last_became_peered":"2024-09-21T06:32:59.839187+0000","last_unstale":"2024-09-21T06:33:01.967391+0000","last_undegraded":"2024-09-21T06:33:01.967391+0000","last_fullsized":"2024-09-21T06:33:01.967391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:06.137043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967016+0000","last_change":"2024-09-21T06:32:59.858950+0000","last_active":"2024-09-21T06:33:01.967016+0000","last_peered":"2024-09-21T06:33:01.967016+0000","last_clean":"2024-09-21T06:33:01.967016+0000","last_became_active":"2024-09-21T06:32:59.858604+0000","last_became_peered":"2024-09-21T06:32:59.858604+0000","last_unstale":"2024-09-21T06:33:01.967016+0000","last_undegraded":"2024-09-21T06:33:01.967016+0000","last_fullsized":"2024-09-21T06:33:01.967016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:53.682712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.842507+0000","last_change":"2024-09-21T06:32:59.838769+0000","last_active":"2024-09-21T06:32:59.842507+0000","last_peered":"2024-09-21T06:32:59.842507+0000","last_clean":"2024-09-21T06:32:59.842507+0000","last_became_active":"2024-09-21T06:32:59.833006+0000","last_became_peered":"2024-09-21T06:32:59.833006+0000","last_unstale":"2024-09-21T06:32:59.842507+0000","last_undegraded":"2024-09-21T06:32:59.842507+0000","last_fullsized":"2024-09-21T06:32:59.842507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:28.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966864+0000","last_change":"2024-09-21T06:32:59.851329+0000","last_active":"2024-09-21T06:33:01.966864+0000","last_peered":"2024-09-21T06:33:01.966864+0000","last_clean":"2024-09-21T06:33:01.966864+0000","last_became_active":"2024-09-21T06:32:59.850950+0000","last_became_peered":"2024-09-21T06:32:59.850950+0000","last_unstale":"2024-09-21T06:33:01.966864+0000","last_undegraded":"2024-09-21T06:33:01.966864+0000","last_fullsized":"2024-09-21T06:33:01.966864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.840820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826507+0000","last_change":"2024-09-21T06:32:59.847314+0000","last_active":"2024-09-21T06:33:00.826507+0000","last_peered":"2024-09-21T06:33:00.826507+0000","last_clean":"2024-09-21T06:33:00.826507+0000","last_became_active":"2024-09-21T06:32:59.846841+0000","last_became_peered":"2024-09-21T06:32:59.846841+0000","last_unstale":"2024-09-21T06:33:00.826507+0000","last_undegraded":"2024-09-21T06:33:00.826507+0000","last_fullsized":"2024-09-21T06:33:00.826507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:51.095219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967796+0000","last_change":"2024-09-21T06:32:59.857107+0000","last_active":"2024-09-21T06:33:01.967796+0000","last_peered":"2024-09-21T06:33:01.967796+0000","last_clean":"2024-09-21T06:33:01.967796+0000","last_became_active":"2024-09-21T06:32:59.856762+0000","last_became_peered":"2024-09-21T06:32:59.856762+0000","last_unstale":"2024-09-21T06:33:01.967796+0000","last_undegraded":"2024-09-21T06:33:01.967796+0000","last_fullsized":"2024-09-21T06:33:01.967796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.397998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824500+0000","last_change":"2024-09-21T06:32:59.846347+0000","last_active":"2024-09-21T06:33:00.824500+0000","last_peered":"2024-09-21T06:33:00.824500+0000","last_clean":"2024-09-21T06:33:00.824500+0000","last_became_active":"2024-09-21T06:32:59.846039+0000","last_became_peered":"2024-09-21T06:32:59.846039+0000","last_unstale":"2024-09-21T06:33:00.824500+0000","last_undegraded":"2024-09-21T06:33:00.824500+0000","last_fullsized":"2024-09-21T06:33:00.824500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:03.329349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850165+0000","last_change":"2024-09-21T06:32:59.853858+0000","last_active":"2024-09-21T06:33:00.850165+0000","last_peered":"2024-09-21T06:33:00.850165+0000","last_clean":"2024-09-21T06:33:00.850165+0000","last_became_active":"2024-09-21T06:32:59.853180+0000","last_became_peered":"2024-09-21T06:32:59.853180+0000","last_unstale":"2024-09-21T06:33:00.850165+0000","last_undegraded":"2024-09-21T06:33:00.850165+0000","last_fullsized":"2024-09-21T06:33:00.850165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.318029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967697+0000","last_change":"2024-09-21T06:32:59.843842+0000","last_active":"2024-09-21T06:33:01.967697+0000","last_peered":"2024-09-21T06:33:01.967697+0000","last_clean":"2024-09-21T06:33:01.967697+0000","last_became_active":"2024-09-21T06:32:59.843549+0000","last_became_peered":"2024-09-21T06:32:59.843549+0000","last_unstale":"2024-09-21T06:33:01.967697+0000","last_undegraded":"2024-09-21T06:33:01.967697+0000","last_fullsized":"2024-09-21T06:33:01.967697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:41.902877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850294+0000","last_change":"2024-09-21T06:32:59.862805+0000","last_active":"2024-09-21T06:33:00.850294+0000","last_peered":"2024-09-21T06:33:00.850294+0000","last_clean":"2024-09-21T06:33:00.850294+0000","last_became_active":"2024-09-21T06:32:59.862482+0000","last_became_peered":"2024-09-21T06:32:59.862482+0000","last_unstale":"2024-09-21T06:33:00.850294+0000","last_undegraded":"2024-09-21T06:33:00.850294+0000","last_fullsized":"2024-09-21T06:33:00.850294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.586721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827312+0000","last_change":"2024-09-21T06:32:59.836878+0000","last_active":"2024-09-21T06:33:00.827312+0000","last_peered":"2024-09-21T06:33:00.827312+0000","last_clean":"2024-09-21T06:33:00.827312+0000","last_became_active":"2024-09-21T06:32:59.836258+0000","last_became_peered":"2024-09-21T06:32:59.836258+0000","last_unstale":"2024-09-21T06:33:00.827312+0000","last_undegraded":"2024-09-21T06:33:00.827312+0000","last_fullsized":"2024-09-21T06:33:00.827312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:36.037339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966773+0000","last_change":"2024-09-21T06:32:59.866515+0000","last_active":"2024-09-21T06:33:01.966773+0000","last_peered":"2024-09-21T06:33:01.966773+0000","last_clean":"2024-09-21T06:33:01.966773+0000","last_became_active":"2024-09-21T06:32:59.865412+0000","last_became_peered":"2024-09-21T06:32:59.865412+0000","last_unstale":"2024-09-21T06:33:01.966773+0000","last_undegraded":"2024-09-21T06:33:01.966773+0000","last_fullsized":"2024-09-21T06:33:01.966773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:55.775749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966729+0000","last_change":"2024-09-21T06:32:59.847310+0000","last_active":"2024-09-21T06:33:01.966729+0000","last_peered":"2024-09-21T06:33:01.966729+0000","last_clean":"2024-09-21T06:33:01.966729+0000","last_became_active":"2024-09-21T06:32:59.846939+0000","last_became_peered":"2024-09-21T06:32:59.846939+0000","last_unstale":"2024-09-21T06:33:01.966729+0000","last_undegraded":"2024-09-21T06:33:01.966729+0000","last_fullsized":"2024-09-21T06:33:01.966729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827634+0000","last_change":"2024-09-21T06:32:59.847284+0000","last_active":"2024-09-21T06:33:00.827634+0000","last_peered":"2024-09-21T06:33:00.827634+0000","last_clean":"2024-09-21T06:33:00.827634+0000","last_became_active":"2024-09-21T06:32:59.846874+0000","last_became_peered":"2024-09-21T06:32:59.846874+0000","last_unstale":"2024-09-21T06:33:00.827634+0000","last_undegraded":"2024-09-21T06:33:00.827634+0000","last_fullsized":"2024-09-21T06:33:00.827634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:20.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850289+0000","last_change":"2024-09-21T06:32:59.853963+0000","last_active":"2024-09-21T06:33:00.850289+0000","last_peered":"2024-09-21T06:33:00.850289+0000","last_clean":"2024-09-21T06:33:00.850289+0000","last_became_active":"2024-09-21T06:32:59.853493+0000","last_became_peered":"2024-09-21T06:32:59.853493+0000","last_unstale":"2024-09-21T06:33:00.850289+0000","last_undegraded":"2024-09-21T06:33:00.850289+0000","last_fullsized":"2024-09-21T06:33:00.850289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:47.613122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827763+0000","last_change":"2024-09-21T06:32:59.860332+0000","last_active":"2024-09-21T06:33:00.827763+0000","last_peered":"2024-09-21T06:33:00.827763+0000","last_clean":"2024-09-21T06:33:00.827763+0000","last_became_active":"2024-09-21T06:32:59.860026+0000","last_became_peered":"2024-09-21T06:32:59.860026+0000","last_unstale":"2024-09-21T06:33:00.827763+0000","last_undegraded":"2024-09-21T06:33:00.827763+0000","last_fullsized":"2024-09-21T06:33:00.827763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:55.745827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.865066+0000","last_change":"2024-09-21T06:32:59.850614+0000","last_active":"2024-09-21T06:32:59.865066+0000","last_peered":"2024-09-21T06:32:59.865066+0000","last_clean":"2024-09-21T06:32:59.865066+0000","last_became_active":"2024-09-21T06:32:59.849535+0000","last_became_peered":"2024-09-21T06:32:59.849535+0000","last_unstale":"2024-09-21T06:32:59.865066+0000","last_undegraded":"2024-09-21T06:32:59.865066+0000","last_fullsized":"2024-09-21T06:32:59.865066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:45.152420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966743+0000","last_change":"2024-09-21T06:32:59.837970+0000","last_active":"2024-09-21T06:33:01.966743+0000","last_peered":"2024-09-21T06:33:01.966743+0000","last_clean":"2024-09-21T06:33:01.966743+0000","last_became_active":"2024-09-21T06:32:59.837545+0000","last_became_peered":"2024-09-21T06:32:59.837545+0000","last_unstale":"2024-09-21T06:33:01.966743+0000","last_undegraded":"2024-09-21T06:33:01.966743+0000","last_fullsized":"2024-09-21T06:33:01.966743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:34.568193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849964+0000","last_change":"2024-09-21T06:32:59.862915+0000","last_active":"2024-09-21T06:33:00.849964+0000","last_peered":"2024-09-21T06:33:00.849964+0000","last_clean":"2024-09-21T06:33:00.849964+0000","last_became_active":"2024-09-21T06:32:59.862703+0000","last_became_peered":"2024-09-21T06:32:59.862703+0000","last_unstale":"2024-09-21T06:33:00.849964+0000","last_undegraded":"2024-09-21T06:33:00.849964+0000","last_fullsized":"2024-09-21T06:33:00.849964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:55.330268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824208+0000","last_change":"2024-09-21T06:32:59.837066+0000","last_active":"2024-09-21T06:33:00.824208+0000","last_peered":"2024-09-21T06:33:00.824208+0000","last_clean":"2024-09-21T06:33:00.824208+0000","last_became_active":"2024-09-21T06:32:59.836756+0000","last_became_peered":"2024-09-21T06:32:59.836756+0000","last_unstale":"2024-09-21T06:33:00.824208+0000","last_undegraded":"2024-09-21T06:33:00.824208+0000","last_fullsized":"2024-09-21T06:33:00.824208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:28.921748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849452+0000","last_change":"2024-09-21T06:32:59.860983+0000","last_active":"2024-09-21T06:33:00.849452+0000","last_peered":"2024-09-21T06:33:00.849452+0000","last_clean":"2024-09-21T06:33:00.849452+0000","last_became_active":"2024-09-21T06:32:59.860635+0000","last_became_peered":"2024-09-21T06:32:59.860635+0000","last_unstale":"2024-09-21T06:33:00.849452+0000","last_undegraded":"2024-09-21T06:33:00.849452+0000","last_fullsized":"2024-09-21T06:33:00.849452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:35.945934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.864557+0000","last_change":"2024-09-21T06:32:59.847290+0000","last_active":"2024-09-21T06:32:59.864557+0000","last_peered":"2024-09-21T06:32:59.864557+0000","last_clean":"2024-09-21T06:32:59.864557+0000","last_became_active":"2024-09-21T06:32:59.846008+0000","last_became_peered":"2024-09-21T06:32:59.846008+0000","last_unstale":"2024-09-21T06:32:59.864557+0000","last_undegraded":"2024-09-21T06:32:59.864557+0000","last_fullsized":"2024-09-21T06:32:59.864557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:21.117853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967788+0000","last_change":"2024-09-21T06:32:59.858959+0000","last_active":"2024-09-21T06:33:01.967788+0000","last_peered":"2024-09-21T06:33:01.967788+0000","last_clean":"2024-09-21T06:33:01.967788+0000","last_became_active":"2024-09-21T06:32:59.858615+0000","last_became_peered":"2024-09-21T06:32:59.858615+0000","last_unstale":"2024-09-21T06:33:01.967788+0000","last_undegraded":"2024-09-21T06:33:01.967788+0000","last_fullsized":"2024-09-21T06:33:01.967788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:21.935781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967497+0000","last_change":"2024-09-21T06:32:59.842692+0000","last_active":"2024-09-21T06:33:01.967497+0000","last_peered":"2024-09-21T06:33:01.967497+0000","last_clean":"2024-09-21T06:33:01.967497+0000","last_became_active":"2024-09-21T06:32:59.842322+0000","last_became_peered":"2024-09-21T06:32:59.842322+0000","last_unstale":"2024-09-21T06:33:01.967497+0000","last_undegraded":"2024-09-21T06:33:01.967497+0000","last_fullsized":"2024-09-21T06:33:01.967497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:35.945934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966926+0000","last_change":"2024-09-21T06:32:59.870736+0000","last_active":"2024-09-21T06:33:01.966926+0000","last_peered":"2024-09-21T06:33:01.966926+0000","last_clean":"2024-09-21T06:33:01.966926+0000","last_became_active":"2024-09-21T06:32:59.870317+0000","last_became_peered":"2024-09-21T06:32:59.870317+0000","last_unstale":"2024-09-21T06:33:01.966926+0000","last_undegraded":"2024-09-21T06:33:01.966926+0000","last_fullsized":"2024-09-21T06:33:01.966926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:27.810982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.864429+0000","last_change":"2024-09-21T06:32:59.845738+0000","last_active":"2024-09-21T06:32:59.864429+0000","last_peered":"2024-09-21T06:32:59.864429+0000","last_clean":"2024-09-21T06:32:59.864429+0000","last_became_active":"2024-09-21T06:32:59.845369+0000","last_became_peered":"2024-09-21T06:32:59.845369+0000","last_unstale":"2024-09-21T06:32:59.864429+0000","last_undegraded":"2024-09-21T06:32:59.864429+0000","last_fullsized":"2024-09-21T06:32:59.864429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:52.816245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824211+0000","last_change":"2024-09-21T06:32:59.829555+0000","last_active":"2024-09-21T06:33:00.824211+0000","last_peered":"2024-09-21T06:33:00.824211+0000","last_clean":"2024-09-21T06:33:00.824211+0000","last_became_active":"2024-09-21T06:32:59.829233+0000","last_became_peered":"2024-09-21T06:32:59.829233+0000","last_unstale":"2024-09-21T06:33:00.824211+0000","last_undegraded":"2024-09-21T06:33:00.824211+0000","last_fullsized":"2024-09-21T06:33:00.824211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:50.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849791+0000","last_change":"2024-09-21T06:32:59.851727+0000","last_active":"2024-09-21T06:33:00.849791+0000","last_peered":"2024-09-21T06:33:00.849791+0000","last_clean":"2024-09-21T06:33:00.849791+0000","last_became_active":"2024-09-21T06:32:59.851023+0000","last_became_peered":"2024-09-21T06:32:59.851023+0000","last_unstale":"2024-09-21T06:33:00.849791+0000","last_undegraded":"2024-09-21T06:33:00.849791+0000","last_fullsized":"2024-09-21T06:33:00.849791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:59.690110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966950+0000","last_change":"2024-09-21T06:32:59.855643+0000","last_active":"2024-09-21T06:33:01.966950+0000","last_peered":"2024-09-21T06:33:01.966950+0000","last_clean":"2024-09-21T06:33:01.966950+0000","last_became_active":"2024-09-21T06:32:59.855128+0000","last_became_peered":"2024-09-21T06:32:59.855128+0000","last_unstale":"2024-09-21T06:33:01.966950+0000","last_undegraded":"2024-09-21T06:33:01.966950+0000","last_fullsized":"2024-09-21T06:33:01.966950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:03.329349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966694+0000","last_change":"2024-09-21T06:32:59.868986+0000","last_active":"2024-09-21T06:33:01.966694+0000","last_peered":"2024-09-21T06:33:01.966694+0000","last_clean":"2024-09-21T06:33:01.966694+0000","last_became_active":"2024-09-21T06:32:59.868255+0000","last_became_peered":"2024-09-21T06:32:59.868255+0000","last_unstale":"2024-09-21T06:33:01.966694+0000","last_undegraded":"2024-09-21T06:33:01.966694+0000","last_fullsized":"2024-09-21T06:33:01.966694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:17.234871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967027+0000","last_change":"2024-09-21T06:32:58.847274+0000","last_active":"2024-09-21T06:33:01.967027+0000","last_peered":"2024-09-21T06:33:01.967027+0000","last_clean":"2024-09-21T06:33:01.967027+0000","last_became_active":"2024-09-21T06:32:58.846939+0000","last_became_peered":"2024-09-21T06:32:58.846939+0000","last_unstale":"2024-09-21T06:33:01.967027+0000","last_undegraded":"2024-09-21T06:33:01.967027+0000","last_fullsized":"2024-09-21T06:33:01.967027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:06.394840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966732+0000","last_change":"2024-09-21T06:32:59.854664+0000","last_active":"2024-09-21T06:33:01.966732+0000","last_peered":"2024-09-21T06:33:01.966732+0000","last_clean":"2024-09-21T06:33:01.966732+0000","last_became_active":"2024-09-21T06:32:59.849095+0000","last_became_peered":"2024-09-21T06:32:59.849095+0000","last_unstale":"2024-09-21T06:33:01.966732+0000","last_undegraded":"2024-09-21T06:33:01.966732+0000","last_fullsized":"2024-09-21T06:33:01.966732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:26.470146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968145+0000","last_change":"2024-09-21T06:32:58.839708+0000","last_active":"2024-09-21T06:33:01.968145+0000","last_peered":"2024-09-21T06:33:01.968145+0000","last_clean":"2024-09-21T06:33:01.968145+0000","last_became_active":"2024-09-21T06:32:58.839261+0000","last_became_peered":"2024-09-21T06:32:58.839261+0000","last_unstale":"2024-09-21T06:33:01.968145+0000","last_undegraded":"2024-09-21T06:33:01.968145+0000","last_fullsized":"2024-09-21T06:33:01.968145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:21.684102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967108+0000","last_change":"2024-09-21T06:32:59.850512+0000","last_active":"2024-09-21T06:33:01.967108+0000","last_peered":"2024-09-21T06:33:01.967108+0000","last_clean":"2024-09-21T06:33:01.967108+0000","last_became_active":"2024-09-21T06:32:59.850066+0000","last_became_peered":"2024-09-21T06:32:59.850066+0000","last_unstale":"2024-09-21T06:33:01.967108+0000","last_undegraded":"2024-09-21T06:33:01.967108+0000","last_fullsized":"2024-09-21T06:33:01.967108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.771035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824133+0000","last_change":"2024-09-21T06:32:58.847346+0000","last_active":"2024-09-21T06:33:00.824133+0000","last_peered":"2024-09-21T06:33:00.824133+0000","last_clean":"2024-09-21T06:33:00.824133+0000","last_became_active":"2024-09-21T06:32:58.847025+0000","last_became_peered":"2024-09-21T06:32:58.847025+0000","last_unstale":"2024-09-21T06:33:00.824133+0000","last_undegraded":"2024-09-21T06:33:00.824133+0000","last_fullsized":"2024-09-21T06:33:00.824133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.028650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967393+0000","last_change":"2024-09-21T06:32:59.833499+0000","last_active":"2024-09-21T06:33:01.967393+0000","last_peered":"2024-09-21T06:33:01.967393+0000","last_clean":"2024-09-21T06:33:01.967393+0000","last_became_active":"2024-09-21T06:32:59.833236+0000","last_became_peered":"2024-09-21T06:32:59.833236+0000","last_unstale":"2024-09-21T06:33:01.967393+0000","last_undegraded":"2024-09-21T06:33:01.967393+0000","last_fullsized":"2024-09-21T06:33:01.967393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:55.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968336+0000","last_change":"2024-09-21T06:32:58.840659+0000","last_active":"2024-09-21T06:33:01.968336+0000","last_peered":"2024-09-21T06:33:01.968336+0000","last_clean":"2024-09-21T06:33:01.968336+0000","last_became_active":"2024-09-21T06:32:58.840411+0000","last_became_peered":"2024-09-21T06:32:58.840411+0000","last_unstale":"2024-09-21T06:33:01.968336+0000","last_undegraded":"2024-09-21T06:33:01.968336+0000","last_fullsized":"2024-09-21T06:33:01.968336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:54.846180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.865026+0000","last_change":"2024-09-21T06:32:59.850291+0000","last_active":"2024-09-21T06:32:59.865026+0000","last_peered":"2024-09-21T06:32:59.865026+0000","last_clean":"2024-09-21T06:32:59.865026+0000","last_became_active":"2024-09-21T06:32:59.849723+0000","last_became_peered":"2024-09-21T06:32:59.849723+0000","last_unstale":"2024-09-21T06:32:59.865026+0000","last_undegraded":"2024-09-21T06:32:59.865026+0000","last_fullsized":"2024-09-21T06:32:59.865026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:51.620455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966837+0000","last_change":"2024-09-21T06:32:58.839877+0000","last_active":"2024-09-21T06:33:01.966837+0000","last_peered":"2024-09-21T06:33:01.966837+0000","last_clean":"2024-09-21T06:33:01.966837+0000","last_became_active":"2024-09-21T06:32:58.839316+0000","last_became_peered":"2024-09-21T06:32:58.839316+0000","last_unstale":"2024-09-21T06:33:01.966837+0000","last_undegraded":"2024-09-21T06:33:01.966837+0000","last_fullsized":"2024-09-21T06:33:01.966837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:53.672997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850470+0000","last_change":"2024-09-21T06:32:59.841422+0000","last_active":"2024-09-21T06:33:00.850470+0000","last_peered":"2024-09-21T06:33:00.850470+0000","last_clean":"2024-09-21T06:33:00.850470+0000","last_became_active":"2024-09-21T06:32:59.841042+0000","last_became_peered":"2024-09-21T06:32:59.841042+0000","last_unstale":"2024-09-21T06:33:00.850470+0000","last_undegraded":"2024-09-21T06:33:00.850470+0000","last_fullsized":"2024-09-21T06:33:00.850470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.771035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967745+0000","last_change":"2024-09-21T06:32:58.844502+0000","last_active":"2024-09-21T06:33:01.967745+0000","last_peered":"2024-09-21T06:33:01.967745+0000","last_clean":"2024-09-21T06:33:01.967745+0000","last_became_active":"2024-09-21T06:32:58.843728+0000","last_became_peered":"2024-09-21T06:32:58.843728+0000","last_unstale":"2024-09-21T06:33:01.967745+0000","last_undegraded":"2024-09-21T06:33:01.967745+0000","last_fullsized":"2024-09-21T06:33:01.967745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:38.274905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968624+0000","last_change":"2024-09-21T06:32:59.865377+0000","last_active":"2024-09-21T06:33:01.968624+0000","last_peered":"2024-09-21T06:33:01.968624+0000","last_clean":"2024-09-21T06:33:01.968624+0000","last_became_active":"2024-09-21T06:32:59.864228+0000","last_became_peered":"2024-09-21T06:32:59.864228+0000","last_unstale":"2024-09-21T06:33:01.968624+0000","last_undegraded":"2024-09-21T06:33:01.968624+0000","last_fullsized":"2024-09-21T06:33:01.968624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:20.301779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826830+0000","last_change":"2024-09-21T06:32:58.847972+0000","last_active":"2024-09-21T06:33:00.826830+0000","last_peered":"2024-09-21T06:33:00.826830+0000","last_clean":"2024-09-21T06:33:00.826830+0000","last_became_active":"2024-09-21T06:32:58.847647+0000","last_became_peered":"2024-09-21T06:32:58.847647+0000","last_unstale":"2024-09-21T06:33:00.826830+0000","last_undegraded":"2024-09-21T06:33:00.826830+0000","last_fullsized":"2024-09-21T06:33:00.826830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:38.340252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968245+0000","last_change":"2024-09-21T06:32:59.846260+0000","last_active":"2024-09-21T06:33:01.968245+0000","last_peered":"2024-09-21T06:33:01.968245+0000","last_clean":"2024-09-21T06:33:01.968245+0000","last_became_active":"2024-09-21T06:32:59.845990+0000","last_became_peered":"2024-09-21T06:32:59.845990+0000","last_unstale":"2024-09-21T06:33:01.968245+0000","last_undegraded":"2024-09-21T06:33:01.968245+0000","last_fullsized":"2024-09-21T06:33:01.968245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:29.755064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849533+0000","last_change":"2024-09-21T06:32:58.843605+0000","last_active":"2024-09-21T06:33:00.849533+0000","last_peered":"2024-09-21T06:33:00.849533+0000","last_clean":"2024-09-21T06:33:00.849533+0000","last_became_active":"2024-09-21T06:32:58.843292+0000","last_became_peered":"2024-09-21T06:32:58.843292+0000","last_unstale":"2024-09-21T06:33:00.849533+0000","last_undegraded":"2024-09-21T06:33:00.849533+0000","last_fullsized":"2024-09-21T06:33:00.849533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:54.767330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849533+0000","last_change":"2024-09-21T06:32:59.856336+0000","last_active":"2024-09-21T06:33:00.849533+0000","last_peered":"2024-09-21T06:33:00.849533+0000","last_clean":"2024-09-21T06:33:00.849533+0000","last_became_active":"2024-09-21T06:32:59.851978+0000","last_became_peered":"2024-09-21T06:32:59.851978+0000","last_unstale":"2024-09-21T06:33:00.849533+0000","last_undegraded":"2024-09-21T06:33:00.849533+0000","last_fullsized":"2024-09-21T06:33:00.849533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:21.935781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967707+0000","last_change":"2024-09-21T06:32:58.841901+0000","last_active":"2024-09-21T06:33:01.967707+0000","last_peered":"2024-09-21T06:33:01.967707+0000","last_clean":"2024-09-21T06:33:01.967707+0000","last_became_active":"2024-09-21T06:32:58.841530+0000","last_became_peered":"2024-09-21T06:32:58.841530+0000","last_unstale":"2024-09-21T06:33:01.967707+0000","last_undegraded":"2024-09-21T06:33:01.967707+0000","last_fullsized":"2024-09-21T06:33:01.967707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:48.714380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966750+0000","last_change":"2024-09-21T06:32:59.846646+0000","last_active":"2024-09-21T06:33:01.966750+0000","last_peered":"2024-09-21T06:33:01.966750+0000","last_clean":"2024-09-21T06:33:01.966750+0000","last_became_active":"2024-09-21T06:32:59.846312+0000","last_became_peered":"2024-09-21T06:32:59.846312+0000","last_unstale":"2024-09-21T06:33:01.966750+0000","last_undegraded":"2024-09-21T06:33:01.966750+0000","last_fullsized":"2024-09-21T06:33:01.966750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.586721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824376+0000","last_change":"2024-09-21T06:32:58.841364+0000","last_active":"2024-09-21T06:33:00.824376+0000","last_peered":"2024-09-21T06:33:00.824376+0000","last_clean":"2024-09-21T06:33:00.824376+0000","last_became_active":"2024-09-21T06:32:58.840640+0000","last_became_peered":"2024-09-21T06:32:58.840640+0000","last_unstale":"2024-09-21T06:33:00.824376+0000","last_undegraded":"2024-09-21T06:33:00.824376+0000","last_fullsized":"2024-09-21T06:33:00.824376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:19.360430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827789+0000","last_change":"2024-09-21T06:32:59.844602+0000","last_active":"2024-09-21T06:33:00.827789+0000","last_peered":"2024-09-21T06:33:00.827789+0000","last_clean":"2024-09-21T06:33:00.827789+0000","last_became_active":"2024-09-21T06:32:59.844205+0000","last_became_peered":"2024-09-21T06:32:59.844205+0000","last_unstale":"2024-09-21T06:33:00.827789+0000","last_undegraded":"2024-09-21T06:33:00.827789+0000","last_fullsized":"2024-09-21T06:33:00.827789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:14.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.870410+0000","last_change":"2024-09-21T06:32:58.840210+0000","last_active":"2024-09-21T06:32:59.870410+0000","last_peered":"2024-09-21T06:32:59.870410+0000","last_clean":"2024-09-21T06:32:59.870410+0000","last_became_active":"2024-09-21T06:32:58.839642+0000","last_became_peered":"2024-09-21T06:32:58.839642+0000","last_unstale":"2024-09-21T06:32:59.870410+0000","last_undegraded":"2024-09-21T06:32:59.870410+0000","last_fullsized":"2024-09-21T06:32:59.870410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.838454+0000","last_change":"2024-09-21T06:32:59.832346+0000","last_active":"2024-09-21T06:32:59.838454+0000","last_peered":"2024-09-21T06:32:59.838454+0000","last_clean":"2024-09-21T06:32:59.838454+0000","last_became_active":"2024-09-21T06:32:59.832008+0000","last_became_peered":"2024-09-21T06:32:59.832008+0000","last_unstale":"2024-09-21T06:32:59.838454+0000","last_undegraded":"2024-09-21T06:32:59.838454+0000","last_fullsized":"2024-09-21T06:32:59.838454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:35.305486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968616+0000","last_change":"2024-09-21T06:32:58.840694+0000","last_active":"2024-09-21T06:33:01.968616+0000","last_peered":"2024-09-21T06:33:01.968616+0000","last_clean":"2024-09-21T06:33:01.968616+0000","last_became_active":"2024-09-21T06:32:58.840397+0000","last_became_peered":"2024-09-21T06:32:58.840397+0000","last_unstale":"2024-09-21T06:33:01.968616+0000","last_undegraded":"2024-09-21T06:33:01.968616+0000","last_fullsized":"2024-09-21T06:33:01.968616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:25.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967395+0000","last_change":"2024-09-21T06:32:59.855745+0000","last_active":"2024-09-21T06:33:01.967395+0000","last_peered":"2024-09-21T06:33:01.967395+0000","last_clean":"2024-09-21T06:33:01.967395+0000","last_became_active":"2024-09-21T06:32:59.855443+0000","last_became_peered":"2024-09-21T06:32:59.855443+0000","last_unstale":"2024-09-21T06:33:01.967395+0000","last_undegraded":"2024-09-21T06:33:01.967395+0000","last_fullsized":"2024-09-21T06:33:01.967395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:08.351207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824150+0000","last_change":"2024-09-21T06:32:58.840545+0000","last_active":"2024-09-21T06:33:00.824150+0000","last_peered":"2024-09-21T06:33:00.824150+0000","last_clean":"2024-09-21T06:33:00.824150+0000","last_became_active":"2024-09-21T06:32:58.840087+0000","last_became_peered":"2024-09-21T06:32:58.840087+0000","last_unstale":"2024-09-21T06:33:00.824150+0000","last_undegraded":"2024-09-21T06:33:00.824150+0000","last_fullsized":"2024-09-21T06:33:00.824150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:48.714380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824229+0000","last_change":"2024-09-21T06:32:59.849428+0000","last_active":"2024-09-21T06:33:00.824229+0000","last_peered":"2024-09-21T06:33:00.824229+0000","last_clean":"2024-09-21T06:33:00.824229+0000","last_became_active":"2024-09-21T06:32:59.849114+0000","last_became_peered":"2024-09-21T06:32:59.849114+0000","last_unstale":"2024-09-21T06:33:00.824229+0000","last_undegraded":"2024-09-21T06:33:00.824229+0000","last_fullsized":"2024-09-21T06:33:00.824229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966825+0000","last_change":"2024-09-21T06:32:58.843667+0000","last_active":"2024-09-21T06:33:01.966825+0000","last_peered":"2024-09-21T06:33:01.966825+0000","last_clean":"2024-09-21T06:33:01.966825+0000","last_became_active":"2024-09-21T06:32:58.843141+0000","last_became_peered":"2024-09-21T06:32:58.843141+0000","last_unstale":"2024-09-21T06:33:01.966825+0000","last_undegraded":"2024-09-21T06:33:01.966825+0000","last_fullsized":"2024-09-21T06:33:01.966825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:16.059805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968730+0000","last_change":"2024-09-21T06:32:59.842809+0000","last_active":"2024-09-21T06:33:01.968730+0000","last_peered":"2024-09-21T06:33:01.968730+0000","last_clean":"2024-09-21T06:33:01.968730+0000","last_became_active":"2024-09-21T06:32:59.842433+0000","last_became_peered":"2024-09-21T06:32:59.842433+0000","last_unstale":"2024-09-21T06:33:01.968730+0000","last_undegraded":"2024-09-21T06:33:01.968730+0000","last_fullsized":"2024-09-21T06:33:01.968730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:38.976131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967075+0000","last_change":"2024-09-21T06:32:58.841937+0000","last_active":"2024-09-21T06:33:01.967075+0000","last_peered":"2024-09-21T06:33:01.967075+0000","last_clean":"2024-09-21T06:33:01.967075+0000","last_became_active":"2024-09-21T06:32:58.841628+0000","last_became_peered":"2024-09-21T06:32:58.841628+0000","last_unstale":"2024-09-21T06:33:01.967075+0000","last_undegraded":"2024-09-21T06:33:01.967075+0000","last_fullsized":"2024-09-21T06:33:01.967075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:11.986551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824694+0000","last_change":"2024-09-21T06:32:59.831529+0000","last_active":"2024-09-21T06:33:00.824694+0000","last_peered":"2024-09-21T06:33:00.824694+0000","last_clean":"2024-09-21T06:33:00.824694+0000","last_became_active":"2024-09-21T06:32:59.830948+0000","last_became_peered":"2024-09-21T06:32:59.830948+0000","last_unstale":"2024-09-21T06:33:00.824694+0000","last_undegraded":"2024-09-21T06:33:00.824694+0000","last_fullsized":"2024-09-21T06:33:00.824694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:54.681416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.869374+0000","last_change":"2024-09-21T06:32:58.847929+0000","last_active":"2024-09-21T06:32:59.869374+0000","last_peered":"2024-09-21T06:32:59.869374+0000","last_clean":"2024-09-21T06:32:59.869374+0000","last_became_active":"2024-09-21T06:32:58.847571+0000","last_became_peered":"2024-09-21T06:32:58.847571+0000","last_unstale":"2024-09-21T06:32:59.869374+0000","last_undegraded":"2024-09-21T06:32:59.869374+0000","last_fullsized":"2024-09-21T06:32:59.869374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:31.689104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966913+0000","last_change":"2024-09-21T06:32:59.869317+0000","last_active":"2024-09-21T06:33:01.966913+0000","last_peered":"2024-09-21T06:33:01.966913+0000","last_clean":"2024-09-21T06:33:01.966913+0000","last_became_active":"2024-09-21T06:32:59.868680+0000","last_became_peered":"2024-09-21T06:32:59.868680+0000","last_unstale":"2024-09-21T06:33:01.966913+0000","last_undegraded":"2024-09-21T06:33:01.966913+0000","last_fullsized":"2024-09-21T06:33:01.966913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:39.345334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967820+0000","last_change":"2024-09-21T06:32:58.847303+0000","last_active":"2024-09-21T06:33:01.967820+0000","last_peered":"2024-09-21T06:33:01.967820+0000","last_clean":"2024-09-21T06:33:01.967820+0000","last_became_active":"2024-09-21T06:32:58.846843+0000","last_became_peered":"2024-09-21T06:32:58.846843+0000","last_unstale":"2024-09-21T06:33:01.967820+0000","last_undegraded":"2024-09-21T06:33:01.967820+0000","last_fullsized":"2024-09-21T06:33:01.967820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.028650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849771+0000","last_change":"2024-09-21T06:32:59.852062+0000","last_active":"2024-09-21T06:33:00.849771+0000","last_peered":"2024-09-21T06:33:00.849771+0000","last_clean":"2024-09-21T06:33:00.849771+0000","last_became_active":"2024-09-21T06:32:59.851430+0000","last_became_peered":"2024-09-21T06:32:59.851430+0000","last_unstale":"2024-09-21T06:33:00.849771+0000","last_undegraded":"2024-09-21T06:33:00.849771+0000","last_fullsized":"2024-09-21T06:33:00.849771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.397998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967859+0000","last_change":"2024-09-21T06:32:58.834861+0000","last_active":"2024-09-21T06:33:01.967859+0000","last_peered":"2024-09-21T06:33:01.967859+0000","last_clean":"2024-09-21T06:33:01.967859+0000","last_became_active":"2024-09-21T06:32:58.834041+0000","last_became_peered":"2024-09-21T06:32:58.834041+0000","last_unstale":"2024-09-21T06:33:01.967859+0000","last_undegraded":"2024-09-21T06:33:01.967859+0000","last_fullsized":"2024-09-21T06:33:01.967859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:35.774799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827603+0000","last_change":"2024-09-21T06:32:59.836560+0000","last_active":"2024-09-21T06:33:00.827603+0000","last_peered":"2024-09-21T06:33:00.827603+0000","last_clean":"2024-09-21T06:33:00.827603+0000","last_became_active":"2024-09-21T06:32:59.835815+0000","last_became_peered":"2024-09-21T06:32:59.835815+0000","last_unstale":"2024-09-21T06:33:00.827603+0000","last_undegraded":"2024-09-21T06:33:00.827603+0000","last_fullsized":"2024-09-21T06:33:00.827603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:15.356828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967239+0000","last_change":"2024-09-21T06:32:58.841198+0000","last_active":"2024-09-21T06:33:01.967239+0000","last_peered":"2024-09-21T06:33:01.967239+0000","last_clean":"2024-09-21T06:33:01.967239+0000","last_became_active":"2024-09-21T06:32:58.838947+0000","last_became_peered":"2024-09-21T06:32:58.838947+0000","last_unstale":"2024-09-21T06:33:01.967239+0000","last_undegraded":"2024-09-21T06:33:01.967239+0000","last_fullsized":"2024-09-21T06:33:01.967239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:07.376937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967104+0000","last_change":"2024-09-21T06:32:59.858174+0000","last_active":"2024-09-21T06:33:01.967104+0000","last_peered":"2024-09-21T06:33:01.967104+0000","last_clean":"2024-09-21T06:33:01.967104+0000","last_became_active":"2024-09-21T06:32:59.853755+0000","last_became_peered":"2024-09-21T06:32:59.853755+0000","last_unstale":"2024-09-21T06:33:01.967104+0000","last_undegraded":"2024-09-21T06:33:01.967104+0000","last_fullsized":"2024-09-21T06:33:01.967104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:46.238981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967052+0000","last_change":"2024-09-21T06:32:58.847149+0000","last_active":"2024-09-21T06:33:01.967052+0000","last_peered":"2024-09-21T06:33:01.967052+0000","last_clean":"2024-09-21T06:33:01.967052+0000","last_became_active":"2024-09-21T06:32:58.846529+0000","last_became_peered":"2024-09-21T06:32:58.846529+0000","last_unstale":"2024-09-21T06:33:01.967052+0000","last_undegraded":"2024-09-21T06:33:01.967052+0000","last_fullsized":"2024-09-21T06:33:01.967052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:43.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967151+0000","last_change":"2024-09-21T06:32:59.866301+0000","last_active":"2024-09-21T06:33:01.967151+0000","last_peered":"2024-09-21T06:33:01.967151+0000","last_clean":"2024-09-21T06:33:01.967151+0000","last_became_active":"2024-09-21T06:32:59.865924+0000","last_became_peered":"2024-09-21T06:32:59.865924+0000","last_unstale":"2024-09-21T06:33:01.967151+0000","last_undegraded":"2024-09-21T06:33:01.967151+0000","last_fullsized":"2024-09-21T06:33:01.967151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:28.921748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850075+0000","last_change":"2024-09-21T06:32:58.849629+0000","last_active":"2024-09-21T06:33:00.850075+0000","last_peered":"2024-09-21T06:33:00.850075+0000","last_clean":"2024-09-21T06:33:00.850075+0000","last_became_active":"2024-09-21T06:32:58.848996+0000","last_became_peered":"2024-09-21T06:32:58.848996+0000","last_unstale":"2024-09-21T06:33:00.850075+0000","last_undegraded":"2024-09-21T06:33:00.850075+0000","last_fullsized":"2024-09-21T06:33:00.850075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:16.226451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967314+0000","last_change":"2024-09-21T06:32:59.846339+0000","last_active":"2024-09-21T06:33:01.967314+0000","last_peered":"2024-09-21T06:33:01.967314+0000","last_clean":"2024-09-21T06:33:01.967314+0000","last_became_active":"2024-09-21T06:32:59.845392+0000","last_became_peered":"2024-09-21T06:32:59.845392+0000","last_unstale":"2024-09-21T06:33:01.967314+0000","last_undegraded":"2024-09-21T06:33:01.967314+0000","last_fullsized":"2024-09-21T06:33:01.967314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826903+0000","last_change":"2024-09-21T06:32:58.850262+0000","last_active":"2024-09-21T06:33:00.826903+0000","last_peered":"2024-09-21T06:33:00.826903+0000","last_clean":"2024-09-21T06:33:00.826903+0000","last_became_active":"2024-09-21T06:32:58.849928+0000","last_became_peered":"2024-09-21T06:32:58.849928+0000","last_unstale":"2024-09-21T06:33:00.826903+0000","last_undegraded":"2024-09-21T06:33:00.826903+0000","last_fullsized":"2024-09-21T06:33:00.826903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:22.635877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968494+0000","last_change":"2024-09-21T06:32:59.841703+0000","last_active":"2024-09-21T06:33:01.968494+0000","last_peered":"2024-09-21T06:33:01.968494+0000","last_clean":"2024-09-21T06:33:01.968494+0000","last_became_active":"2024-09-21T06:32:59.840836+0000","last_became_peered":"2024-09-21T06:32:59.840836+0000","last_unstale":"2024-09-21T06:33:01.968494+0000","last_undegraded":"2024-09-21T06:33:01.968494+0000","last_fullsized":"2024-09-21T06:33:01.968494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:16.727147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849690+0000","last_change":"2024-09-21T06:32:58.843915+0000","last_active":"2024-09-21T06:33:00.849690+0000","last_peered":"2024-09-21T06:33:00.849690+0000","last_clean":"2024-09-21T06:33:00.849690+0000","last_became_active":"2024-09-21T06:32:58.843488+0000","last_became_peered":"2024-09-21T06:32:58.843488+0000","last_unstale":"2024-09-21T06:33:00.849690+0000","last_undegraded":"2024-09-21T06:33:00.849690+0000","last_fullsized":"2024-09-21T06:33:00.849690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.336915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967170+0000","last_change":"2024-09-21T06:32:59.846403+0000","last_active":"2024-09-21T06:33:01.967170+0000","last_peered":"2024-09-21T06:33:01.967170+0000","last_clean":"2024-09-21T06:33:01.967170+0000","last_became_active":"2024-09-21T06:32:59.846045+0000","last_became_peered":"2024-09-21T06:32:59.846045+0000","last_unstale":"2024-09-21T06:33:01.967170+0000","last_undegraded":"2024-09-21T06:33:01.967170+0000","last_fullsized":"2024-09-21T06:33:01.967170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:34.295415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824761+0000","last_change":"2024-09-21T06:32:58.842624+0000","last_active":"2024-09-21T06:33:00.824761+0000","last_peered":"2024-09-21T06:33:00.824761+0000","last_clean":"2024-09-21T06:33:00.824761+0000","last_became_active":"2024-09-21T06:32:58.842191+0000","last_became_peered":"2024-09-21T06:32:58.842191+0000","last_unstale":"2024-09-21T06:33:00.824761+0000","last_undegraded":"2024-09-21T06:33:00.824761+0000","last_fullsized":"2024-09-21T06:33:00.824761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:40.219723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824478+0000","last_change":"2024-09-21T06:32:59.841805+0000","last_active":"2024-09-21T06:33:00.824478+0000","last_peered":"2024-09-21T06:33:00.824478+0000","last_clean":"2024-09-21T06:33:00.824478+0000","last_became_active":"2024-09-21T06:32:59.841523+0000","last_became_peered":"2024-09-21T06:32:59.841523+0000","last_unstale":"2024-09-21T06:33:00.824478+0000","last_undegraded":"2024-09-21T06:33:00.824478+0000","last_fullsized":"2024-09-21T06:33:00.824478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.840820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827344+0000","last_change":"2024-09-21T06:32:58.842291+0000","last_active":"2024-09-21T06:32:59.827344+0000","last_peered":"2024-09-21T06:32:59.827344+0000","last_clean":"2024-09-21T06:32:59.827344+0000","last_became_active":"2024-09-21T06:32:58.841798+0000","last_became_peered":"2024-09-21T06:32:58.841798+0000","last_unstale":"2024-09-21T06:32:59.827344+0000","last_undegraded":"2024-09-21T06:32:59.827344+0000","last_fullsized":"2024-09-21T06:32:59.827344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:41.269562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966759+0000","last_change":"2024-09-21T06:32:59.850214+0000","last_active":"2024-09-21T06:33:01.966759+0000","last_peered":"2024-09-21T06:33:01.966759+0000","last_clean":"2024-09-21T06:33:01.966759+0000","last_became_active":"2024-09-21T06:32:59.849598+0000","last_became_peered":"2024-09-21T06:32:59.849598+0000","last_unstale":"2024-09-21T06:33:01.966759+0000","last_undegraded":"2024-09-21T06:33:01.966759+0000","last_fullsized":"2024-09-21T06:33:01.966759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:34.828987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824207+0000","last_change":"2024-09-21T06:32:58.848333+0000","last_active":"2024-09-21T06:33:00.824207+0000","last_peered":"2024-09-21T06:33:00.824207+0000","last_clean":"2024-09-21T06:33:00.824207+0000","last_became_active":"2024-09-21T06:32:58.847085+0000","last_became_peered":"2024-09-21T06:32:58.847085+0000","last_unstale":"2024-09-21T06:33:00.824207+0000","last_undegraded":"2024-09-21T06:33:00.824207+0000","last_fullsized":"2024-09-21T06:33:00.824207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:49.153292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827827+0000","last_change":"2024-09-21T06:32:59.848770+0000","last_active":"2024-09-21T06:33:00.827827+0000","last_peered":"2024-09-21T06:33:00.827827+0000","last_clean":"2024-09-21T06:33:00.827827+0000","last_became_active":"2024-09-21T06:32:59.847200+0000","last_became_peered":"2024-09-21T06:32:59.847200+0000","last_unstale":"2024-09-21T06:33:00.827827+0000","last_undegraded":"2024-09-21T06:33:00.827827+0000","last_fullsized":"2024-09-21T06:33:00.827827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:42.153494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967462+0000","last_change":"2024-09-21T06:32:58.842042+0000","last_active":"2024-09-21T06:33:01.967462+0000","last_peered":"2024-09-21T06:33:01.967462+0000","last_clean":"2024-09-21T06:33:01.967462+0000","last_became_active":"2024-09-21T06:32:58.841730+0000","last_became_peered":"2024-09-21T06:32:58.841730+0000","last_unstale":"2024-09-21T06:33:01.967462+0000","last_undegraded":"2024-09-21T06:33:01.967462+0000","last_fullsized":"2024-09-21T06:33:01.967462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:49.153292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968032+0000","last_change":"2024-09-21T06:32:59.865377+0000","last_active":"2024-09-21T06:33:01.968032+0000","last_peered":"2024-09-21T06:33:01.968032+0000","last_clean":"2024-09-21T06:33:01.968032+0000","last_became_active":"2024-09-21T06:32:59.865025+0000","last_became_peered":"2024-09-21T06:32:59.865025+0000","last_unstale":"2024-09-21T06:33:01.968032+0000","last_undegraded":"2024-09-21T06:33:01.968032+0000","last_fullsized":"2024-09-21T06:33:01.968032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:35.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968160+0000","last_change":"2024-09-21T06:32:58.837354+0000","last_active":"2024-09-21T06:33:01.968160+0000","last_peered":"2024-09-21T06:33:01.968160+0000","last_clean":"2024-09-21T06:33:01.968160+0000","last_became_active":"2024-09-21T06:32:58.836991+0000","last_became_peered":"2024-09-21T06:32:58.836991+0000","last_unstale":"2024-09-21T06:33:01.968160+0000","last_undegraded":"2024-09-21T06:33:01.968160+0000","last_fullsized":"2024-09-21T06:33:01.968160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:11.640799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968339+0000","last_change":"2024-09-21T06:32:59.853900+0000","last_active":"2024-09-21T06:33:01.968339+0000","last_peered":"2024-09-21T06:33:01.968339+0000","last_clean":"2024-09-21T06:33:01.968339+0000","last_became_active":"2024-09-21T06:32:59.853587+0000","last_became_peered":"2024-09-21T06:32:59.853587+0000","last_unstale":"2024-09-21T06:33:01.968339+0000","last_undegraded":"2024-09-21T06:33:01.968339+0000","last_fullsized":"2024-09-21T06:33:01.968339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:59.690110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827931+0000","last_change":"2024-09-21T06:32:58.843925+0000","last_active":"2024-09-21T06:32:59.827931+0000","last_peered":"2024-09-21T06:32:59.827931+0000","last_clean":"2024-09-21T06:32:59.827931+0000","last_became_active":"2024-09-21T06:32:58.843660+0000","last_became_peered":"2024-09-21T06:32:58.843660+0000","last_unstale":"2024-09-21T06:32:59.827931+0000","last_undegraded":"2024-09-21T06:32:59.827931+0000","last_fullsized":"2024-09-21T06:32:59.827931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:22.642787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850464+0000","last_change":"2024-09-21T06:32:59.853844+0000","last_active":"2024-09-21T06:33:00.850464+0000","last_peered":"2024-09-21T06:33:00.850464+0000","last_clean":"2024-09-21T06:33:00.850464+0000","last_became_active":"2024-09-21T06:32:59.853302+0000","last_became_peered":"2024-09-21T06:32:59.853302+0000","last_unstale":"2024-09-21T06:33:00.850464+0000","last_undegraded":"2024-09-21T06:33:00.850464+0000","last_fullsized":"2024-09-21T06:33:00.850464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:41.902877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966929+0000","last_change":"2024-09-21T06:32:58.836393+0000","last_active":"2024-09-21T06:33:01.966929+0000","last_peered":"2024-09-21T06:33:01.966929+0000","last_clean":"2024-09-21T06:33:01.966929+0000","last_became_active":"2024-09-21T06:32:58.835402+0000","last_became_peered":"2024-09-21T06:32:58.835402+0000","last_unstale":"2024-09-21T06:33:01.966929+0000","last_undegraded":"2024-09-21T06:33:01.966929+0000","last_fullsized":"2024-09-21T06:33:01.966929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:06.204232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967272+0000","last_change":"2024-09-21T06:32:59.869158+0000","last_active":"2024-09-21T06:33:01.967272+0000","last_peered":"2024-09-21T06:33:01.967272+0000","last_clean":"2024-09-21T06:33:01.967272+0000","last_became_active":"2024-09-21T06:32:59.868169+0000","last_became_peered":"2024-09-21T06:32:59.868169+0000","last_unstale":"2024-09-21T06:33:01.967272+0000","last_undegraded":"2024-09-21T06:33:01.967272+0000","last_fullsized":"2024-09-21T06:33:01.967272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:11.078357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849804+0000","last_change":"2024-09-21T06:32:58.842862+0000","last_active":"2024-09-21T06:33:00.849804+0000","last_peered":"2024-09-21T06:33:00.849804+0000","last_clean":"2024-09-21T06:33:00.849804+0000","last_became_active":"2024-09-21T06:32:58.840475+0000","last_became_peered":"2024-09-21T06:32:58.840475+0000","last_unstale":"2024-09-21T06:33:00.849804+0000","last_undegraded":"2024-09-21T06:33:00.849804+0000","last_fullsized":"2024-09-21T06:33:00.849804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:26.802562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968376+0000","last_change":"2024-09-21T06:32:59.847041+0000","last_active":"2024-09-21T06:33:01.968376+0000","last_peered":"2024-09-21T06:33:01.968376+0000","last_clean":"2024-09-21T06:33:01.968376+0000","last_became_active":"2024-09-21T06:32:59.846748+0000","last_became_peered":"2024-09-21T06:32:59.846748+0000","last_unstale":"2024-09-21T06:33:01.968376+0000","last_undegraded":"2024-09-21T06:33:01.968376+0000","last_fullsized":"2024-09-21T06:33:01.968376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:01.671559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827837+0000","last_change":"2024-09-21T06:32:58.838303+0000","last_active":"2024-09-21T06:32:59.827837+0000","last_peered":"2024-09-21T06:32:59.827837+0000","last_clean":"2024-09-21T06:32:59.827837+0000","last_became_active":"2024-09-21T06:32:58.837979+0000","last_became_peered":"2024-09-21T06:32:58.837979+0000","last_unstale":"2024-09-21T06:32:59.827837+0000","last_undegraded":"2024-09-21T06:32:59.827837+0000","last_fullsized":"2024-09-21T06:32:59.827837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.835519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":219,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.823883+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:33:00.823883+0000","last_peered":"2024-09-21T06:33:00.823883+0000","last_clean":"2024-09-21T06:33:00.823883+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:33:00.823883+0000","last_undegraded":"2024-09-21T06:33:00.823883+0000","last_fullsized":"2024-09-21T06:33:00.823883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509506,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":445293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":445313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":991232,"data_stored":446357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":444641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509508,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1581056,"data_stored":1036655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":1021150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:33:04.334 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:33:04.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 -- 172.21.15.37:0/2236868121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 msgr2=0x7f058c154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 --2- 172.21.15.37:0/2236868121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c154ea0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f058c162210 tx=0x7f058402aac0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 -- 172.21.15.37:0/2236868121 shutdown_connections 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 --2- 172.21.15.37:0/2236868121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c154ea0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 --2- 172.21.15.37:0/2236868121 >> v2:172.21.15.150:3300/0 conn(0x7f058c1629c0 0x7f058c06d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 -- 172.21.15.37:0/2236868121 >> 172.21.15.37:0/2236868121 conn(0x7f058c0d1930 msgr2=0x7f058c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 -- 172.21.15.37:0/2236868121 shutdown_connections 2024-09-21T06:33:04.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.594+0000 7f0592200640 1 -- 172.21.15.37:0/2236868121 wait complete. 2024-09-21T06:33:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.596+0000 7f0592200640 1 Processor -- start 2024-09-21T06:33:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.596+0000 7f0592200640 1 -- start start 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058c0b9e40 0x7f058c1f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f058c1f8f10 0x7f058c1f9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f058c155820 con 0x7f058c06d7f0 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1556a0 con 0x7f058c0b9e40 2024-09-21T06:33:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0592200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f058c1559a0 con 0x7f058c1f8f10 2024-09-21T06:33:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0590e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0590e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54514/0 (socket says 172.21.15.37:54514) 2024-09-21T06:33:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0590e00640 1 -- 172.21.15.37:0/1497932251 learned_addr learned my addr 172.21.15.37:0/1497932251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0583e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058c0b9e40 0x7f058c1f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.598+0000 7f0591800640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f058c1f8f10 0x7f058c1f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 -- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f058c1f8f10 msgr2=0x7f058c1f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f058c1f8f10 0x7f058c1f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 -- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058c0b9e40 msgr2=0x7f058c1f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058c0b9e40 0x7f058c1f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 -- 172.21.15.37:0/1497932251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c0cbaf0 con 0x7f058c06d7f0 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0590e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c3f0 tx=0x7f057c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:04.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c017070 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f057c0035f0 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c003a40 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058c0b8bf0 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.599+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058c0b91b0 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.600+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058c154ea0 con 0x7f058c06d7f0 2024-09-21T06:33:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.602+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f057c01a070 con 0x7f058c06d7f0 2024-09-21T06:33:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.603+0000 7f0589600640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 0x7f056807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.603+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f057c09a260 con 0x7f058c06d7f0 2024-09-21T06:33:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.603+0000 7f0583e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 0x7f056807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:04.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.604+0000 7f0583e00640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 0x7f056807b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0574005fd0 tx=0x7f0574005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:04.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.607+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f058c163170 con 0x7f058c06d7f0 2024-09-21T06:33:04.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.803+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f058c162d80 con 0x7f0568078bc0 2024-09-21T06:33:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.809+0000 7f0589600640 1 -- 172.21.15.37:0/1497932251 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7f058c162d80 con 0x7f0568078bc0 2024-09-21T06:33:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:33:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 msgr2=0x7f056807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 0x7f056807b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0574005fd0 tx=0x7f0574005850 comp rx=0 tx=0).stop 2024-09-21T06:33:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 msgr2=0x7f058c0c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c3f0 tx=0x7f057c00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 shutdown_connections 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0568078bc0 0x7f056807b0d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f058c1f8f10 0x7f058c1f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058c0b9e40 0x7f058c1f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 --2- 172.21.15.37:0/1497932251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f058c06d7f0 0x7f058c0c83e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 >> 172.21.15.37:0/1497932251 conn(0x7f058c0d1930 msgr2=0x7f058c14d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 shutdown_connections 2024-09-21T06:33:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:04.813+0000 7f0592200640 1 -- 172.21.15.37:0/1497932251 wait complete. 2024-09-21T06:33:04.838 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-21T06:33:03.171920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258312,"kb_used_data":8780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716408,"statfs":{"total":773094113280,"available":772829601792,"internally_reserved":0,"allocated":8990720,"data_stored":4728735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.357267"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824683+0000","last_change":"2024-09-21T06:32:59.838263+0000","last_active":"2024-09-21T06:33:00.824683+0000","last_peered":"2024-09-21T06:33:00.824683+0000","last_clean":"2024-09-21T06:33:00.824683+0000","last_became_active":"2024-09-21T06:32:59.837943+0000","last_became_peered":"2024-09-21T06:32:59.837943+0000","last_unstale":"2024-09-21T06:33:00.824683+0000","last_undegraded":"2024-09-21T06:33:00.824683+0000","last_fullsized":"2024-09-21T06:33:00.824683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:34.295415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850302+0000","last_change":"2024-09-21T06:32:59.853583+0000","last_active":"2024-09-21T06:33:00.850302+0000","last_peered":"2024-09-21T06:33:00.850302+0000","last_clean":"2024-09-21T06:33:00.850302+0000","last_became_active":"2024-09-21T06:32:59.853204+0000","last_became_peered":"2024-09-21T06:32:59.853204+0000","last_unstale":"2024-09-21T06:33:00.850302+0000","last_undegraded":"2024-09-21T06:33:00.850302+0000","last_fullsized":"2024-09-21T06:33:00.850302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:34.568193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850092+0000","last_change":"2024-09-21T06:32:59.848913+0000","last_active":"2024-09-21T06:33:00.850092+0000","last_peered":"2024-09-21T06:33:00.850092+0000","last_clean":"2024-09-21T06:33:00.850092+0000","last_became_active":"2024-09-21T06:32:59.848556+0000","last_became_peered":"2024-09-21T06:32:59.848556+0000","last_unstale":"2024-09-21T06:33:00.850092+0000","last_undegraded":"2024-09-21T06:33:00.850092+0000","last_fullsized":"2024-09-21T06:33:00.850092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:38.985928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967391+0000","last_change":"2024-09-21T06:32:59.839766+0000","last_active":"2024-09-21T06:33:01.967391+0000","last_peered":"2024-09-21T06:33:01.967391+0000","last_clean":"2024-09-21T06:33:01.967391+0000","last_became_active":"2024-09-21T06:32:59.839187+0000","last_became_peered":"2024-09-21T06:32:59.839187+0000","last_unstale":"2024-09-21T06:33:01.967391+0000","last_undegraded":"2024-09-21T06:33:01.967391+0000","last_fullsized":"2024-09-21T06:33:01.967391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:06.137043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967016+0000","last_change":"2024-09-21T06:32:59.858950+0000","last_active":"2024-09-21T06:33:01.967016+0000","last_peered":"2024-09-21T06:33:01.967016+0000","last_clean":"2024-09-21T06:33:01.967016+0000","last_became_active":"2024-09-21T06:32:59.858604+0000","last_became_peered":"2024-09-21T06:32:59.858604+0000","last_unstale":"2024-09-21T06:33:01.967016+0000","last_undegraded":"2024-09-21T06:33:01.967016+0000","last_fullsized":"2024-09-21T06:33:01.967016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:53.682712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.842507+0000","last_change":"2024-09-21T06:32:59.838769+0000","last_active":"2024-09-21T06:32:59.842507+0000","last_peered":"2024-09-21T06:32:59.842507+0000","last_clean":"2024-09-21T06:32:59.842507+0000","last_became_active":"2024-09-21T06:32:59.833006+0000","last_became_peered":"2024-09-21T06:32:59.833006+0000","last_unstale":"2024-09-21T06:32:59.842507+0000","last_undegraded":"2024-09-21T06:32:59.842507+0000","last_fullsized":"2024-09-21T06:32:59.842507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:28.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966864+0000","last_change":"2024-09-21T06:32:59.851329+0000","last_active":"2024-09-21T06:33:01.966864+0000","last_peered":"2024-09-21T06:33:01.966864+0000","last_clean":"2024-09-21T06:33:01.966864+0000","last_became_active":"2024-09-21T06:32:59.850950+0000","last_became_peered":"2024-09-21T06:32:59.850950+0000","last_unstale":"2024-09-21T06:33:01.966864+0000","last_undegraded":"2024-09-21T06:33:01.966864+0000","last_fullsized":"2024-09-21T06:33:01.966864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.840820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826507+0000","last_change":"2024-09-21T06:32:59.847314+0000","last_active":"2024-09-21T06:33:00.826507+0000","last_peered":"2024-09-21T06:33:00.826507+0000","last_clean":"2024-09-21T06:33:00.826507+0000","last_became_active":"2024-09-21T06:32:59.846841+0000","last_became_peered":"2024-09-21T06:32:59.846841+0000","last_unstale":"2024-09-21T06:33:00.826507+0000","last_undegraded":"2024-09-21T06:33:00.826507+0000","last_fullsized":"2024-09-21T06:33:00.826507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:51.095219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967796+0000","last_change":"2024-09-21T06:32:59.857107+0000","last_active":"2024-09-21T06:33:01.967796+0000","last_peered":"2024-09-21T06:33:01.967796+0000","last_clean":"2024-09-21T06:33:01.967796+0000","last_became_active":"2024-09-21T06:32:59.856762+0000","last_became_peered":"2024-09-21T06:32:59.856762+0000","last_unstale":"2024-09-21T06:33:01.967796+0000","last_undegraded":"2024-09-21T06:33:01.967796+0000","last_fullsized":"2024-09-21T06:33:01.967796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.397998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824500+0000","last_change":"2024-09-21T06:32:59.846347+0000","last_active":"2024-09-21T06:33:00.824500+0000","last_peered":"2024-09-21T06:33:00.824500+0000","last_clean":"2024-09-21T06:33:00.824500+0000","last_became_active":"2024-09-21T06:32:59.846039+0000","last_became_peered":"2024-09-21T06:32:59.846039+0000","last_unstale":"2024-09-21T06:33:00.824500+0000","last_undegraded":"2024-09-21T06:33:00.824500+0000","last_fullsized":"2024-09-21T06:33:00.824500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:03.329349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850165+0000","last_change":"2024-09-21T06:32:59.853858+0000","last_active":"2024-09-21T06:33:00.850165+0000","last_peered":"2024-09-21T06:33:00.850165+0000","last_clean":"2024-09-21T06:33:00.850165+0000","last_became_active":"2024-09-21T06:32:59.853180+0000","last_became_peered":"2024-09-21T06:32:59.853180+0000","last_unstale":"2024-09-21T06:33:00.850165+0000","last_undegraded":"2024-09-21T06:33:00.850165+0000","last_fullsized":"2024-09-21T06:33:00.850165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.318029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967697+0000","last_change":"2024-09-21T06:32:59.843842+0000","last_active":"2024-09-21T06:33:01.967697+0000","last_peered":"2024-09-21T06:33:01.967697+0000","last_clean":"2024-09-21T06:33:01.967697+0000","last_became_active":"2024-09-21T06:32:59.843549+0000","last_became_peered":"2024-09-21T06:32:59.843549+0000","last_unstale":"2024-09-21T06:33:01.967697+0000","last_undegraded":"2024-09-21T06:33:01.967697+0000","last_fullsized":"2024-09-21T06:33:01.967697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:41.902877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850294+0000","last_change":"2024-09-21T06:32:59.862805+0000","last_active":"2024-09-21T06:33:00.850294+0000","last_peered":"2024-09-21T06:33:00.850294+0000","last_clean":"2024-09-21T06:33:00.850294+0000","last_became_active":"2024-09-21T06:32:59.862482+0000","last_became_peered":"2024-09-21T06:32:59.862482+0000","last_unstale":"2024-09-21T06:33:00.850294+0000","last_undegraded":"2024-09-21T06:33:00.850294+0000","last_fullsized":"2024-09-21T06:33:00.850294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.586721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827312+0000","last_change":"2024-09-21T06:32:59.836878+0000","last_active":"2024-09-21T06:33:00.827312+0000","last_peered":"2024-09-21T06:33:00.827312+0000","last_clean":"2024-09-21T06:33:00.827312+0000","last_became_active":"2024-09-21T06:32:59.836258+0000","last_became_peered":"2024-09-21T06:32:59.836258+0000","last_unstale":"2024-09-21T06:33:00.827312+0000","last_undegraded":"2024-09-21T06:33:00.827312+0000","last_fullsized":"2024-09-21T06:33:00.827312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:36.037339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966773+0000","last_change":"2024-09-21T06:32:59.866515+0000","last_active":"2024-09-21T06:33:01.966773+0000","last_peered":"2024-09-21T06:33:01.966773+0000","last_clean":"2024-09-21T06:33:01.966773+0000","last_became_active":"2024-09-21T06:32:59.865412+0000","last_became_peered":"2024-09-21T06:32:59.865412+0000","last_unstale":"2024-09-21T06:33:01.966773+0000","last_undegraded":"2024-09-21T06:33:01.966773+0000","last_fullsized":"2024-09-21T06:33:01.966773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:55.775749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966729+0000","last_change":"2024-09-21T06:32:59.847310+0000","last_active":"2024-09-21T06:33:01.966729+0000","last_peered":"2024-09-21T06:33:01.966729+0000","last_clean":"2024-09-21T06:33:01.966729+0000","last_became_active":"2024-09-21T06:32:59.846939+0000","last_became_peered":"2024-09-21T06:32:59.846939+0000","last_unstale":"2024-09-21T06:33:01.966729+0000","last_undegraded":"2024-09-21T06:33:01.966729+0000","last_fullsized":"2024-09-21T06:33:01.966729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827634+0000","last_change":"2024-09-21T06:32:59.847284+0000","last_active":"2024-09-21T06:33:00.827634+0000","last_peered":"2024-09-21T06:33:00.827634+0000","last_clean":"2024-09-21T06:33:00.827634+0000","last_became_active":"2024-09-21T06:32:59.846874+0000","last_became_peered":"2024-09-21T06:32:59.846874+0000","last_unstale":"2024-09-21T06:33:00.827634+0000","last_undegraded":"2024-09-21T06:33:00.827634+0000","last_fullsized":"2024-09-21T06:33:00.827634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:20.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850289+0000","last_change":"2024-09-21T06:32:59.853963+0000","last_active":"2024-09-21T06:33:00.850289+0000","last_peered":"2024-09-21T06:33:00.850289+0000","last_clean":"2024-09-21T06:33:00.850289+0000","last_became_active":"2024-09-21T06:32:59.853493+0000","last_became_peered":"2024-09-21T06:32:59.853493+0000","last_unstale":"2024-09-21T06:33:00.850289+0000","last_undegraded":"2024-09-21T06:33:00.850289+0000","last_fullsized":"2024-09-21T06:33:00.850289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:47.613122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827763+0000","last_change":"2024-09-21T06:32:59.860332+0000","last_active":"2024-09-21T06:33:00.827763+0000","last_peered":"2024-09-21T06:33:00.827763+0000","last_clean":"2024-09-21T06:33:00.827763+0000","last_became_active":"2024-09-21T06:32:59.860026+0000","last_became_peered":"2024-09-21T06:32:59.860026+0000","last_unstale":"2024-09-21T06:33:00.827763+0000","last_undegraded":"2024-09-21T06:33:00.827763+0000","last_fullsized":"2024-09-21T06:33:00.827763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:55.745827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.865066+0000","last_change":"2024-09-21T06:32:59.850614+0000","last_active":"2024-09-21T06:32:59.865066+0000","last_peered":"2024-09-21T06:32:59.865066+0000","last_clean":"2024-09-21T06:32:59.865066+0000","last_became_active":"2024-09-21T06:32:59.849535+0000","last_became_peered":"2024-09-21T06:32:59.849535+0000","last_unstale":"2024-09-21T06:32:59.865066+0000","last_undegraded":"2024-09-21T06:32:59.865066+0000","last_fullsized":"2024-09-21T06:32:59.865066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:45.152420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966743+0000","last_change":"2024-09-21T06:32:59.837970+0000","last_active":"2024-09-21T06:33:01.966743+0000","last_peered":"2024-09-21T06:33:01.966743+0000","last_clean":"2024-09-21T06:33:01.966743+0000","last_became_active":"2024-09-21T06:32:59.837545+0000","last_became_peered":"2024-09-21T06:32:59.837545+0000","last_unstale":"2024-09-21T06:33:01.966743+0000","last_undegraded":"2024-09-21T06:33:01.966743+0000","last_fullsized":"2024-09-21T06:33:01.966743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:34.568193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849964+0000","last_change":"2024-09-21T06:32:59.862915+0000","last_active":"2024-09-21T06:33:00.849964+0000","last_peered":"2024-09-21T06:33:00.849964+0000","last_clean":"2024-09-21T06:33:00.849964+0000","last_became_active":"2024-09-21T06:32:59.862703+0000","last_became_peered":"2024-09-21T06:32:59.862703+0000","last_unstale":"2024-09-21T06:33:00.849964+0000","last_undegraded":"2024-09-21T06:33:00.849964+0000","last_fullsized":"2024-09-21T06:33:00.849964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:55.330268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824208+0000","last_change":"2024-09-21T06:32:59.837066+0000","last_active":"2024-09-21T06:33:00.824208+0000","last_peered":"2024-09-21T06:33:00.824208+0000","last_clean":"2024-09-21T06:33:00.824208+0000","last_became_active":"2024-09-21T06:32:59.836756+0000","last_became_peered":"2024-09-21T06:32:59.836756+0000","last_unstale":"2024-09-21T06:33:00.824208+0000","last_undegraded":"2024-09-21T06:33:00.824208+0000","last_fullsized":"2024-09-21T06:33:00.824208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:28.921748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849452+0000","last_change":"2024-09-21T06:32:59.860983+0000","last_active":"2024-09-21T06:33:00.849452+0000","last_peered":"2024-09-21T06:33:00.849452+0000","last_clean":"2024-09-21T06:33:00.849452+0000","last_became_active":"2024-09-21T06:32:59.860635+0000","last_became_peered":"2024-09-21T06:32:59.860635+0000","last_unstale":"2024-09-21T06:33:00.849452+0000","last_undegraded":"2024-09-21T06:33:00.849452+0000","last_fullsized":"2024-09-21T06:33:00.849452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:35.945934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.864557+0000","last_change":"2024-09-21T06:32:59.847290+0000","last_active":"2024-09-21T06:32:59.864557+0000","last_peered":"2024-09-21T06:32:59.864557+0000","last_clean":"2024-09-21T06:32:59.864557+0000","last_became_active":"2024-09-21T06:32:59.846008+0000","last_became_peered":"2024-09-21T06:32:59.846008+0000","last_unstale":"2024-09-21T06:32:59.864557+0000","last_undegraded":"2024-09-21T06:32:59.864557+0000","last_fullsized":"2024-09-21T06:32:59.864557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:21.117853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967788+0000","last_change":"2024-09-21T06:32:59.858959+0000","last_active":"2024-09-21T06:33:01.967788+0000","last_peered":"2024-09-21T06:33:01.967788+0000","last_clean":"2024-09-21T06:33:01.967788+0000","last_became_active":"2024-09-21T06:32:59.858615+0000","last_became_peered":"2024-09-21T06:32:59.858615+0000","last_unstale":"2024-09-21T06:33:01.967788+0000","last_undegraded":"2024-09-21T06:33:01.967788+0000","last_fullsized":"2024-09-21T06:33:01.967788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:21.935781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967497+0000","last_change":"2024-09-21T06:32:59.842692+0000","last_active":"2024-09-21T06:33:01.967497+0000","last_peered":"2024-09-21T06:33:01.967497+0000","last_clean":"2024-09-21T06:33:01.967497+0000","last_became_active":"2024-09-21T06:32:59.842322+0000","last_became_peered":"2024-09-21T06:32:59.842322+0000","last_unstale":"2024-09-21T06:33:01.967497+0000","last_undegraded":"2024-09-21T06:33:01.967497+0000","last_fullsized":"2024-09-21T06:33:01.967497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:35.945934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966926+0000","last_change":"2024-09-21T06:32:59.870736+0000","last_active":"2024-09-21T06:33:01.966926+0000","last_peered":"2024-09-21T06:33:01.966926+0000","last_clean":"2024-09-21T06:33:01.966926+0000","last_became_active":"2024-09-21T06:32:59.870317+0000","last_became_peered":"2024-09-21T06:32:59.870317+0000","last_unstale":"2024-09-21T06:33:01.966926+0000","last_undegraded":"2024-09-21T06:33:01.966926+0000","last_fullsized":"2024-09-21T06:33:01.966926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:27.810982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.864429+0000","last_change":"2024-09-21T06:32:59.845738+0000","last_active":"2024-09-21T06:32:59.864429+0000","last_peered":"2024-09-21T06:32:59.864429+0000","last_clean":"2024-09-21T06:32:59.864429+0000","last_became_active":"2024-09-21T06:32:59.845369+0000","last_became_peered":"2024-09-21T06:32:59.845369+0000","last_unstale":"2024-09-21T06:32:59.864429+0000","last_undegraded":"2024-09-21T06:32:59.864429+0000","last_fullsized":"2024-09-21T06:32:59.864429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:52.816245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824211+0000","last_change":"2024-09-21T06:32:59.829555+0000","last_active":"2024-09-21T06:33:00.824211+0000","last_peered":"2024-09-21T06:33:00.824211+0000","last_clean":"2024-09-21T06:33:00.824211+0000","last_became_active":"2024-09-21T06:32:59.829233+0000","last_became_peered":"2024-09-21T06:32:59.829233+0000","last_unstale":"2024-09-21T06:33:00.824211+0000","last_undegraded":"2024-09-21T06:33:00.824211+0000","last_fullsized":"2024-09-21T06:33:00.824211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:50.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849791+0000","last_change":"2024-09-21T06:32:59.851727+0000","last_active":"2024-09-21T06:33:00.849791+0000","last_peered":"2024-09-21T06:33:00.849791+0000","last_clean":"2024-09-21T06:33:00.849791+0000","last_became_active":"2024-09-21T06:32:59.851023+0000","last_became_peered":"2024-09-21T06:32:59.851023+0000","last_unstale":"2024-09-21T06:33:00.849791+0000","last_undegraded":"2024-09-21T06:33:00.849791+0000","last_fullsized":"2024-09-21T06:33:00.849791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:59.690110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966950+0000","last_change":"2024-09-21T06:32:59.855643+0000","last_active":"2024-09-21T06:33:01.966950+0000","last_peered":"2024-09-21T06:33:01.966950+0000","last_clean":"2024-09-21T06:33:01.966950+0000","last_became_active":"2024-09-21T06:32:59.855128+0000","last_became_peered":"2024-09-21T06:32:59.855128+0000","last_unstale":"2024-09-21T06:33:01.966950+0000","last_undegraded":"2024-09-21T06:33:01.966950+0000","last_fullsized":"2024-09-21T06:33:01.966950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:03.329349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966694+0000","last_change":"2024-09-21T06:32:59.868986+0000","last_active":"2024-09-21T06:33:01.966694+0000","last_peered":"2024-09-21T06:33:01.966694+0000","last_clean":"2024-09-21T06:33:01.966694+0000","last_became_active":"2024-09-21T06:32:59.868255+0000","last_became_peered":"2024-09-21T06:32:59.868255+0000","last_unstale":"2024-09-21T06:33:01.966694+0000","last_undegraded":"2024-09-21T06:33:01.966694+0000","last_fullsized":"2024-09-21T06:33:01.966694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:17.234871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967027+0000","last_change":"2024-09-21T06:32:58.847274+0000","last_active":"2024-09-21T06:33:01.967027+0000","last_peered":"2024-09-21T06:33:01.967027+0000","last_clean":"2024-09-21T06:33:01.967027+0000","last_became_active":"2024-09-21T06:32:58.846939+0000","last_became_peered":"2024-09-21T06:32:58.846939+0000","last_unstale":"2024-09-21T06:33:01.967027+0000","last_undegraded":"2024-09-21T06:33:01.967027+0000","last_fullsized":"2024-09-21T06:33:01.967027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:06.394840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966732+0000","last_change":"2024-09-21T06:32:59.854664+0000","last_active":"2024-09-21T06:33:01.966732+0000","last_peered":"2024-09-21T06:33:01.966732+0000","last_clean":"2024-09-21T06:33:01.966732+0000","last_became_active":"2024-09-21T06:32:59.849095+0000","last_became_peered":"2024-09-21T06:32:59.849095+0000","last_unstale":"2024-09-21T06:33:01.966732+0000","last_undegraded":"2024-09-21T06:33:01.966732+0000","last_fullsized":"2024-09-21T06:33:01.966732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:26.470146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968145+0000","last_change":"2024-09-21T06:32:58.839708+0000","last_active":"2024-09-21T06:33:01.968145+0000","last_peered":"2024-09-21T06:33:01.968145+0000","last_clean":"2024-09-21T06:33:01.968145+0000","last_became_active":"2024-09-21T06:32:58.839261+0000","last_became_peered":"2024-09-21T06:32:58.839261+0000","last_unstale":"2024-09-21T06:33:01.968145+0000","last_undegraded":"2024-09-21T06:33:01.968145+0000","last_fullsized":"2024-09-21T06:33:01.968145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:21.684102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967108+0000","last_change":"2024-09-21T06:32:59.850512+0000","last_active":"2024-09-21T06:33:01.967108+0000","last_peered":"2024-09-21T06:33:01.967108+0000","last_clean":"2024-09-21T06:33:01.967108+0000","last_became_active":"2024-09-21T06:32:59.850066+0000","last_became_peered":"2024-09-21T06:32:59.850066+0000","last_unstale":"2024-09-21T06:33:01.967108+0000","last_undegraded":"2024-09-21T06:33:01.967108+0000","last_fullsized":"2024-09-21T06:33:01.967108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.771035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824133+0000","last_change":"2024-09-21T06:32:58.847346+0000","last_active":"2024-09-21T06:33:00.824133+0000","last_peered":"2024-09-21T06:33:00.824133+0000","last_clean":"2024-09-21T06:33:00.824133+0000","last_became_active":"2024-09-21T06:32:58.847025+0000","last_became_peered":"2024-09-21T06:32:58.847025+0000","last_unstale":"2024-09-21T06:33:00.824133+0000","last_undegraded":"2024-09-21T06:33:00.824133+0000","last_fullsized":"2024-09-21T06:33:00.824133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.028650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967393+0000","last_change":"2024-09-21T06:32:59.833499+0000","last_active":"2024-09-21T06:33:01.967393+0000","last_peered":"2024-09-21T06:33:01.967393+0000","last_clean":"2024-09-21T06:33:01.967393+0000","last_became_active":"2024-09-21T06:32:59.833236+0000","last_became_peered":"2024-09-21T06:32:59.833236+0000","last_unstale":"2024-09-21T06:33:01.967393+0000","last_undegraded":"2024-09-21T06:33:01.967393+0000","last_fullsized":"2024-09-21T06:33:01.967393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:55.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968336+0000","last_change":"2024-09-21T06:32:58.840659+0000","last_active":"2024-09-21T06:33:01.968336+0000","last_peered":"2024-09-21T06:33:01.968336+0000","last_clean":"2024-09-21T06:33:01.968336+0000","last_became_active":"2024-09-21T06:32:58.840411+0000","last_became_peered":"2024-09-21T06:32:58.840411+0000","last_unstale":"2024-09-21T06:33:01.968336+0000","last_undegraded":"2024-09-21T06:33:01.968336+0000","last_fullsized":"2024-09-21T06:33:01.968336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:54.846180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.865026+0000","last_change":"2024-09-21T06:32:59.850291+0000","last_active":"2024-09-21T06:32:59.865026+0000","last_peered":"2024-09-21T06:32:59.865026+0000","last_clean":"2024-09-21T06:32:59.865026+0000","last_became_active":"2024-09-21T06:32:59.849723+0000","last_became_peered":"2024-09-21T06:32:59.849723+0000","last_unstale":"2024-09-21T06:32:59.865026+0000","last_undegraded":"2024-09-21T06:32:59.865026+0000","last_fullsized":"2024-09-21T06:32:59.865026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:51.620455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966837+0000","last_change":"2024-09-21T06:32:58.839877+0000","last_active":"2024-09-21T06:33:01.966837+0000","last_peered":"2024-09-21T06:33:01.966837+0000","last_clean":"2024-09-21T06:33:01.966837+0000","last_became_active":"2024-09-21T06:32:58.839316+0000","last_became_peered":"2024-09-21T06:32:58.839316+0000","last_unstale":"2024-09-21T06:33:01.966837+0000","last_undegraded":"2024-09-21T06:33:01.966837+0000","last_fullsized":"2024-09-21T06:33:01.966837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:53.672997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850470+0000","last_change":"2024-09-21T06:32:59.841422+0000","last_active":"2024-09-21T06:33:00.850470+0000","last_peered":"2024-09-21T06:33:00.850470+0000","last_clean":"2024-09-21T06:33:00.850470+0000","last_became_active":"2024-09-21T06:32:59.841042+0000","last_became_peered":"2024-09-21T06:32:59.841042+0000","last_unstale":"2024-09-21T06:33:00.850470+0000","last_undegraded":"2024-09-21T06:33:00.850470+0000","last_fullsized":"2024-09-21T06:33:00.850470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.771035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967745+0000","last_change":"2024-09-21T06:32:58.844502+0000","last_active":"2024-09-21T06:33:01.967745+0000","last_peered":"2024-09-21T06:33:01.967745+0000","last_clean":"2024-09-21T06:33:01.967745+0000","last_became_active":"2024-09-21T06:32:58.843728+0000","last_became_peered":"2024-09-21T06:32:58.843728+0000","last_unstale":"2024-09-21T06:33:01.967745+0000","last_undegraded":"2024-09-21T06:33:01.967745+0000","last_fullsized":"2024-09-21T06:33:01.967745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:38.274905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968624+0000","last_change":"2024-09-21T06:32:59.865377+0000","last_active":"2024-09-21T06:33:01.968624+0000","last_peered":"2024-09-21T06:33:01.968624+0000","last_clean":"2024-09-21T06:33:01.968624+0000","last_became_active":"2024-09-21T06:32:59.864228+0000","last_became_peered":"2024-09-21T06:32:59.864228+0000","last_unstale":"2024-09-21T06:33:01.968624+0000","last_undegraded":"2024-09-21T06:33:01.968624+0000","last_fullsized":"2024-09-21T06:33:01.968624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:20.301779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826830+0000","last_change":"2024-09-21T06:32:58.847972+0000","last_active":"2024-09-21T06:33:00.826830+0000","last_peered":"2024-09-21T06:33:00.826830+0000","last_clean":"2024-09-21T06:33:00.826830+0000","last_became_active":"2024-09-21T06:32:58.847647+0000","last_became_peered":"2024-09-21T06:32:58.847647+0000","last_unstale":"2024-09-21T06:33:00.826830+0000","last_undegraded":"2024-09-21T06:33:00.826830+0000","last_fullsized":"2024-09-21T06:33:00.826830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:38.340252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968245+0000","last_change":"2024-09-21T06:32:59.846260+0000","last_active":"2024-09-21T06:33:01.968245+0000","last_peered":"2024-09-21T06:33:01.968245+0000","last_clean":"2024-09-21T06:33:01.968245+0000","last_became_active":"2024-09-21T06:32:59.845990+0000","last_became_peered":"2024-09-21T06:32:59.845990+0000","last_unstale":"2024-09-21T06:33:01.968245+0000","last_undegraded":"2024-09-21T06:33:01.968245+0000","last_fullsized":"2024-09-21T06:33:01.968245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:29.755064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849533+0000","last_change":"2024-09-21T06:32:58.843605+0000","last_active":"2024-09-21T06:33:00.849533+0000","last_peered":"2024-09-21T06:33:00.849533+0000","last_clean":"2024-09-21T06:33:00.849533+0000","last_became_active":"2024-09-21T06:32:58.843292+0000","last_became_peered":"2024-09-21T06:32:58.843292+0000","last_unstale":"2024-09-21T06:33:00.849533+0000","last_undegraded":"2024-09-21T06:33:00.849533+0000","last_fullsized":"2024-09-21T06:33:00.849533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:54.767330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849533+0000","last_change":"2024-09-21T06:32:59.856336+0000","last_active":"2024-09-21T06:33:00.849533+0000","last_peered":"2024-09-21T06:33:00.849533+0000","last_clean":"2024-09-21T06:33:00.849533+0000","last_became_active":"2024-09-21T06:32:59.851978+0000","last_became_peered":"2024-09-21T06:32:59.851978+0000","last_unstale":"2024-09-21T06:33:00.849533+0000","last_undegraded":"2024-09-21T06:33:00.849533+0000","last_fullsized":"2024-09-21T06:33:00.849533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:21.935781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967707+0000","last_change":"2024-09-21T06:32:58.841901+0000","last_active":"2024-09-21T06:33:01.967707+0000","last_peered":"2024-09-21T06:33:01.967707+0000","last_clean":"2024-09-21T06:33:01.967707+0000","last_became_active":"2024-09-21T06:32:58.841530+0000","last_became_peered":"2024-09-21T06:32:58.841530+0000","last_unstale":"2024-09-21T06:33:01.967707+0000","last_undegraded":"2024-09-21T06:33:01.967707+0000","last_fullsized":"2024-09-21T06:33:01.967707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:48.714380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966750+0000","last_change":"2024-09-21T06:32:59.846646+0000","last_active":"2024-09-21T06:33:01.966750+0000","last_peered":"2024-09-21T06:33:01.966750+0000","last_clean":"2024-09-21T06:33:01.966750+0000","last_became_active":"2024-09-21T06:32:59.846312+0000","last_became_peered":"2024-09-21T06:32:59.846312+0000","last_unstale":"2024-09-21T06:33:01.966750+0000","last_undegraded":"2024-09-21T06:33:01.966750+0000","last_fullsized":"2024-09-21T06:33:01.966750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.586721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824376+0000","last_change":"2024-09-21T06:32:58.841364+0000","last_active":"2024-09-21T06:33:00.824376+0000","last_peered":"2024-09-21T06:33:00.824376+0000","last_clean":"2024-09-21T06:33:00.824376+0000","last_became_active":"2024-09-21T06:32:58.840640+0000","last_became_peered":"2024-09-21T06:32:58.840640+0000","last_unstale":"2024-09-21T06:33:00.824376+0000","last_undegraded":"2024-09-21T06:33:00.824376+0000","last_fullsized":"2024-09-21T06:33:00.824376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:19.360430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827789+0000","last_change":"2024-09-21T06:32:59.844602+0000","last_active":"2024-09-21T06:33:00.827789+0000","last_peered":"2024-09-21T06:33:00.827789+0000","last_clean":"2024-09-21T06:33:00.827789+0000","last_became_active":"2024-09-21T06:32:59.844205+0000","last_became_peered":"2024-09-21T06:32:59.844205+0000","last_unstale":"2024-09-21T06:33:00.827789+0000","last_undegraded":"2024-09-21T06:33:00.827789+0000","last_fullsized":"2024-09-21T06:33:00.827789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:14.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.870410+0000","last_change":"2024-09-21T06:32:58.840210+0000","last_active":"2024-09-21T06:32:59.870410+0000","last_peered":"2024-09-21T06:32:59.870410+0000","last_clean":"2024-09-21T06:32:59.870410+0000","last_became_active":"2024-09-21T06:32:58.839642+0000","last_became_peered":"2024-09-21T06:32:58.839642+0000","last_unstale":"2024-09-21T06:32:59.870410+0000","last_undegraded":"2024-09-21T06:32:59.870410+0000","last_fullsized":"2024-09-21T06:32:59.870410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.838454+0000","last_change":"2024-09-21T06:32:59.832346+0000","last_active":"2024-09-21T06:32:59.838454+0000","last_peered":"2024-09-21T06:32:59.838454+0000","last_clean":"2024-09-21T06:32:59.838454+0000","last_became_active":"2024-09-21T06:32:59.832008+0000","last_became_peered":"2024-09-21T06:32:59.832008+0000","last_unstale":"2024-09-21T06:32:59.838454+0000","last_undegraded":"2024-09-21T06:32:59.838454+0000","last_fullsized":"2024-09-21T06:32:59.838454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:35.305486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968616+0000","last_change":"2024-09-21T06:32:58.840694+0000","last_active":"2024-09-21T06:33:01.968616+0000","last_peered":"2024-09-21T06:33:01.968616+0000","last_clean":"2024-09-21T06:33:01.968616+0000","last_became_active":"2024-09-21T06:32:58.840397+0000","last_became_peered":"2024-09-21T06:32:58.840397+0000","last_unstale":"2024-09-21T06:33:01.968616+0000","last_undegraded":"2024-09-21T06:33:01.968616+0000","last_fullsized":"2024-09-21T06:33:01.968616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:25.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967395+0000","last_change":"2024-09-21T06:32:59.855745+0000","last_active":"2024-09-21T06:33:01.967395+0000","last_peered":"2024-09-21T06:33:01.967395+0000","last_clean":"2024-09-21T06:33:01.967395+0000","last_became_active":"2024-09-21T06:32:59.855443+0000","last_became_peered":"2024-09-21T06:32:59.855443+0000","last_unstale":"2024-09-21T06:33:01.967395+0000","last_undegraded":"2024-09-21T06:33:01.967395+0000","last_fullsized":"2024-09-21T06:33:01.967395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:08.351207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824150+0000","last_change":"2024-09-21T06:32:58.840545+0000","last_active":"2024-09-21T06:33:00.824150+0000","last_peered":"2024-09-21T06:33:00.824150+0000","last_clean":"2024-09-21T06:33:00.824150+0000","last_became_active":"2024-09-21T06:32:58.840087+0000","last_became_peered":"2024-09-21T06:32:58.840087+0000","last_unstale":"2024-09-21T06:33:00.824150+0000","last_undegraded":"2024-09-21T06:33:00.824150+0000","last_fullsized":"2024-09-21T06:33:00.824150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:48.714380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824229+0000","last_change":"2024-09-21T06:32:59.849428+0000","last_active":"2024-09-21T06:33:00.824229+0000","last_peered":"2024-09-21T06:33:00.824229+0000","last_clean":"2024-09-21T06:33:00.824229+0000","last_became_active":"2024-09-21T06:32:59.849114+0000","last_became_peered":"2024-09-21T06:32:59.849114+0000","last_unstale":"2024-09-21T06:33:00.824229+0000","last_undegraded":"2024-09-21T06:33:00.824229+0000","last_fullsized":"2024-09-21T06:33:00.824229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966825+0000","last_change":"2024-09-21T06:32:58.843667+0000","last_active":"2024-09-21T06:33:01.966825+0000","last_peered":"2024-09-21T06:33:01.966825+0000","last_clean":"2024-09-21T06:33:01.966825+0000","last_became_active":"2024-09-21T06:32:58.843141+0000","last_became_peered":"2024-09-21T06:32:58.843141+0000","last_unstale":"2024-09-21T06:33:01.966825+0000","last_undegraded":"2024-09-21T06:33:01.966825+0000","last_fullsized":"2024-09-21T06:33:01.966825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:16.059805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968730+0000","last_change":"2024-09-21T06:32:59.842809+0000","last_active":"2024-09-21T06:33:01.968730+0000","last_peered":"2024-09-21T06:33:01.968730+0000","last_clean":"2024-09-21T06:33:01.968730+0000","last_became_active":"2024-09-21T06:32:59.842433+0000","last_became_peered":"2024-09-21T06:32:59.842433+0000","last_unstale":"2024-09-21T06:33:01.968730+0000","last_undegraded":"2024-09-21T06:33:01.968730+0000","last_fullsized":"2024-09-21T06:33:01.968730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:38.976131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967075+0000","last_change":"2024-09-21T06:32:58.841937+0000","last_active":"2024-09-21T06:33:01.967075+0000","last_peered":"2024-09-21T06:33:01.967075+0000","last_clean":"2024-09-21T06:33:01.967075+0000","last_became_active":"2024-09-21T06:32:58.841628+0000","last_became_peered":"2024-09-21T06:32:58.841628+0000","last_unstale":"2024-09-21T06:33:01.967075+0000","last_undegraded":"2024-09-21T06:33:01.967075+0000","last_fullsized":"2024-09-21T06:33:01.967075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:11.986551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824694+0000","last_change":"2024-09-21T06:32:59.831529+0000","last_active":"2024-09-21T06:33:00.824694+0000","last_peered":"2024-09-21T06:33:00.824694+0000","last_clean":"2024-09-21T06:33:00.824694+0000","last_became_active":"2024-09-21T06:32:59.830948+0000","last_became_peered":"2024-09-21T06:32:59.830948+0000","last_unstale":"2024-09-21T06:33:00.824694+0000","last_undegraded":"2024-09-21T06:33:00.824694+0000","last_fullsized":"2024-09-21T06:33:00.824694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:54.681416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.869374+0000","last_change":"2024-09-21T06:32:58.847929+0000","last_active":"2024-09-21T06:32:59.869374+0000","last_peered":"2024-09-21T06:32:59.869374+0000","last_clean":"2024-09-21T06:32:59.869374+0000","last_became_active":"2024-09-21T06:32:58.847571+0000","last_became_peered":"2024-09-21T06:32:58.847571+0000","last_unstale":"2024-09-21T06:32:59.869374+0000","last_undegraded":"2024-09-21T06:32:59.869374+0000","last_fullsized":"2024-09-21T06:32:59.869374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:31.689104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966913+0000","last_change":"2024-09-21T06:32:59.869317+0000","last_active":"2024-09-21T06:33:01.966913+0000","last_peered":"2024-09-21T06:33:01.966913+0000","last_clean":"2024-09-21T06:33:01.966913+0000","last_became_active":"2024-09-21T06:32:59.868680+0000","last_became_peered":"2024-09-21T06:32:59.868680+0000","last_unstale":"2024-09-21T06:33:01.966913+0000","last_undegraded":"2024-09-21T06:33:01.966913+0000","last_fullsized":"2024-09-21T06:33:01.966913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:39.345334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967820+0000","last_change":"2024-09-21T06:32:58.847303+0000","last_active":"2024-09-21T06:33:01.967820+0000","last_peered":"2024-09-21T06:33:01.967820+0000","last_clean":"2024-09-21T06:33:01.967820+0000","last_became_active":"2024-09-21T06:32:58.846843+0000","last_became_peered":"2024-09-21T06:32:58.846843+0000","last_unstale":"2024-09-21T06:33:01.967820+0000","last_undegraded":"2024-09-21T06:33:01.967820+0000","last_fullsized":"2024-09-21T06:33:01.967820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.028650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849771+0000","last_change":"2024-09-21T06:32:59.852062+0000","last_active":"2024-09-21T06:33:00.849771+0000","last_peered":"2024-09-21T06:33:00.849771+0000","last_clean":"2024-09-21T06:33:00.849771+0000","last_became_active":"2024-09-21T06:32:59.851430+0000","last_became_peered":"2024-09-21T06:32:59.851430+0000","last_unstale":"2024-09-21T06:33:00.849771+0000","last_undegraded":"2024-09-21T06:33:00.849771+0000","last_fullsized":"2024-09-21T06:33:00.849771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.397998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967859+0000","last_change":"2024-09-21T06:32:58.834861+0000","last_active":"2024-09-21T06:33:01.967859+0000","last_peered":"2024-09-21T06:33:01.967859+0000","last_clean":"2024-09-21T06:33:01.967859+0000","last_became_active":"2024-09-21T06:32:58.834041+0000","last_became_peered":"2024-09-21T06:32:58.834041+0000","last_unstale":"2024-09-21T06:33:01.967859+0000","last_undegraded":"2024-09-21T06:33:01.967859+0000","last_fullsized":"2024-09-21T06:33:01.967859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:35.774799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827603+0000","last_change":"2024-09-21T06:32:59.836560+0000","last_active":"2024-09-21T06:33:00.827603+0000","last_peered":"2024-09-21T06:33:00.827603+0000","last_clean":"2024-09-21T06:33:00.827603+0000","last_became_active":"2024-09-21T06:32:59.835815+0000","last_became_peered":"2024-09-21T06:32:59.835815+0000","last_unstale":"2024-09-21T06:33:00.827603+0000","last_undegraded":"2024-09-21T06:33:00.827603+0000","last_fullsized":"2024-09-21T06:33:00.827603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:15.356828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967239+0000","last_change":"2024-09-21T06:32:58.841198+0000","last_active":"2024-09-21T06:33:01.967239+0000","last_peered":"2024-09-21T06:33:01.967239+0000","last_clean":"2024-09-21T06:33:01.967239+0000","last_became_active":"2024-09-21T06:32:58.838947+0000","last_became_peered":"2024-09-21T06:32:58.838947+0000","last_unstale":"2024-09-21T06:33:01.967239+0000","last_undegraded":"2024-09-21T06:33:01.967239+0000","last_fullsized":"2024-09-21T06:33:01.967239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:07.376937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967104+0000","last_change":"2024-09-21T06:32:59.858174+0000","last_active":"2024-09-21T06:33:01.967104+0000","last_peered":"2024-09-21T06:33:01.967104+0000","last_clean":"2024-09-21T06:33:01.967104+0000","last_became_active":"2024-09-21T06:32:59.853755+0000","last_became_peered":"2024-09-21T06:32:59.853755+0000","last_unstale":"2024-09-21T06:33:01.967104+0000","last_undegraded":"2024-09-21T06:33:01.967104+0000","last_fullsized":"2024-09-21T06:33:01.967104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:46.238981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967052+0000","last_change":"2024-09-21T06:32:58.847149+0000","last_active":"2024-09-21T06:33:01.967052+0000","last_peered":"2024-09-21T06:33:01.967052+0000","last_clean":"2024-09-21T06:33:01.967052+0000","last_became_active":"2024-09-21T06:32:58.846529+0000","last_became_peered":"2024-09-21T06:32:58.846529+0000","last_unstale":"2024-09-21T06:33:01.967052+0000","last_undegraded":"2024-09-21T06:33:01.967052+0000","last_fullsized":"2024-09-21T06:33:01.967052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:43.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967151+0000","last_change":"2024-09-21T06:32:59.866301+0000","last_active":"2024-09-21T06:33:01.967151+0000","last_peered":"2024-09-21T06:33:01.967151+0000","last_clean":"2024-09-21T06:33:01.967151+0000","last_became_active":"2024-09-21T06:32:59.865924+0000","last_became_peered":"2024-09-21T06:32:59.865924+0000","last_unstale":"2024-09-21T06:33:01.967151+0000","last_undegraded":"2024-09-21T06:33:01.967151+0000","last_fullsized":"2024-09-21T06:33:01.967151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:28.921748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850075+0000","last_change":"2024-09-21T06:32:58.849629+0000","last_active":"2024-09-21T06:33:00.850075+0000","last_peered":"2024-09-21T06:33:00.850075+0000","last_clean":"2024-09-21T06:33:00.850075+0000","last_became_active":"2024-09-21T06:32:58.848996+0000","last_became_peered":"2024-09-21T06:32:58.848996+0000","last_unstale":"2024-09-21T06:33:00.850075+0000","last_undegraded":"2024-09-21T06:33:00.850075+0000","last_fullsized":"2024-09-21T06:33:00.850075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:16.226451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967314+0000","last_change":"2024-09-21T06:32:59.846339+0000","last_active":"2024-09-21T06:33:01.967314+0000","last_peered":"2024-09-21T06:33:01.967314+0000","last_clean":"2024-09-21T06:33:01.967314+0000","last_became_active":"2024-09-21T06:32:59.845392+0000","last_became_peered":"2024-09-21T06:32:59.845392+0000","last_unstale":"2024-09-21T06:33:01.967314+0000","last_undegraded":"2024-09-21T06:33:01.967314+0000","last_fullsized":"2024-09-21T06:33:01.967314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:39.954509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.826903+0000","last_change":"2024-09-21T06:32:58.850262+0000","last_active":"2024-09-21T06:33:00.826903+0000","last_peered":"2024-09-21T06:33:00.826903+0000","last_clean":"2024-09-21T06:33:00.826903+0000","last_became_active":"2024-09-21T06:32:58.849928+0000","last_became_peered":"2024-09-21T06:32:58.849928+0000","last_unstale":"2024-09-21T06:33:00.826903+0000","last_undegraded":"2024-09-21T06:33:00.826903+0000","last_fullsized":"2024-09-21T06:33:00.826903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:22.635877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968494+0000","last_change":"2024-09-21T06:32:59.841703+0000","last_active":"2024-09-21T06:33:01.968494+0000","last_peered":"2024-09-21T06:33:01.968494+0000","last_clean":"2024-09-21T06:33:01.968494+0000","last_became_active":"2024-09-21T06:32:59.840836+0000","last_became_peered":"2024-09-21T06:32:59.840836+0000","last_unstale":"2024-09-21T06:33:01.968494+0000","last_undegraded":"2024-09-21T06:33:01.968494+0000","last_fullsized":"2024-09-21T06:33:01.968494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:16.727147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849690+0000","last_change":"2024-09-21T06:32:58.843915+0000","last_active":"2024-09-21T06:33:00.849690+0000","last_peered":"2024-09-21T06:33:00.849690+0000","last_clean":"2024-09-21T06:33:00.849690+0000","last_became_active":"2024-09-21T06:32:58.843488+0000","last_became_peered":"2024-09-21T06:32:58.843488+0000","last_unstale":"2024-09-21T06:33:00.849690+0000","last_undegraded":"2024-09-21T06:33:00.849690+0000","last_fullsized":"2024-09-21T06:33:00.849690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.336915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967170+0000","last_change":"2024-09-21T06:32:59.846403+0000","last_active":"2024-09-21T06:33:01.967170+0000","last_peered":"2024-09-21T06:33:01.967170+0000","last_clean":"2024-09-21T06:33:01.967170+0000","last_became_active":"2024-09-21T06:32:59.846045+0000","last_became_peered":"2024-09-21T06:32:59.846045+0000","last_unstale":"2024-09-21T06:33:01.967170+0000","last_undegraded":"2024-09-21T06:33:01.967170+0000","last_fullsized":"2024-09-21T06:33:01.967170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:34.295415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824761+0000","last_change":"2024-09-21T06:32:58.842624+0000","last_active":"2024-09-21T06:33:00.824761+0000","last_peered":"2024-09-21T06:33:00.824761+0000","last_clean":"2024-09-21T06:33:00.824761+0000","last_became_active":"2024-09-21T06:32:58.842191+0000","last_became_peered":"2024-09-21T06:32:58.842191+0000","last_unstale":"2024-09-21T06:33:00.824761+0000","last_undegraded":"2024-09-21T06:33:00.824761+0000","last_fullsized":"2024-09-21T06:33:00.824761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:40.219723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824478+0000","last_change":"2024-09-21T06:32:59.841805+0000","last_active":"2024-09-21T06:33:00.824478+0000","last_peered":"2024-09-21T06:33:00.824478+0000","last_clean":"2024-09-21T06:33:00.824478+0000","last_became_active":"2024-09-21T06:32:59.841523+0000","last_became_peered":"2024-09-21T06:32:59.841523+0000","last_unstale":"2024-09-21T06:33:00.824478+0000","last_undegraded":"2024-09-21T06:33:00.824478+0000","last_fullsized":"2024-09-21T06:33:00.824478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.840820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827344+0000","last_change":"2024-09-21T06:32:58.842291+0000","last_active":"2024-09-21T06:32:59.827344+0000","last_peered":"2024-09-21T06:32:59.827344+0000","last_clean":"2024-09-21T06:32:59.827344+0000","last_became_active":"2024-09-21T06:32:58.841798+0000","last_became_peered":"2024-09-21T06:32:58.841798+0000","last_unstale":"2024-09-21T06:32:59.827344+0000","last_undegraded":"2024-09-21T06:32:59.827344+0000","last_fullsized":"2024-09-21T06:32:59.827344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:41.269562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966759+0000","last_change":"2024-09-21T06:32:59.850214+0000","last_active":"2024-09-21T06:33:01.966759+0000","last_peered":"2024-09-21T06:33:01.966759+0000","last_clean":"2024-09-21T06:33:01.966759+0000","last_became_active":"2024-09-21T06:32:59.849598+0000","last_became_peered":"2024-09-21T06:32:59.849598+0000","last_unstale":"2024-09-21T06:33:01.966759+0000","last_undegraded":"2024-09-21T06:33:01.966759+0000","last_fullsized":"2024-09-21T06:33:01.966759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:34.828987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.824207+0000","last_change":"2024-09-21T06:32:58.848333+0000","last_active":"2024-09-21T06:33:00.824207+0000","last_peered":"2024-09-21T06:33:00.824207+0000","last_clean":"2024-09-21T06:33:00.824207+0000","last_became_active":"2024-09-21T06:32:58.847085+0000","last_became_peered":"2024-09-21T06:32:58.847085+0000","last_unstale":"2024-09-21T06:33:00.824207+0000","last_undegraded":"2024-09-21T06:33:00.824207+0000","last_fullsized":"2024-09-21T06:33:00.824207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:49.153292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.827827+0000","last_change":"2024-09-21T06:32:59.848770+0000","last_active":"2024-09-21T06:33:00.827827+0000","last_peered":"2024-09-21T06:33:00.827827+0000","last_clean":"2024-09-21T06:33:00.827827+0000","last_became_active":"2024-09-21T06:32:59.847200+0000","last_became_peered":"2024-09-21T06:32:59.847200+0000","last_unstale":"2024-09-21T06:33:00.827827+0000","last_undegraded":"2024-09-21T06:33:00.827827+0000","last_fullsized":"2024-09-21T06:33:00.827827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:42.153494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967462+0000","last_change":"2024-09-21T06:32:58.842042+0000","last_active":"2024-09-21T06:33:01.967462+0000","last_peered":"2024-09-21T06:33:01.967462+0000","last_clean":"2024-09-21T06:33:01.967462+0000","last_became_active":"2024-09-21T06:32:58.841730+0000","last_became_peered":"2024-09-21T06:32:58.841730+0000","last_unstale":"2024-09-21T06:33:01.967462+0000","last_undegraded":"2024-09-21T06:33:01.967462+0000","last_fullsized":"2024-09-21T06:33:01.967462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:49.153292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968032+0000","last_change":"2024-09-21T06:32:59.865377+0000","last_active":"2024-09-21T06:33:01.968032+0000","last_peered":"2024-09-21T06:33:01.968032+0000","last_clean":"2024-09-21T06:33:01.968032+0000","last_became_active":"2024-09-21T06:32:59.865025+0000","last_became_peered":"2024-09-21T06:32:59.865025+0000","last_unstale":"2024-09-21T06:33:01.968032+0000","last_undegraded":"2024-09-21T06:33:01.968032+0000","last_fullsized":"2024-09-21T06:33:01.968032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:35.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968160+0000","last_change":"2024-09-21T06:32:58.837354+0000","last_active":"2024-09-21T06:33:01.968160+0000","last_peered":"2024-09-21T06:33:01.968160+0000","last_clean":"2024-09-21T06:33:01.968160+0000","last_became_active":"2024-09-21T06:32:58.836991+0000","last_became_peered":"2024-09-21T06:32:58.836991+0000","last_unstale":"2024-09-21T06:33:01.968160+0000","last_undegraded":"2024-09-21T06:33:01.968160+0000","last_fullsized":"2024-09-21T06:33:01.968160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:11.640799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968339+0000","last_change":"2024-09-21T06:32:59.853900+0000","last_active":"2024-09-21T06:33:01.968339+0000","last_peered":"2024-09-21T06:33:01.968339+0000","last_clean":"2024-09-21T06:33:01.968339+0000","last_became_active":"2024-09-21T06:32:59.853587+0000","last_became_peered":"2024-09-21T06:32:59.853587+0000","last_unstale":"2024-09-21T06:33:01.968339+0000","last_undegraded":"2024-09-21T06:33:01.968339+0000","last_fullsized":"2024-09-21T06:33:01.968339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:59.690110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827931+0000","last_change":"2024-09-21T06:32:58.843925+0000","last_active":"2024-09-21T06:32:59.827931+0000","last_peered":"2024-09-21T06:32:59.827931+0000","last_clean":"2024-09-21T06:32:59.827931+0000","last_became_active":"2024-09-21T06:32:58.843660+0000","last_became_peered":"2024-09-21T06:32:58.843660+0000","last_unstale":"2024-09-21T06:32:59.827931+0000","last_undegraded":"2024-09-21T06:32:59.827931+0000","last_fullsized":"2024-09-21T06:32:59.827931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:22.642787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.850464+0000","last_change":"2024-09-21T06:32:59.853844+0000","last_active":"2024-09-21T06:33:00.850464+0000","last_peered":"2024-09-21T06:33:00.850464+0000","last_clean":"2024-09-21T06:33:00.850464+0000","last_became_active":"2024-09-21T06:32:59.853302+0000","last_became_peered":"2024-09-21T06:32:59.853302+0000","last_unstale":"2024-09-21T06:33:00.850464+0000","last_undegraded":"2024-09-21T06:33:00.850464+0000","last_fullsized":"2024-09-21T06:33:00.850464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:41.902877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.966929+0000","last_change":"2024-09-21T06:32:58.836393+0000","last_active":"2024-09-21T06:33:01.966929+0000","last_peered":"2024-09-21T06:33:01.966929+0000","last_clean":"2024-09-21T06:33:01.966929+0000","last_became_active":"2024-09-21T06:32:58.835402+0000","last_became_peered":"2024-09-21T06:32:58.835402+0000","last_unstale":"2024-09-21T06:33:01.966929+0000","last_undegraded":"2024-09-21T06:33:01.966929+0000","last_fullsized":"2024-09-21T06:33:01.966929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:06.204232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.967272+0000","last_change":"2024-09-21T06:32:59.869158+0000","last_active":"2024-09-21T06:33:01.967272+0000","last_peered":"2024-09-21T06:33:01.967272+0000","last_clean":"2024-09-21T06:33:01.967272+0000","last_became_active":"2024-09-21T06:32:59.868169+0000","last_became_peered":"2024-09-21T06:32:59.868169+0000","last_unstale":"2024-09-21T06:33:01.967272+0000","last_undegraded":"2024-09-21T06:33:01.967272+0000","last_fullsized":"2024-09-21T06:33:01.967272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:11.078357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.849804+0000","last_change":"2024-09-21T06:32:58.842862+0000","last_active":"2024-09-21T06:33:00.849804+0000","last_peered":"2024-09-21T06:33:00.849804+0000","last_clean":"2024-09-21T06:33:00.849804+0000","last_became_active":"2024-09-21T06:32:58.840475+0000","last_became_peered":"2024-09-21T06:32:58.840475+0000","last_unstale":"2024-09-21T06:33:00.849804+0000","last_undegraded":"2024-09-21T06:33:00.849804+0000","last_fullsized":"2024-09-21T06:33:00.849804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:26.802562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:01.968376+0000","last_change":"2024-09-21T06:32:59.847041+0000","last_active":"2024-09-21T06:33:01.968376+0000","last_peered":"2024-09-21T06:33:01.968376+0000","last_clean":"2024-09-21T06:33:01.968376+0000","last_became_active":"2024-09-21T06:32:59.846748+0000","last_became_peered":"2024-09-21T06:32:59.846748+0000","last_unstale":"2024-09-21T06:33:01.968376+0000","last_undegraded":"2024-09-21T06:33:01.968376+0000","last_fullsized":"2024-09-21T06:33:01.968376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:58.809645+0000","last_clean_scrub_stamp":"2024-09-21T06:32:58.809645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:01.671559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:32:59.827837+0000","last_change":"2024-09-21T06:32:58.838303+0000","last_active":"2024-09-21T06:32:59.827837+0000","last_peered":"2024-09-21T06:32:59.827837+0000","last_clean":"2024-09-21T06:32:59.827837+0000","last_became_active":"2024-09-21T06:32:58.837979+0000","last_became_peered":"2024-09-21T06:32:58.837979+0000","last_unstale":"2024-09-21T06:32:59.827837+0000","last_undegraded":"2024-09-21T06:32:59.827837+0000","last_fullsized":"2024-09-21T06:32:59.827837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:32:57.801225+0000","last_clean_scrub_stamp":"2024-09-21T06:32:57.801225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.835519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":219,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T06:33:00.823883+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:33:00.823883+0000","last_peered":"2024-09-21T06:33:00.823883+0000","last_clean":"2024-09-21T06:33:00.823883+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:33:00.823883+0000","last_undegraded":"2024-09-21T06:33:00.823883+0000","last_fullsized":"2024-09-21T06:33:00.823883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509506,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":445293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":445313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":991232,"data_stored":446357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":444641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509508,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1581056,"data_stored":1036655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":1021150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:33:04.842 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:33:04.842 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:33:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 -- 172.21.15.37:0/228067145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86a0004800 msgr2=0x7f86a0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 --2- 172.21.15.37:0/228067145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86a0004800 0x7f86a0004c10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f86a8001690 tx=0x7f86a0005830 comp rx=0 tx=0).stop 2024-09-21T06:33:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 -- 172.21.15.37:0/228067145 shutdown_connections 2024-09-21T06:33:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 --2- 172.21.15.37:0/228067145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86a0004800 0x7f86a0004c10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 -- 172.21.15.37:0/228067145 >> 172.21.15.37:0/228067145 conn(0x7f86b80d0a30 msgr2=0x7f86b80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 -- 172.21.15.37:0/228067145 shutdown_connections 2024-09-21T06:33:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.112+0000 7f86c0800640 1 -- 172.21.15.37:0/228067145 wait complete. 2024-09-21T06:33:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.114+0000 7f86c0800640 1 Processor -- start 2024-09-21T06:33:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.114+0000 7f86c0800640 1 -- start start 2024-09-21T06:33:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86a0004800 0x7f86b81f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86b814da50 0x7f86b81f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f86b80affc0 con 0x7f86b814d570 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86b80afe40 con 0x7f86b814da50 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86c0800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f86b80b0140 con 0x7f86a0004800 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86bd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86bd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54538/0 (socket says 172.21.15.37:54538) 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86bd400640 1 -- 172.21.15.37:0/147351135 learned_addr learned my addr 172.21.15.37:0/147351135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86bde00640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86a0004800 0x7f86b81f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.116+0000 7f86be800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86b814da50 0x7f86b81f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 -- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86a0004800 msgr2=0x7f86b81f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86a0004800 0x7f86b81f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 -- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86b814da50 msgr2=0x7f86b81f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86b814da50 0x7f86b81f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 -- 172.21.15.37:0/147351135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b815a340 con 0x7f86b814d570 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86bd400640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f86a8001780 tx=0x7f86a80063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003a00 con 0x7f86b814d570 2024-09-21T06:33:05.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a8039070 con 0x7f86b814d570 2024-09-21T06:33:05.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8033420 con 0x7f86b814d570 2024-09-21T06:33:05.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b81572c0 con 0x7f86b814d570 2024-09-21T06:33:05.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.117+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b8157880 con 0x7f86b814d570 2024-09-21T06:33:05.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.119+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b8157ac0 con 0x7f86b814d570 2024-09-21T06:33:05.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.120+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86a8044020 con 0x7f86b814d570 2024-09-21T06:33:05.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.121+0000 7f86b2000640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 0x7f869c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.121+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f86a80ba750 con 0x7f86b814d570 2024-09-21T06:33:05.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.121+0000 7f86bde00640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 0x7f869c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.122+0000 7f86bde00640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 0x7f869c07b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f86b806bb20 tx=0x7f86ac005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:05.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.125+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86b8157ac0 con 0x7f86b814d570 2024-09-21T06:33:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.400+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f86b80cfd90 con 0x7f86b814d570 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.405+0000 7f86b2000640 1 -- 172.21.15.37:0/147351135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f86a8082310 con 0x7f86b814d570 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:33:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:33:05.408 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:33:05.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 msgr2=0x7f869c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 0x7f869c07b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f86b806bb20 tx=0x7f86ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 msgr2=0x7f86b8159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f86a8001780 tx=0x7f86a80063a0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 shutdown_connections 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f869c078b90 0x7f869c07b0a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b814d570 0x7f86b8159e00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86b814da50 0x7f86b81f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 --2- 172.21.15.37:0/147351135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86a0004800 0x7f86b81f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 >> 172.21.15.37:0/147351135 conn(0x7f86b80d0a30 msgr2=0x7f86b8151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 shutdown_connections 2024-09-21T06:33:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.408+0000 7f86c0800640 1 -- 172.21.15.37:0/147351135 wait complete. 2024-09-21T06:33:05.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 -- 172.21.15.37:0/2234546482 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c480cc8d0 msgr2=0x7f9c480ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 --2- 172.21.15.37:0/2234546482 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c480cc8d0 0x7f9c480ccce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c40005fd0 tx=0x7f9c4002a820 comp rx=0 tx=0).stop 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 -- 172.21.15.37:0/2234546482 shutdown_connections 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 --2- 172.21.15.37:0/2234546482 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c480cc8d0 0x7f9c480ccce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 --2- 172.21.15.37:0/2234546482 >> v2:172.21.15.150:3300/0 conn(0x7f9c4806e160 0x7f9c4806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 -- 172.21.15.37:0/2234546482 >> 172.21.15.37:0/2234546482 conn(0x7f9c4806f940 msgr2=0x7f9c480b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 -- 172.21.15.37:0/2234546482 shutdown_connections 2024-09-21T06:33:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.660+0000 7f9c47200640 1 -- 172.21.15.37:0/2234546482 wait complete. 2024-09-21T06:33:05.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.663+0000 7f9c47200640 1 Processor -- start 2024-09-21T06:33:05.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.663+0000 7f9c47200640 1 -- start start 2024-09-21T06:33:05.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4806e160 0x7f9c48159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 0x7f9c481f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c480b4c20 con 0x7f9c4815dfe0 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c480b4aa0 con 0x7f9c480cc8d0 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.664+0000 7f9c47200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9c480b4da0 con 0x7f9c4806e160 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54552/0 (socket says 172.21.15.37:54552) 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 -- 172.21.15.37:0/4107950478 learned_addr learned my addr 172.21.15.37:0/4107950478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c45400640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 0x7f9c481f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c45e00640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4806e160 0x7f9c48159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 -- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4806e160 msgr2=0x7f9c48159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4806e160 0x7f9c48159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 -- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 msgr2=0x7f9c481f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 0x7f9c481f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 -- 172.21.15.37:0/4107950478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c481f7d80 con 0x7f9c4815dfe0 2024-09-21T06:33:05.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c45400640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 0x7f9c481f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:05.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.665+0000 7f9c46800640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c40006010 tx=0x7f9c40032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:05.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.666+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40037040 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.666+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c40042070 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.666+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400042a0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.666+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c481f74e0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.666+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c481f7aa0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.667+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c480b5250 con 0x7f9c4815dfe0 2024-09-21T06:33:05.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.668+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c40004440 con 0x7f9c4815dfe0 2024-09-21T06:33:05.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.669+0000 7f9c3e000640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 0x7f9c2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:05.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.669+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f9c400bbb90 con 0x7f9c4815dfe0 2024-09-21T06:33:05.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.670+0000 7f9c45e00640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 0x7f9c2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:05.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.670+0000 7f9c45e00640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 0x7f9c2807b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c38007fd0 tx=0x7f9c38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:05.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.674+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c40083ec0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.912+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c481e86d0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.913+0000 7f9c3e000640 1 -- 172.21.15.37:0/4107950478 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5231 (secure 0 0 0) 0x7f9c400837c0 con 0x7f9c4815dfe0 2024-09-21T06:33:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:05.915 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":72,"btime":"2024-09-21T06:33:02:854262+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6852,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:02.363468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870]},"id":5}]} 2024-09-21T06:33:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 72 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.915+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 msgr2=0x7f9c2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 0x7f9c2807b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c38007fd0 tx=0x7f9c38009040 comp rx=0 tx=0).stop 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 msgr2=0x7f9c481f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c40006010 tx=0x7f9c40032c90 comp rx=0 tx=0).stop 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 shutdown_connections 2024-09-21T06:33:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c28078b90 0x7f9c2807b0a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c4815dfe0 0x7f9c481f3eb0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c480cc8d0 0x7f9c481f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 --2- 172.21.15.37:0/4107950478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4806e160 0x7f9c48159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 >> 172.21.15.37:0/4107950478 conn(0x7f9c4806f940 msgr2=0x7f9c481457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 shutdown_connections 2024-09-21T06:33:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:05.916+0000 7f9c47200640 1 -- 172.21.15.37:0/4107950478 wait complete. 2024-09-21T06:33:06.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:32:59.345604+0000', 'modified': '2024-09-21T06:33:02.363468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/1559036255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1559036255}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1559036255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6870, 'qdb_cluster': [6870]} 2024-09-21T06:33:06.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:33:06.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 -- 172.21.15.37:0/1477724203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc0af5b0 msgr2=0x7f70bc06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 --2- 172.21.15.37:0/1477724203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc0af5b0 0x7f70bc06ddf0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f70bc072a30 tx=0x7f70a402a560 comp rx=0 tx=0).stop 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 -- 172.21.15.37:0/1477724203 shutdown_connections 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 --2- 172.21.15.37:0/1477724203 >> v2:172.21.15.150:3300/0 conn(0x7f70bc06e330 0x7f70bc0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 --2- 172.21.15.37:0/1477724203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc0af5b0 0x7f70bc06ddf0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 --2- 172.21.15.37:0/1477724203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0cc530 0x7f70bc15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 -- 172.21.15.37:0/1477724203 >> 172.21.15.37:0/1477724203 conn(0x7f70bc0b1b90 msgr2=0x7f70bc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 -- 172.21.15.37:0/1477724203 shutdown_connections 2024-09-21T06:33:06.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.571+0000 7f70c0c00640 1 -- 172.21.15.37:0/1477724203 wait complete. 2024-09-21T06:33:06.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.574+0000 7f70c0c00640 1 Processor -- start 2024-09-21T06:33:06.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.574+0000 7f70c0c00640 1 -- start start 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0af5b0 0x7f70bc09c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70bc0cc530 0x7f70bc09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0cadb0 con 0x7f70bc06e330 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0cac30 con 0x7f70bc0cc530 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.575+0000 7f70c0c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f70bc0caf30 con 0x7f70bc0af5b0 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54574/0 (socket says 172.21.15.37:54574) 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 -- 172.21.15.37:0/454311211 learned_addr learned my addr 172.21.15.37:0/454311211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b3e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0af5b0 0x7f70bc09c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70ba800640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70bc0cc530 0x7f70bc09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 -- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0af5b0 msgr2=0x7f70bc09c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0af5b0 0x7f70bc09c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 -- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70bc0cc530 msgr2=0x7f70bc09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70bc0cc530 0x7f70bc09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 -- 172.21.15.37:0/454311211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc1f5530 con 0x7f70bc06e330 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.576+0000 7f70b9e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00d700 tx=0x7f70ac00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.577+0000 7f70b2000640 1 -- 172.21.15.37:0/454311211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac017070 con 0x7f70bc06e330 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.577+0000 7f70b2000640 1 -- 172.21.15.37:0/454311211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70ac005310 con 0x7f70bc06e330 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.577+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70bc09d4a0 con 0x7f70bc06e330 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.577+0000 7f70b2000640 1 -- 172.21.15.37:0/454311211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac009820 con 0x7f70bc06e330 2024-09-21T06:33:06.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.577+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70bc0b3250 con 0x7f70bc06e330 2024-09-21T06:33:06.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.579+0000 7f70b2000640 1 -- 172.21.15.37:0/454311211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70ac0099c0 con 0x7f70bc06e330 2024-09-21T06:33:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.580+0000 7f70b2000640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 0x7f709807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.580+0000 7f70b2000640 1 -- 172.21.15.37:0/454311211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f70ac09b070 con 0x7f70bc06e330 2024-09-21T06:33:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.581+0000 7f70b3e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 0x7f709807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.581+0000 7f70c7be2740 1 Processor -- start 2024-09-21T06:33:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.581+0000 7f70c7be2740 1 -- start start 2024-09-21T06:33:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.581+0000 7f70b3e00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 0x7f709807b0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f70a4033000 tx=0x7f70a4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e27c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e24cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e17e1d6430 con 0x55e17de833b0 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e17e1c65d0 con 0x55e17de8b590 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55e17e1b5a00 con 0x55e17deac270 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.582+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54578/0 (socket says 172.21.15.37:54578) 2024-09-21T06:33:06.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 -- 172.21.15.37:0/850943225 learned_addr learned my addr 172.21.15.37:0/850943225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b9e00640 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e27c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70ba800640 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e24cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 -- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 msgr2=0x55e17e27c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e27c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 -- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 msgr2=0x55e17e24cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e24cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 -- 172.21.15.37:0/850943225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e17e17e520 con 0x55e17de833b0 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.583+0000 7f70b3e00640 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f70a402ab50 tx=0x7f70a403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bd9a0d64a47bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70aa600640 1 -- 172.21.15.37:0/850943225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4035040 con 0x55e17de833b0 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70aa600640 1 -- 172.21.15.37:0/850943225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a404d070 con 0x55e17de833b0 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70aa600640 1 -- 172.21.15.37:0/850943225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4003ce0 con 0x55e17de833b0 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 -- 172.21.15.37:0/850943225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 msgr2=0x55e17e27ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f70a402ab50 tx=0x7f70a403c000 comp rx=0 tx=0).stop 2024-09-21T06:33:06.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 -- 172.21.15.37:0/850943225 shutdown_connections 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e27ceb0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e24cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 --2- 172.21.15.37:0/850943225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e27c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.584+0000 7f70c7be2740 1 -- 172.21.15.37:0/850943225 >> 172.21.15.37:0/850943225 conn(0x55e17dea03c0 msgr2=0x55e17e252970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.585+0000 7f70c7be2740 1 -- 172.21.15.37:0/850943225 shutdown_connections 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.585+0000 7f70c7be2740 1 -- 172.21.15.37:0/850943225 wait complete. 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.586+0000 7f70c7be2740 1 -- start start 2024-09-21T06:33:06.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.586+0000 7f70c7be2740 1 Processor -- start 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e2732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e299e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e17e15c330 con 0x55e17de833b0 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e17e1b9810 con 0x55e17de8b590 2024-09-21T06:33:06.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55e17df26620 con 0x55e17deac270 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.587+0000 7f70c7be2740 10 client.0 _ll_register_callbacks cb 0x55e17e161420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54584/0 (socket says 172.21.15.37:54584) 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 -- 172.21.15.37:0/4070249792 learned_addr learned my addr 172.21.15.37:0/4070249792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b3e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e2732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70ba800640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e299e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 -- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 msgr2=0x55e17e299e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e299e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 -- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 msgr2=0x55e17e2732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e2732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 -- 172.21.15.37:0/4070249792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e17df17850 con 0x55e17de833b0 2024-09-21T06:33:06.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f70b9e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f70ac099370 tx=0x7f70ac0624b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4eaf6f468587f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f709ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.588+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac020040 con 0x55e17de833b0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f70ac061e70 con 0x55e17de833b0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a404d070 con 0x55e17de833b0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f70ac07f020 con 0x55e17de833b0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 10 client.6984 fetch_fsmap learned FSMap version 72 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55e17df8e6b0 con 0x55e17de833b0 2024-09-21T06:33:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a404d070 con 0x55e17de833b0 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7f70ac05e560 con 0x55e17de833b0 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 10 client.6984 fetch_fsmap finished waiting for FSMap version 72 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 10 client.6984 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 20 client.6984 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17de6ae40 0x55e17e2aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 4 client.6984 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.589+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e17e1c8520 con 0x55e17de6ae40 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.590+0000 7f70b3e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17de6ae40 0x55e17e2aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.591+0000 7f70b3e00640 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17de6ae40 0x55e17e2aa360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b849c9330934d2 server_cookie=e4e9dc31d74ab7c0 in_seq=0 out_seq=0 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.591+0000 7f709ea00640 10 client.6984 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f709ea00640 1 -- 172.21.15.37:0/4070249792 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e17e1c8520 con 0x55e17de6ae40 2024-09-21T06:33:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f709ea00640 10 client.6984 handle_command_reply: tid=0 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 client.6984 shutdown 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 msgr2=0x55e17e291d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f70ac099370 tx=0x7f70ac0624b0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 shutdown_connections 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17de6ae40 0x55e17e2aa360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17deac270 0x55e17e299e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e2732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f709ea00640 0 client.6984 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 --2- 172.21.15.37:0/4070249792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de833b0 0x55e17e291d40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 >> 172.21.15.37:0/4070249792 conn(0x55e17dea03c0 msgr2=0x55e17e273ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 shutdown_connections 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 1 -- 172.21.15.37:0/4070249792 wait complete. 2024-09-21T06:33:06.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.592+0000 7f70c7be2740 20 client.6984 trim_cache size 0 max 16384 2024-09-21T06:33:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.612+0000 7f70c7be2740 1 Processor -- start 2024-09-21T06:33:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.612+0000 7f70c7be2740 1 -- start start 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e278790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e278cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc201490 con 0x55e17deac270 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0b4860 con 0x55e17de8b590 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f70bc200e90 con 0x55e17de833b0 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70ba800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70ba800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54596/0 (socket says 172.21.15.37:54596) 2024-09-21T06:33:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.613+0000 7f70ba800640 1 -- 172.21.15.37:0/2548565809 learned_addr learned my addr 172.21.15.37:0/2548565809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70b9e00640 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e278790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70b3e00640 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e278cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 -- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 msgr2=0x55e17e278790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e278790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 -- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 msgr2=0x55e17e278cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e278cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 -- 172.21.15.37:0/2548565809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e17e253650 con 0x55e17deac270 2024-09-21T06:33:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70ba800640 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009e40 tx=0x7f70b40127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c983934a958fac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70a9c00640 1 -- 172.21.15.37:0/2548565809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4004b20 con 0x55e17deac270 2024-09-21T06:33:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70a9c00640 1 -- 172.21.15.37:0/2548565809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70b4004cc0 con 0x55e17deac270 2024-09-21T06:33:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.614+0000 7f70a9c00640 1 -- 172.21.15.37:0/2548565809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4005280 con 0x55e17deac270 2024-09-21T06:33:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 -- 172.21.15.37:0/2548565809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 msgr2=0x55e17e272970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009e40 tx=0x7f70b40127c0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 -- 172.21.15.37:0/2548565809 shutdown_connections 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17deac270 0x55e17e272970 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17de8b590 0x55e17e278cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2548565809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e278790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 -- 172.21.15.37:0/2548565809 >> 172.21.15.37:0/2548565809 conn(0x55e17dea03c0 msgr2=0x55e17e29b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 -- 172.21.15.37:0/2548565809 shutdown_connections 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.615+0000 7f70c7be2740 1 -- 172.21.15.37:0/2548565809 wait complete. 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.616+0000 7f70c7be2740 1 -- start start 2024-09-21T06:33:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.616+0000 7f70c7be2740 1 Processor -- start 2024-09-21T06:33:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e311400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17deac270 0x55e17e29ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e17e2a7a30 con 0x55e17de8b590 2024-09-21T06:33:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e17e273830 con 0x55e17deac270 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55e17e277a20 con 0x55e17de833b0 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70c7be2740 10 client.0 _ll_register_callbacks cb 0x7f70ac062610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54602/0 (socket says 172.21.15.37:54602) 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70b3e00640 1 -- 172.21.15.37:0/2576653826 learned_addr learned my addr 172.21.15.37:0/2576653826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70b9e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e311400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.618+0000 7f70ba800640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17deac270 0x55e17e29ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 -- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 msgr2=0x55e17e311400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e311400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 -- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17deac270 msgr2=0x55e17e29ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17deac270 0x55e17e29ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 -- 172.21.15.37:0/2576653826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e17e27ca40 con 0x55e17de8b590 2024-09-21T06:33:06.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70b3e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004650 tx=0x7f70a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4082679960e0111 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f709ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4035040 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f70a405d8b0 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b4004cc0 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.619+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f70a405dab0 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 10 client.6990 fetch_fsmap learned FSMap version 72 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55e17e278d60 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70b4004cc0 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7f70a403a040 con 0x55e17de8b590 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 10 client.6990 fetch_fsmap finished waiting for FSMap version 72 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 10 client.6990 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 20 client.6990 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:06.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17e248830 0x55e17e29b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 4 client.6990 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "status"}) -- 0x55e17e251fb0 con 0x55e17e248830 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.620+0000 7f70b9e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17e248830 0x55e17e29b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f70b9e00640 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17e248830 0x55e17e29b040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2be1a08f2519e3 server_cookie=a2797f3ba33ced8c in_seq=0 out_seq=0 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f709ea00640 10 client.6990 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f709ea00640 1 -- 172.21.15.37:0/2576653826 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e17e251fb0 con 0x55e17e248830 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f709ea00640 10 client.6990 handle_command_reply: tid=0 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f70c7be2740 1 client.6990 shutdown 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 msgr2=0x55e17e272ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.621+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004650 tx=0x7f70a40046c0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 shutdown_connections 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55e17e248830 0x55e17e29b040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e17de8b590 0x55e17e272ea0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e17deac270 0x55e17e29ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 --2- 172.21.15.37:0/2576653826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e17de833b0 0x55e17e311400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 >> 172.21.15.37:0/2576653826 conn(0x55e17dea03c0 msgr2=0x55e17e2a5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f709ea00640 0 client.6990 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 shutdown_connections 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 1 -- 172.21.15.37:0/2576653826 wait complete. 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.622+0000 7f70c7be2740 20 client.6990 trim_cache size 0 max 16384 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:33:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6870, 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 6.7906339349999998, 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 72, 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 80, 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.760008074 2024-09-21T06:33:06.628 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 msgr2=0x7f709807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 0x7f709807b0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f70a4033000 tx=0x7f70a4004730 comp rx=0 tx=0).stop 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 msgr2=0x7f70bc1f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00d700 tx=0x7f70ac00dbc0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 shutdown_connections 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7098078b90 0x7f709807b0a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70bc0cc530 0x7f70bc09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70bc0af5b0 0x7f70bc09c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 --2- 172.21.15.37:0/454311211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70bc06e330 0x7f70bc1f4ff0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 >> 172.21.15.37:0/454311211 conn(0x7f70bc0b1b90 msgr2=0x7f70bc0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 shutdown_connections 2024-09-21T06:33:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.623+0000 7f70c0c00640 1 -- 172.21.15.37:0/454311211 wait complete. 2024-09-21T06:33:06.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:33:06.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 -- 172.21.15.37:0/3887098690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 msgr2=0x7f629c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 --2- 172.21.15.37:0/3887098690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c0cf930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c02aaa0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 -- 172.21.15.37:0/3887098690 shutdown_connections 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 --2- 172.21.15.37:0/3887098690 >> v2:172.21.15.150:3300/0 conn(0x7f629c0cefa0 0x7f629c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 --2- 172.21.15.37:0/3887098690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c0cf930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 -- 172.21.15.37:0/3887098690 >> 172.21.15.37:0/3887098690 conn(0x7f629c0cca40 msgr2=0x7f629c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 -- 172.21.15.37:0/3887098690 shutdown_connections 2024-09-21T06:33:06.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.873+0000 7f62a1000640 1 -- 172.21.15.37:0/3887098690 wait complete. 2024-09-21T06:33:06.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.875+0000 7f62a1000640 1 Processor -- start 2024-09-21T06:33:06.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.876+0000 7f62a1000640 1 -- start start 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f629c0cefa0 0x7f629c156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f629c06df70 con 0x7f629c0ce860 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f629c06ddf0 con 0x7f629c0cefa0 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f62a1000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f629c06e0f0 con 0x7f629c15b9f0 2024-09-21T06:33:06.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f6299e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f6299e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54616/0 (socket says 172.21.15.37:54616) 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f6299e00640 1 -- 172.21.15.37:0/2231064343 learned_addr learned my addr 172.21.15.37:0/2231064343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f6291400640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f629c0cefa0 0x7f629c156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.877+0000 7f629a800640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 -- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 msgr2=0x7f629c156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 -- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f629c0cefa0 msgr2=0x7f629c156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f629c0cefa0 0x7f629c156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 -- 172.21.15.37:0/2231064343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c15d5e0 con 0x7f629c0ce860 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f629a800640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:06.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6299e00640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f628c009bf0 tx=0x7f628c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c044070 con 0x7f629c0ce860 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f628c004250 con 0x7f629c0ce860 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c004660 con 0x7f629c0ce860 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.878+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c15d8c0 con 0x7f629c0ce860 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.879+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c15de80 con 0x7f629c0ce860 2024-09-21T06:33:06.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.880+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c06e610 con 0x7f629c0ce860 2024-09-21T06:33:06.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.881+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f628c047070 con 0x7f629c0ce860 2024-09-21T06:33:06.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.883+0000 7f6292a00640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 0x7f627807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:06.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.883+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f628c0bbe30 con 0x7f629c0ce860 2024-09-21T06:33:06.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.883+0000 7f6291400640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 0x7f627807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:06.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.884+0000 7f6291400640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 0x7f627807b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b1a20 tx=0x7f6284005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:06.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:06.887+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f628c032030 con 0x7f629c0ce860 2024-09-21T06:33:07.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.123+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f629c073ec0 con 0x7f629c0ce860 2024-09-21T06:33:07.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.123+0000 7f6292a00640 1 -- 172.21.15.37:0/2231064343 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7f628c0839f0 con 0x7f629c0ce860 2024-09-21T06:33:07.126 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 msgr2=0x7f627807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 0x7f627807b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b1a20 tx=0x7f6284005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 msgr2=0x7f629c1f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f628c009bf0 tx=0x7f628c003ce0 comp rx=0 tx=0).stop 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 shutdown_connections 2024-09-21T06:33:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6278078bc0 0x7f627807b0d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f629c15b9f0 0x7f629c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f629c0cefa0 0x7f629c156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 --2- 172.21.15.37:0/2231064343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f629c0ce860 0x7f629c1f5340 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 >> 172.21.15.37:0/2231064343 conn(0x7f629c0cca40 msgr2=0x7f629c072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 shutdown_connections 2024-09-21T06:33:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07.126+0000 7f62a1000640 1 -- 172.21.15.37:0/2231064343 wait complete. 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:33:07.151 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:07.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.152 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:33:07.188 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42404sec preferred_lft 42404sec 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:33:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:33:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:07.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.191 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:33:07.282 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:33:07.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.285 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:33:07.321 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:33:07.321 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:33:07.322 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:33:07.322 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:33:07.322 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:33:07.322 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:33:07.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:33:07.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:33:07.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.519 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:33:07.554 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:33:07.554 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:33:07.555 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:07.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:33:07.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:33:07.651 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:07.687 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:07.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:33:07.687 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:07.762 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:33:07.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:07.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:07.964 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:07.964 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:07.964 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:07.985 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:07.985 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:07.985 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:07.985 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:33:07.985 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:33:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:33:08.021 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:33:08.021 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:33:08.022 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:33:08.022 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:33:08.022 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:33:08.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:33:08.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:08.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:08.081 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:33:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:33:08.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:33:08.184 INFO:teuthology.orchestra.run.smithi037.stdout:client6772 2024-09-21T06:33:08.184 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:08.184 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:08.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:33:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.490+0000 7fbf87200640 1 -- 172.21.15.37:0/1091185814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf68003a50 msgr2=0x7fbf68003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.490+0000 7fbf87200640 1 --2- 172.21.15.37:0/1091185814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf68003a50 0x7fbf68003ec0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009b50 tx=0x7fbf68005830 comp rx=0 tx=0).stop 2024-09-21T06:33:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.490+0000 7fbf87200640 1 -- 172.21.15.37:0/1091185814 shutdown_connections 2024-09-21T06:33:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.490+0000 7fbf87200640 1 --2- 172.21.15.37:0/1091185814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf68003a50 0x7fbf68003ec0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.490+0000 7fbf87200640 1 -- 172.21.15.37:0/1091185814 >> 172.21.15.37:0/1091185814 conn(0x7fbf8006f920 msgr2=0x7fbf800b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.491+0000 7fbf87200640 1 -- 172.21.15.37:0/1091185814 shutdown_connections 2024-09-21T06:33:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.491+0000 7fbf87200640 1 -- 172.21.15.37:0/1091185814 wait complete. 2024-09-21T06:33:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.493+0000 7fbf87200640 1 Processor -- start 2024-09-21T06:33:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.493+0000 7fbf87200640 1 -- start start 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf68003a50 0x7fbf8013b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8015f2c0 0x7fbf8015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800ccce0 con 0x7fbf801f7640 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800b5300 con 0x7fbf8015f2c0 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.494+0000 7fbf87200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbf800cce60 con 0x7fbf68003a50 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:08.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54630/0 (socket says 172.21.15.37:54630) 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 -- 172.21.15.37:0/2208660449 learned_addr learned my addr 172.21.15.37:0/2208660449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf86800640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8015f2c0 0x7fbf8015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85e00640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf68003a50 0x7fbf8013b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 -- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf68003a50 msgr2=0x7fbf8013b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf68003a50 0x7fbf8013b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 -- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8015f2c0 msgr2=0x7fbf8015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8015f2c0 0x7fbf8015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 -- 172.21.15.37:0/2208660449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8015fc10 con 0x7fbf801f7640 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.495+0000 7fbf85400640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e510 tx=0x7fbf7000e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.496+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70015070 con 0x7fbf801f7640 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.496+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf70010980 con 0x7fbf801f7640 2024-09-21T06:33:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.496+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8013be30 con 0x7fbf801f7640 2024-09-21T06:33:08.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.496+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70010dd0 con 0x7fbf801f7640 2024-09-21T06:33:08.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.496+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8013c3f0 con 0x7fbf801f7640 2024-09-21T06:33:08.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.497+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8006d780 con 0x7fbf801f7640 2024-09-21T06:33:08.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.498+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf70018070 con 0x7fbf801f7640 2024-09-21T06:33:08.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.500+0000 7fbf7a000640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 0x7fbf6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:08.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.500+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fbf7009b270 con 0x7fbf801f7640 2024-09-21T06:33:08.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.500+0000 7fbf85e00640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 0x7fbf6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:08.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.501+0000 7fbf85e00640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 0x7fbf6407b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf74035000 tx=0x7fbf740046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:08.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.504+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf8015fc10 con 0x7fbf801f7640 2024-09-21T06:33:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.745+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbf800b20a0 con 0x7fbf801f7640 2024-09-21T06:33:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.797+0000 7fbf7a000640 1 -- 172.21.15.37:0/2208660449 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v73) ==== 105+0+0 (secure 0 0 0) 0x7fbf70063480 con 0x7fbf801f7640 2024-09-21T06:33:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 msgr2=0x7fbf6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 0x7fbf6407b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf74035000 tx=0x7fbf740046c0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 msgr2=0x7fbf80138a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e510 tx=0x7fbf7000e9d0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 shutdown_connections 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbf64078b90 0x7fbf6407b0a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf801f7640 0x7fbf80138a00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf8015f2c0 0x7fbf8015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 --2- 172.21.15.37:0/2208660449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf68003a50 0x7fbf8013b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 >> 172.21.15.37:0/2208660449 conn(0x7fbf8006f920 msgr2=0x7fbf800b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 shutdown_connections 2024-09-21T06:33:08.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:08.800+0000 7fbf87200640 1 -- 172.21.15.37:0/2208660449 wait complete. 2024-09-21T06:33:08.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.057+0000 7f1fe8e00640 1 -- 172.21.15.37:0/1385616320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fc8004d90 msgr2=0x7f1fc8005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.057+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/1385616320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fc8004d90 0x7f1fc8005240 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00c640 tx=0x7f1fc8005830 comp rx=0 tx=0).stop 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.057+0000 7f1fe3e00640 1 -- 172.21.15.37:0/1385616320 reap_dead start 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.057+0000 7f1fe8e00640 1 -- 172.21.15.37:0/1385616320 shutdown_connections 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.058+0000 7f1fe8e00640 1 -- 172.21.15.37:0/1385616320 >> 172.21.15.37:0/1385616320 conn(0x7f1fe406f940 msgr2=0x7f1fe40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.058+0000 7f1fe8e00640 1 -- 172.21.15.37:0/1385616320 shutdown_connections 2024-09-21T06:33:09.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.058+0000 7f1fe8e00640 1 -- 172.21.15.37:0/1385616320 wait complete. 2024-09-21T06:33:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.060+0000 7f1fe8e00640 1 Processor -- start 2024-09-21T06:33:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.060+0000 7f1fe8e00640 1 -- start start 2024-09-21T06:33:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc8004d90 0x7f1fe40b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4142200 0x7f1fe415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41635d0 con 0x7f1fe415ba70 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4163450 con 0x7f1fe4142200 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.061+0000 7f1fe8e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe4163750 con 0x7f1fc8004d90 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54648/0 (socket says 172.21.15.37:54648) 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 -- 172.21.15.37:0/3485492691 learned_addr learned my addr 172.21.15.37:0/3485492691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3400640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc8004d90 0x7f1fe40b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fdbe00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4142200 0x7f1fe415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 -- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc8004d90 msgr2=0x7f1fe40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc8004d90 0x7f1fe40b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 -- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4142200 msgr2=0x7f1fe415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4142200 0x7f1fe415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.062+0000 7f1fe3e00640 1 -- 172.21.15.37:0/3485492691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41f9bb0 con 0x7f1fe415ba70 2024-09-21T06:33:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fe3e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00c760 tx=0x7f1fdc001480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc039070 con 0x7f1fe415ba70 2024-09-21T06:33:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fdc02b750 con 0x7f1fe415ba70 2024-09-21T06:33:09.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc02bba0 con 0x7f1fe415ba70 2024-09-21T06:33:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe415be80 con 0x7f1fe415ba70 2024-09-21T06:33:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.063+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41f9310 con 0x7f1fe415ba70 2024-09-21T06:33:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.064+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4160a40 con 0x7f1fe415ba70 2024-09-21T06:33:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.065+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1fdc03c070 con 0x7f1fe415ba70 2024-09-21T06:33:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.066+0000 7f1fdb400640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.066+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1fdc0ba3a0 con 0x7f1fe415ba70 2024-09-21T06:33:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.067+0000 7f1fe3400640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.067+0000 7f1fe3400640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 0x7f1fc007b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:09.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.071+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1fe415be80 con 0x7f1fe415ba70 2024-09-21T06:33:09.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.305+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fe40cc0e0 con 0x7f1fe415ba70 2024-09-21T06:33:09.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.305+0000 7f1fdb400640 1 -- 172.21.15.37:0/3485492691 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5247 (secure 0 0 0) 0x7f1fdc0820b0 con 0x7f1fe415ba70 2024-09-21T06:33:09.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:09.308 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":74,"btime":"2024-09-21T06:33:08:801748+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6852,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:08.801733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6870,"mds_1":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:creating","state_seq":1,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870]},"id":5}]} 2024-09-21T06:33:09.308 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 74 2024-09-21T06:33:09.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 msgr2=0x7f1fc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:09.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 0x7f1fc007b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006b10 comp rx=0 tx=0).stop 2024-09-21T06:33:09.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 msgr2=0x7f1fe40a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00c760 tx=0x7f1fdc001480 comp rx=0 tx=0).stop 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 shutdown_connections 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fe415ba70 0x7f1fe40a96e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4142200 0x7f1fe415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 --2- 172.21.15.37:0/3485492691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc8004d90 0x7f1fe40b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 >> 172.21.15.37:0/3485492691 conn(0x7f1fe406f940 msgr2=0x7f1fe415ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 shutdown_connections 2024-09-21T06:33:09.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:09.308+0000 7f1fe8e00640 1 -- 172.21.15.37:0/3485492691 wait complete. 2024-09-21T06:33:09.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:32:59.345604+0000', 'modified': '2024-09-21T06:33:08.801733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6870, 'mds_1': 6858}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6858': {'gid': 6858, 'name': 'c', 'rank': 1, 'incarnation': 74, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.37:6835/2799311131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2799311131}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2799311131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6870': {'gid': 6870, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/1559036255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1559036255}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1559036255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6870, 'qdb_cluster': [6870]} 2024-09-21T06:33:09.739 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6858:up:creating 2024-09-21T06:33:10.740 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 -- 172.21.15.37:0/3301738860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10074d90 msgr2=0x7f5c1006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 --2- 172.21.15.37:0/3301738860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10074d90 0x7f5c1006f710 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009830 tx=0x7f5c0002a500 comp rx=0 tx=0).stop 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 -- 172.21.15.37:0/3301738860 shutdown_connections 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 --2- 172.21.15.37:0/3301738860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c1006e500 0x7f5c100cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 --2- 172.21.15.37:0/3301738860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10074d90 0x7f5c1006f710 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 -- 172.21.15.37:0/3301738860 >> 172.21.15.37:0/3301738860 conn(0x7f5c100cca40 msgr2=0x7f5c100cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:10.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 -- 172.21.15.37:0/3301738860 shutdown_connections 2024-09-21T06:33:10.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.977+0000 7f5c16600640 1 -- 172.21.15.37:0/3301738860 wait complete. 2024-09-21T06:33:10.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.980+0000 7f5c16600640 1 Processor -- start 2024-09-21T06:33:10.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.980+0000 7f5c16600640 1 -- start start 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 0x7f5c100c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c10074d90 0x7f5c100c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10070e80 con 0x7f5c10155bc0 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10071000 con 0x7f5c10074d90 2024-09-21T06:33:10.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c16600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5c100ce0d0 con 0x7f5c10070030 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.981+0000 7f5c0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54670/0 (socket says 172.21.15.37:54670) 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 -- 172.21.15.37:0/2200046649 learned_addr learned my addr 172.21.15.37:0/2200046649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0f400640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 0x7f5c100c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0ea00640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c10074d90 0x7f5c100c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 -- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 msgr2=0x7f5c100c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 0x7f5c100c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 -- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c10074d90 msgr2=0x7f5c100c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c10074d90 0x7f5c100c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 -- 172.21.15.37:0/2200046649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1015d750 con 0x7f5c10155bc0 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0f400640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 0x7f5c100c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:10.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c0fe00640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e990 tx=0x7f5c0800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.982+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08004d20 con 0x7f5c10155bc0 2024-09-21T06:33:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.983+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c0800caa0 con 0x7f5c10155bc0 2024-09-21T06:33:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.983+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08005020 con 0x7f5c10155bc0 2024-09-21T06:33:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.983+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1015da30 con 0x7f5c10155bc0 2024-09-21T06:33:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.983+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1009f4c0 con 0x7f5c10155bc0 2024-09-21T06:33:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.984+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1009f720 con 0x7f5c10155bc0 2024-09-21T06:33:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.985+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c080051c0 con 0x7f5c10155bc0 2024-09-21T06:33:10.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.986+0000 7f5c07400640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 0x7f5bf007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:10.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.986+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f5c0809b7c0 con 0x7f5c10155bc0 2024-09-21T06:33:10.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.987+0000 7f5c0f400640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 0x7f5bf007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:10.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.987+0000 7f5c0f400640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 0x7f5bf007b0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c00004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:10.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:10.990+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c08016030 con 0x7f5c10155bc0 2024-09-21T06:33:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.229+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c100ce660 con 0x7f5c10155bc0 2024-09-21T06:33:11.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.230+0000 7f5c07400640 1 -- 172.21.15.37:0/2200046649 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5269 (secure 0 0 0) 0x7f5c08063a60 con 0x7f5c10155bc0 2024-09-21T06:33:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:11.232 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":77,"btime":"2024-09-21T06:33:10:816718+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:10.816711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6852,6858,6870]},"id":5}]} 2024-09-21T06:33:11.232 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 77 2024-09-21T06:33:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 msgr2=0x7f5bf007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 0x7f5bf007b0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c00004730 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 msgr2=0x7f5c1015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e990 tx=0x7f5c0800ee50 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 shutdown_connections 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bf0078bc0 0x7f5bf007b0d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c10155bc0 0x7f5c1015c460 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c10074d90 0x7f5c100c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 --2- 172.21.15.37:0/2200046649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c10070030 0x7f5c100c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 >> 172.21.15.37:0/2200046649 conn(0x7f5c100cca40 msgr2=0x7f5c100b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 shutdown_connections 2024-09-21T06:33:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.232+0000 7f5c16600640 1 -- 172.21.15.37:0/2200046649 wait complete. 2024-09-21T06:33:11.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:32:59.345604+0000', 'modified': '2024-09-21T06:33:10.816711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6870, 'mds_1': 6858, 'mds_2': 6852}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6852': {'gid': 6852, 'name': 'a', 'rank': 2, 'incarnation': 76, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6833/3081570660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3081570660}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 3081570660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6858': {'gid': 6858, 'name': 'c', 'rank': 1, 'incarnation': 74, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6835/2799311131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2799311131}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2799311131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6870': {'gid': 6870, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6837/1559036255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1559036255}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1559036255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6870, 'qdb_cluster': [6852, 6858, 6870]} 2024-09-21T06:33:11.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:33:11.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:33:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 -- 172.21.15.37:0/2355703874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cc0e0 msgr2=0x7f41dc0702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 --2- 172.21.15.37:0/2355703874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc0702a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009830 tx=0x7f41cc02a530 comp rx=0 tx=0).stop 2024-09-21T06:33:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 -- 172.21.15.37:0/2355703874 shutdown_connections 2024-09-21T06:33:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 --2- 172.21.15.37:0/2355703874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc0702a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 -- 172.21.15.37:0/2355703874 >> 172.21.15.37:0/2355703874 conn(0x7f41dc0d05c0 msgr2=0x7f41dc06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.888+0000 7f41e1200640 1 -- 172.21.15.37:0/2355703874 shutdown_connections 2024-09-21T06:33:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.889+0000 7f41e1200640 1 -- 172.21.15.37:0/2355703874 wait complete. 2024-09-21T06:33:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.891+0000 7f41e1200640 1 Processor -- start 2024-09-21T06:33:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.891+0000 7f41e1200640 1 -- start start 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 0x7f41dc133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 0x7f41dc1ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc0b3e50 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc0b3cd0 con 0x7f41dc0cf6d0 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41e1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc0b3fd0 con 0x7f41dc1ea450 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54680/0 (socket says 172.21.15.37:54680) 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.892+0000 7f41db400640 1 -- 172.21.15.37:0/998559931 learned_addr learned my addr 172.21.15.37:0/998559931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41d2a00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 0x7f41dc133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41dbe00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 0x7f41dc1ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 -- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 msgr2=0x7f41dc1ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 0x7f41dc1ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 -- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 msgr2=0x7f41dc133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 0x7f41dc133810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 -- 172.21.15.37:0/998559931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc1eada0 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41d2a00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 0x7f41dc133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41dbe00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 0x7f41dc1ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.893+0000 7f41db400640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02ab20 tx=0x7f41cc033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.894+0000 7f41d3e00640 1 -- 172.21.15.37:0/998559931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc031b50 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.894+0000 7f41d3e00640 1 -- 172.21.15.37:0/998559931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41cc031de0 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.894+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc142cb0 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.894+0000 7f41d3e00640 1 -- 172.21.15.37:0/998559931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc005280 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.894+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1434a0 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.896+0000 7f41d3e00640 1 -- 172.21.15.37:0/998559931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41cc005420 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.898+0000 7f41d3e00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 0x7f41b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.898+0000 7f41d3e00640 1 -- 172.21.15.37:0/998559931 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f41cc0bc0f0 con 0x7f41dc0cc0e0 2024-09-21T06:33:11.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.898+0000 7f41d2a00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 0x7f41b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.898+0000 7f41e9840740 1 Processor -- start 2024-09-21T06:33:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.899+0000 7f41e9840740 1 -- start start 2024-09-21T06:33:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.899+0000 7f41d2a00640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 0x7f41b807b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f41dc06d3a0 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 0x564dfebe3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7f2590 0x564dfebb4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564dfeb3d430 con 0x564dfe7ea3b0 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564dfeb2d5d0 con 0x564dfe813270 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564dfeb1ca00 con 0x564dfe7f2590 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54692/0 (socket says 172.21.15.37:54692) 2024-09-21T06:33:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41db400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 0x564dfebe3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41d2a00640 1 -- 172.21.15.37:0/1938163002 learned_addr learned my addr 172.21.15.37:0/1938163002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.900+0000 7f41dbe00640 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7f2590 0x564dfebb4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 -- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7f2590 msgr2=0x564dfebb4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7f2590 0x564dfebb4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 -- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 msgr2=0x564dfebe3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 0x564dfebe3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 -- 172.21.15.37:0/1938163002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfeae5520 con 0x564dfe7ea3b0 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41db400640 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 0x564dfebe3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:33:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001500 tx=0x7f41c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9421f99634b7a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41cbe00640 1 -- 172.21.15.37:0/1938163002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4025070 con 0x564dfe7ea3b0 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41cbe00640 1 -- 172.21.15.37:0/1938163002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c4015700 con 0x564dfe7ea3b0 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.901+0000 7f41cbe00640 1 -- 172.21.15.37:0/1938163002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4015ab0 con 0x564dfe7ea3b0 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 -- 172.21.15.37:0/1938163002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 msgr2=0x564dfebe40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001500 tx=0x7f41c4011000 comp rx=0 tx=0).stop 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 -- 172.21.15.37:0/1938163002 shutdown_connections 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfebe40a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7f2590 0x564dfebb4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 --2- 172.21.15.37:0/1938163002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe813270 0x564dfebe3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 -- 172.21.15.37:0/1938163002 >> 172.21.15.37:0/1938163002 conn(0x564dfe8073c0 msgr2=0x564dfebb9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 -- 172.21.15.37:0/1938163002 shutdown_connections 2024-09-21T06:33:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.902+0000 7f41e9840740 1 -- 172.21.15.37:0/1938163002 wait complete. 2024-09-21T06:33:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.903+0000 7f41e9840740 1 -- start start 2024-09-21T06:33:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.904+0000 7f41e9840740 1 Processor -- start 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7ea3b0 0x564dfebf9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfec10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564dfeac3330 con 0x564dfe7f2590 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564dfeb20810 con 0x564dfe7ea3b0 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564dfe88d620 con 0x564dfe813270 2024-09-21T06:33:11.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41e9840740 10 client.0 _ll_register_callbacks cb 0x564dfeac8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54696/0 (socket says 172.21.15.37:54696) 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.905+0000 7f41d2a00640 1 -- 172.21.15.37:0/603538019 learned_addr learned my addr 172.21.15.37:0/603538019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41db400640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7ea3b0 0x564dfebf9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41dbe00640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfec10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 -- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 msgr2=0x564dfec10eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfec10eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 -- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7ea3b0 msgr2=0x564dfebf9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7ea3b0 0x564dfebf9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 -- 172.21.15.37:0/603538019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dfe87e850 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41d2a00640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f41c4020000 tx=0x7f41c4005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad634b0fee48a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4025040 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f41c402bc80 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c4015700 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.906+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f41c4032050 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 10 client.7026 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x564dfe8f56b0 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c4015700 con 0x564dfe7f2590 2024-09-21T06:33:11.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7f41c4035020 con 0x564dfe7f2590 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 10 client.7026 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 10 client.7026 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 20 client.7026 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfe7d1e40 0x564dfebdf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 4 client.7026 mds_command: new command op to 6852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 --> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dfeb2f520 con 0x564dfe7d1e40 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.907+0000 7f41db400640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfe7d1e40 0x564dfebdf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.908+0000 7f41db400640 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfe7d1e40 0x564dfebdf5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d93201b9df5a1d0b server_cookie=b3a876fc23981e51 in_seq=0 out_seq=0 2024-09-21T06:33:11.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.908+0000 7f41c8c00640 10 client.7026 ms_handle_connect on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41c8c00640 1 -- 172.21.15.37:0/603538019 <== mds.2 v2:172.21.15.37:6832/3081570660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564dfeb2f520 con 0x564dfe7d1e40 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41c8c00640 10 client.7026 handle_command_reply: tid=0 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 client.7026 shutdown 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 msgr2=0x564dfebffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f41c4020000 tx=0x7f41c4005400 comp rx=0 tx=0).stop 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 shutdown_connections 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfe7d1e40 0x564dfebdf5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7f2590 0x564dfebffca0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfec10eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 --2- 172.21.15.37:0/603538019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7ea3b0 0x564dfebf9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41c8c00640 0 client.7026 ms_handle_reset on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:11.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 >> 172.21.15.37:0/603538019 conn(0x564dfe8073c0 msgr2=0x564dfec017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 shutdown_connections 2024-09-21T06:33:11.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.909+0000 7f41e9840740 1 -- 172.21.15.37:0/603538019 wait complete. 2024-09-21T06:33:11.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.910+0000 7f41e9840740 20 client.7026 trim_cache size 0 max 16384 2024-09-21T06:33:11.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.929+0000 7f41e9840740 1 Processor -- start 2024-09-21T06:33:11.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.929+0000 7f41e9840740 1 -- start start 2024-09-21T06:33:11.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 0x564dfebb52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfec217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc2016e0 con 0x564dfe813270 2024-09-21T06:33:11.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc201db0 con 0x564dfe7f2590 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.930+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc1f9000 con 0x564dfe7ea3b0 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54710/0 (socket says 172.21.15.37:54710) 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 -- 172.21.15.37:0/1032033177 learned_addr learned my addr 172.21.15.37:0/1032033177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41db400640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 0x564dfebb52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfec217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 -- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 msgr2=0x564dfebb52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 0x564dfebb52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 -- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 msgr2=0x564dfec217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfec217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 -- 172.21.15.37:0/1032033177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfebda6e0 con 0x564dfe813270 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41db400640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 0x564dfebb52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfec217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.931+0000 7f41dbe00640 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f41d4003d70 tx=0x7f41d4001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0e38b9c1260276 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41cb400640 1 -- 172.21.15.37:0/1032033177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4005440 con 0x564dfe813270 2024-09-21T06:33:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41cb400640 1 -- 172.21.15.37:0/1032033177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d40056d0 con 0x564dfe813270 2024-09-21T06:33:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41cb400640 1 -- 172.21.15.37:0/1032033177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4016670 con 0x564dfe813270 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 -- 172.21.15.37:0/1032033177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 msgr2=0x564dfec21d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f41d4003d70 tx=0x7f41d4001d10 comp rx=0 tx=0).stop 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 -- 172.21.15.37:0/1032033177 shutdown_connections 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe813270 0x564dfec21d20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfec217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 --2- 172.21.15.37:0/1032033177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe7ea3b0 0x564dfebb52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.932+0000 7f41e9840740 1 -- 172.21.15.37:0/1032033177 >> 172.21.15.37:0/1032033177 conn(0x564dfe8073c0 msgr2=0x564dfebcd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.933+0000 7f41e9840740 1 -- 172.21.15.37:0/1032033177 shutdown_connections 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.933+0000 7f41e9840740 1 -- 172.21.15.37:0/1032033177 wait complete. 2024-09-21T06:33:11.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.933+0000 7f41e9840740 1 -- start start 2024-09-21T06:33:11.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.934+0000 7f41e9840740 1 Processor -- start 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfebdf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfebdf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564dfec11bc0 con 0x564dfe7ea3b0 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564dfebe0600 con 0x564dfe7f2590 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x564dfebdd1f0 con 0x564dfe813270 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41e9840740 10 client.0 _ll_register_callbacks cb 0x7f41d40016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54722/0 (socket says 172.21.15.37:54722) 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.935+0000 7f41db400640 1 -- 172.21.15.37:0/1528660704 learned_addr learned my addr 172.21.15.37:0/1528660704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:11.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfebdf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41dbe00640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfebdf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 -- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 msgr2=0x564dfebdf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfebdf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 -- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 msgr2=0x564dfebdf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfebdf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 -- 172.21.15.37:0/1528660704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dfebb6370 con 0x564dfe7ea3b0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41db400640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02aaf0 tx=0x7f41cc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6296952b8dde0e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc048040 con 0x564dfe7ea3b0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f41cc083e70 con 0x564dfe7ea3b0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.936+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d40056d0 con 0x564dfe7ea3b0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f41cc06f170 con 0x564dfe7ea3b0 2024-09-21T06:33:11.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 10 client.7032 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x564dfec15e20 con 0x564dfe7ea3b0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d40056d0 con 0x564dfe7ea3b0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7f41cc08d110 con 0x564dfe7ea3b0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 10 client.7032 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 10 client.7032 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 20 client.7032 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfebdc8d0 0x564dfec818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 4 client.7032 mds_command: new command op to 6852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 --> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] -- command(tid 0: {"prefix": "status"}) -- 0x564dfeb2afd0 con 0x564dfebdc8d0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.937+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfebdc8d0 0x564dfec818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41d2a00640 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfebdc8d0 0x564dfec818e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8de150f54f91648b server_cookie=928105b15f37dba1 in_seq=0 out_seq=0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41c8c00640 10 client.7032 ms_handle_connect on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41c8c00640 1 -- 172.21.15.37:0/1528660704 <== mds.2 v2:172.21.15.37:6832/3081570660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564dfeb2afd0 con 0x564dfebdc8d0 2024-09-21T06:33:11.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41c8c00640 10 client.7032 handle_command_reply: tid=0 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 client.7032 shutdown 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 msgr2=0x564dfec7dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02aaf0 tx=0x7f41cc0057a0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 shutdown_connections 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x564dfebdc8d0 0x564dfec818e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564dfe813270 0x564dfebdf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dfe7f2590 0x564dfebdf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 --2- 172.21.15.37:0/1528660704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dfe7ea3b0 0x564dfec7dd60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41c8c00640 0 client.7032 ms_handle_reset on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 >> 172.21.15.37:0/1528660704 conn(0x564dfe8073c0 msgr2=0x564dfec00ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.938+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 shutdown_connections 2024-09-21T06:33:11.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e9840740 1 -- 172.21.15.37:0/1528660704 wait complete. 2024-09-21T06:33:11.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e9840740 20 client.7032 trim_cache size 0 max 16384 2024-09-21T06:33:11.945 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:33:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6852, 2024-09-21T06:33:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1220850389999999, 2024-09-21T06:33:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 77, 2024-09-21T06:33:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 80, 2024-09-21T06:33:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T06:33:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.091070856999998 2024-09-21T06:33:11.947 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:11.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 msgr2=0x7f41b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 0x7f41b807b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f41dc06d3a0 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 msgr2=0x7f41dc142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02ab20 tx=0x7f41cc033ba0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 shutdown_connections 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41b8078b90 0x7f41b807b0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41dc1ea450 0x7f41dc1ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41dc0cf6d0 0x7f41dc133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 --2- 172.21.15.37:0/998559931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41dc0cc0e0 0x7f41dc142770 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 >> 172.21.15.37:0/998559931 conn(0x7f41dc0d05c0 msgr2=0x7f41dc068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.939+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 shutdown_connections 2024-09-21T06:33:11.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:11.940+0000 7f41e1200640 1 -- 172.21.15.37:0/998559931 wait complete. 2024-09-21T06:33:11.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 -- 172.21.15.37:0/196879041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c0073b10 msgr2=0x7ff3c0073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 --2- 172.21.15.37:0/196879041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c0073b10 0x7ff3c0073f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009830 tx=0x7ff3ac02a480 comp rx=0 tx=0).stop 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 -- 172.21.15.37:0/196879041 shutdown_connections 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 --2- 172.21.15.37:0/196879041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c0073b10 0x7ff3c0073f20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 --2- 172.21.15.37:0/196879041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 0x7ff3c00ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 -- 172.21.15.37:0/196879041 >> 172.21.15.37:0/196879041 conn(0x7ff3c00d1190 msgr2=0x7ff3c0073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 -- 172.21.15.37:0/196879041 shutdown_connections 2024-09-21T06:33:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.191+0000 7ff3c6000640 1 -- 172.21.15.37:0/196879041 wait complete. 2024-09-21T06:33:12.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.194+0000 7ff3c6000640 1 Processor -- start 2024-09-21T06:33:12.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.194+0000 7ff3c6000640 1 -- start start 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c0073b10 0x7ff3c00b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 0x7ff3c00b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c015b760 con 0x7ff3c00b4a10 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0070b60 con 0x7ff3c0073b10 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.195+0000 7ff3c6000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00b5530 con 0x7ff3c00d00e0 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54732/0 (socket says 172.21.15.37:54732) 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 -- 172.21.15.37:0/3071390273 learned_addr learned my addr 172.21.15.37:0/3071390273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3c5600640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 0x7ff3c00b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c0073b10 0x7ff3c00b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 -- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 msgr2=0x7ff3c00b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 0x7ff3c00b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 -- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c0073b10 msgr2=0x7ff3c00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c0073b10 0x7ff3c00b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 -- 172.21.15.37:0/3071390273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c01f9410 con 0x7ff3c00b4a10 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.196+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac02aa70 tx=0x7ff3ac030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.197+0000 7ff3bcc00640 1 -- 172.21.15.37:0/3071390273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac030c50 con 0x7ff3c00b4a10 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.197+0000 7ff3bcc00640 1 -- 172.21.15.37:0/3071390273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3ac03b410 con 0x7ff3c00b4a10 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.197+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c01f96f0 con 0x7ff3c00b4a10 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.197+0000 7ff3bcc00640 1 -- 172.21.15.37:0/3071390273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac03b860 con 0x7ff3c00b4a10 2024-09-21T06:33:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.197+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c00b0d50 con 0x7ff3c00b4a10 2024-09-21T06:33:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.199+0000 7ff3bcc00640 1 -- 172.21.15.37:0/3071390273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3ac03ba00 con 0x7ff3c00b4a10 2024-09-21T06:33:12.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.200+0000 7ff3bcc00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 0x7ff3a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.200+0000 7ff3bcc00640 1 -- 172.21.15.37:0/3071390273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ff3ac0bc260 con 0x7ff3c00b4a10 2024-09-21T06:33:12.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.201+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 0x7ff3a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.201+0000 7ff3ce5c3740 1 Processor -- start 2024-09-21T06:33:12.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.201+0000 7ff3ce5c3740 1 -- start start 2024-09-21T06:33:12.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.201+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 0x7ff3a007b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009b30 tx=0x7ff3b4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb79f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb76f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eedb6f8430 con 0x55eedb3ce270 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eedb6e85d0 con 0x55eedb3ad590 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eedb6d7a00 con 0x55eedb3a53b0 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3c4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.202+0000 7ff3c4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54738/0 (socket says 172.21.15.37:54738) 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 -- 172.21.15.37:0/4224117973 learned_addr learned my addr 172.21.15.37:0/4224117973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb79f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c5600640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb76f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 -- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 msgr2=0x55eedb79f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb79f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 -- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 msgr2=0x55eedb76f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb76f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 -- 172.21.15.37:0/4224117973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eedb6a0520 con 0x55eedb3ce270 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c5600640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb76f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.203+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4011e60 tx=0x7ff3b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f496840239133d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3b1200640 1 -- 172.21.15.37:0/4224117973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400ebf0 con 0x55eedb3ce270 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3b1200640 1 -- 172.21.15.37:0/4224117973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b4004020 con 0x55eedb3ce270 2024-09-21T06:33:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3b1200640 1 -- 172.21.15.37:0/4224117973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4004410 con 0x55eedb3ce270 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/4224117973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 msgr2=0x55eedb79eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4011e60 tx=0x7ff3b4014000 comp rx=0 tx=0).stop 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/4224117973 shutdown_connections 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb76f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb79f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/4224117973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb79eb60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.204+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/4224117973 >> 172.21.15.37:0/4224117973 conn(0x55eedb3c23c0 msgr2=0x55eedb774b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.205+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/4224117973 shutdown_connections 2024-09-21T06:33:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.205+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/4224117973 wait complete. 2024-09-21T06:33:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.206+0000 7ff3ce5c3740 1 -- start start 2024-09-21T06:33:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.206+0000 7ff3ce5c3740 1 Processor -- start 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb7952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb797010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eedb67e330 con 0x55eedb3ce270 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eedb6db810 con 0x55eedb3ad590 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eedb448620 con 0x55eedb3a53b0 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.207+0000 7ff3ce5c3740 10 client.0 _ll_register_callbacks cb 0x55eedb683420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54742/0 (socket says 172.21.15.37:54742) 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 -- 172.21.15.37:0/3531356114 learned_addr learned my addr 172.21.15.37:0/3531356114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb7952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb797010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 -- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 msgr2=0x55eedb7952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb7952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 -- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 msgr2=0x55eedb797010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb797010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 -- 172.21.15.37:0/3531356114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eedb439850 con 0x55eedb3ce270 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3c5600640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005550 tx=0x7ff3b8001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c060e7553266b364 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.208+0000 7ff3a5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80057e0 con 0x55eedb3ce270 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ff3b8013570 con 0x55eedb3ce270 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b4004020 con 0x55eedb3ce270 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff3b8021050 con 0x55eedb3ce270 2024-09-21T06:33:12.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 10 client.7047 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55eedb4b06b0 con 0x55eedb3ce270 2024-09-21T06:33:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b4004020 con 0x55eedb3ce270 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7ff3b8024020 con 0x55eedb3ce270 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 10 client.7047 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 10 client.7047 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 20 client.7047 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb38ce40 0x55eedb798610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 4 client.7047 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.209+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eedb6ea520 con 0x55eedb38ce40 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.210+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb38ce40 0x55eedb798610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.210+0000 7ff3a5600640 10 client.7047 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.210+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb38ce40 0x55eedb798610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ed327445623c559 server_cookie=970a8d460f5a0eb5 in_seq=0 out_seq=0 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3a5600640 1 -- 172.21.15.37:0/3531356114 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eedb6ea520 con 0x55eedb38ce40 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3a5600640 10 client.7047 handle_command_reply: tid=0 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 client.7047 shutdown 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 msgr2=0x55eedb7b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005550 tx=0x7ff3b8001c30 comp rx=0 tx=0).stop 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 shutdown_connections 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb38ce40 0x55eedb798610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7b3090 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ad590 0x55eedb797010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/3531356114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb7952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 >> 172.21.15.37:0/3531356114 conn(0x55eedb3c23c0 msgr2=0x55eedb7bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.211+0000 7ff3a5600640 0 client.7047 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.212+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 shutdown_connections 2024-09-21T06:33:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.212+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/3531356114 wait complete. 2024-09-21T06:33:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.212+0000 7ff3ce5c3740 20 client.7047 trim_cache size 0 max 16384 2024-09-21T06:33:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.231+0000 7ff3ce5c3740 1 Processor -- start 2024-09-21T06:33:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.231+0000 7ff3ce5c3740 1 -- start start 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3a53b0 0x55eedb795090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3ad590 0x55eedb7bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01434e0 con 0x55eedb3ce270 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0201f10 con 0x55eedb3a53b0 2024-09-21T06:33:12.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c01403f0 con 0x55eedb3ad590 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54750/0 (socket says 172.21.15.37:54750) 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 -- 172.21.15.37:0/2058739475 learned_addr learned my addr 172.21.15.37:0/2058739475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 -- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3ad590 msgr2=0x55eedb7bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3ad590 0x55eedb7bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 -- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3a53b0 msgr2=0x55eedb795090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3a53b0 0x55eedb795090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3a53b0 0x55eedb795090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.233+0000 7ff3c5600640 1 -- 172.21.15.37:0/2058739475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eedb7b3400 con 0x55eedb3ce270 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.234+0000 7ff3c5600640 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005140 tx=0x7ff3b8013bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef2e244fc008168 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.234+0000 7ff3a7e00640 1 -- 172.21.15.37:0/2058739475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80057e0 con 0x55eedb3ce270 2024-09-21T06:33:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.234+0000 7ff3a7e00640 1 -- 172.21.15.37:0/2058739475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8019070 con 0x55eedb3ce270 2024-09-21T06:33:12.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.234+0000 7ff3a7e00640 1 -- 172.21.15.37:0/2058739475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8013d70 con 0x55eedb3ce270 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2058739475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 msgr2=0x55eedb7bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005140 tx=0x7ff3b8013bb0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2058739475 shutdown_connections 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ce270 0x55eedb7bb980 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3ad590 0x55eedb7bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2058739475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3a53b0 0x55eedb795090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2058739475 >> 172.21.15.37:0/2058739475 conn(0x55eedb3c23c0 msgr2=0x55eedb7bd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2058739475 shutdown_connections 2024-09-21T06:33:12.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.235+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2058739475 wait complete. 2024-09-21T06:33:12.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.236+0000 7ff3ce5c3740 1 -- start start 2024-09-21T06:33:12.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.236+0000 7ff3ce5c3740 1 Processor -- start 2024-09-21T06:33:12.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb82fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 0x55eedb7cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eedb79b9c0 con 0x55eedb3ad590 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eedb7ccb60 con 0x55eedb3ce270 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eedb79ada0 con 0x55eedb3a53b0 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3ce5c3740 10 client.0 _ll_register_callbacks cb 0x7ff3b80051b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54766/0 (socket says 172.21.15.37:54766) 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 -- 172.21.15.37:0/2123708263 learned_addr learned my addr 172.21.15.37:0/2123708263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3c5600640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 0x55eedb7cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb82fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 -- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 msgr2=0x55eedb82fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb82fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 -- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 msgr2=0x55eedb7cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 0x55eedb7cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3bfe00640 1 -- 172.21.15.37:0/2123708263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eedb797e30 con 0x55eedb3ad590 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3c5600640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 0x55eedb7cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.238+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb82fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3bfe00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0ba4d0 tx=0x7ff3ac080530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed303a783df7ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac09d020 con 0x55eedb3ad590 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ff3ac084280 con 0x55eedb3ad590 2024-09-21T06:33:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8019070 con 0x55eedb3ad590 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac083770 con 0x55eedb3ad590 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 10 client.7053 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55eedb7cbe00 con 0x55eedb3ad590 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8019070 con 0x55eedb3ad590 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7ff3ac04b030 con 0x55eedb3ad590 2024-09-21T06:33:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 10 client.7053 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 10 client.7053 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.239+0000 7ff3ce5c3740 20 client.7053 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb839f00 0x55eedb7bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3ce5c3740 4 client.7053 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "status"}) -- 0x55eedb6e5fd0 con 0x55eedb839f00 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb839f00 0x55eedb7bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3a5600640 10 client.7053 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.240+0000 7ff3c4c00640 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb839f00 0x55eedb7bc5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e92e10ff7a0769ad server_cookie=a08a4c4cad07e57a in_seq=0 out_seq=0 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3a5600640 1 -- 172.21.15.37:0/2123708263 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55eedb6e5fd0 con 0x55eedb839f00 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3a5600640 10 client.7053 handle_command_reply: tid=0 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 client.7053 shutdown 2024-09-21T06:33:12.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 msgr2=0x55eedb7cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0ba4d0 tx=0x7ff3ac080530 comp rx=0 tx=0).stop 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 shutdown_connections 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55eedb839f00 0x55eedb7bc5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eedb3ad590 0x55eedb7cae60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eedb3ce270 0x55eedb7cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 --2- 172.21.15.37:0/2123708263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eedb3a53b0 0x55eedb82fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3a5600640 0 client.7053 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 >> 172.21.15.37:0/2123708263 conn(0x55eedb3c23c0 msgr2=0x55eedb796830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 shutdown_connections 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 1 -- 172.21.15.37:0/2123708263 wait complete. 2024-09-21T06:33:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.241+0000 7ff3ce5c3740 20 client.7053 trim_cache size 0 max 16384 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6858, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.4373593640000002, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 77, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 80, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T06:33:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.390334396 2024-09-21T06:33:12.250 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:12.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 msgr2=0x7ff3a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 0x7ff3a007b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009b30 tx=0x7ff3b4005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:12.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 msgr2=0x7ff3c01f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac02aa70 tx=0x7ff3ac030650 comp rx=0 tx=0).stop 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 shutdown_connections 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff3a0078b90 0x7ff3a007b0a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c00b4a10 0x7ff3c01f8800 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3c00d00e0 0x7ff3c00b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 --2- 172.21.15.37:0/3071390273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c0073b10 0x7ff3c00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 >> 172.21.15.37:0/3071390273 conn(0x7ff3c00d1190 msgr2=0x7ff3c0159460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 shutdown_connections 2024-09-21T06:33:12.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.242+0000 7ff3c6000640 1 -- 172.21.15.37:0/3071390273 wait complete. 2024-09-21T06:33:12.265 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 -- 172.21.15.37:0/3771332035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 msgr2=0x7f7dbc073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/3771332035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc073f20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009830 tx=0x7f7da402a480 comp rx=0 tx=0).stop 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 -- 172.21.15.37:0/3771332035 shutdown_connections 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/3771332035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc073f20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/3771332035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0d00e0 0x7f7dbc0ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.490+0000 7f7dbbe00640 1 -- 172.21.15.37:0/3771332035 >> 172.21.15.37:0/3771332035 conn(0x7f7dbc0d1190 msgr2=0x7f7dbc073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.491+0000 7f7dbbe00640 1 -- 172.21.15.37:0/3771332035 shutdown_connections 2024-09-21T06:33:12.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.491+0000 7f7dbbe00640 1 -- 172.21.15.37:0/3771332035 wait complete. 2024-09-21T06:33:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.493+0000 7f7dbbe00640 1 Processor -- start 2024-09-21T06:33:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.493+0000 7f7dbbe00640 1 -- start start 2024-09-21T06:33:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0b4a10 0x7f7dbc1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dbc0d00e0 0x7f7dbc06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc15b760 con 0x7f7dbc073b10 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc070b60 con 0x7f7dbc0d00e0 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.494+0000 7f7dbbe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc0b5530 con 0x7f7dbc0b4a10 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54784/0 (socket says 172.21.15.37:54784) 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 -- 172.21.15.37:0/785512476 learned_addr learned my addr 172.21.15.37:0/785512476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7db3e00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0b4a10 0x7f7dbc1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbb400640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dbc0d00e0 0x7f7dbc06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 -- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0b4a10 msgr2=0x7f7dbc1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0b4a10 0x7f7dbc1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 -- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dbc0d00e0 msgr2=0x7f7dbc06fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dbc0d00e0 0x7f7dbc06fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.495+0000 7f7dbaa00640 1 -- 172.21.15.37:0/785512476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc15bb80 con 0x7f7dbc073b10 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e790 tx=0x7f7dac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7db2a00640 1 -- 172.21.15.37:0/785512476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00eea0 con 0x7f7dbc073b10 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7db2a00640 1 -- 172.21.15.37:0/785512476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dac00cdb0 con 0x7f7dbc073b10 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc15be60 con 0x7f7dbc073b10 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7db2a00640 1 -- 172.21.15.37:0/785512476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac010690 con 0x7f7dbc073b10 2024-09-21T06:33:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.496+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc15c420 con 0x7f7dbc073b10 2024-09-21T06:33:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.498+0000 7f7db2a00640 1 -- 172.21.15.37:0/785512476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7dac010830 con 0x7f7dbc073b10 2024-09-21T06:33:12.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.499+0000 7f7db2a00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 0x7f7d9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.500+0000 7f7db2a00640 1 -- 172.21.15.37:0/785512476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7dac09b910 con 0x7f7dbc073b10 2024-09-21T06:33:12.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.500+0000 7f7db3e00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 0x7f7d9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.500+0000 7f7dc8c73740 1 Processor -- start 2024-09-21T06:33:12.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.500+0000 7f7dc8c73740 1 -- start start 2024-09-21T06:33:12.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.501+0000 7f7db3e00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 0x7f7d9807b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7da4034000 tx=0x7f7da4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b450d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b44dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5639b4466430 con 0x5639b413c270 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5639b44565d0 con 0x5639b41133b0 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5639b4445a00 con 0x5639b411b590 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54796/0 (socket says 172.21.15.37:54796) 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 -- 172.21.15.37:0/329295086 learned_addr learned my addr 172.21.15.37:0/329295086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbb400640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b44dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7db3e00640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b450d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 -- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 msgr2=0x5639b44dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b44dd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 -- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 msgr2=0x5639b450d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b450d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 -- 172.21.15.37:0/329295086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b440e520 con 0x5639b413c270 2024-09-21T06:33:12.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbb400640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b44dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:12.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.502+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c6a0 tx=0x7f7dac05f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba05386a71639a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.503+0000 7f7dab400640 1 -- 172.21.15.37:0/329295086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac06ce40 con 0x5639b413c270 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.503+0000 7f7dab400640 1 -- 172.21.15.37:0/329295086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dac062730 con 0x5639b413c270 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.503+0000 7f7dab400640 1 -- 172.21.15.37:0/329295086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac010b20 con 0x5639b413c270 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.503+0000 7f7dc8c73740 1 -- 172.21.15.37:0/329295086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 msgr2=0x5639b450cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.503+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c6a0 tx=0x7f7dac05f4a0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 -- 172.21.15.37:0/329295086 shutdown_connections 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b44dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b450d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/329295086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b450cb60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 -- 172.21.15.37:0/329295086 >> 172.21.15.37:0/329295086 conn(0x5639b41303c0 msgr2=0x5639b44e2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 -- 172.21.15.37:0/329295086 shutdown_connections 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.504+0000 7f7dc8c73740 1 -- 172.21.15.37:0/329295086 wait complete. 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.505+0000 7f7dc8c73740 1 -- start start 2024-09-21T06:33:12.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.505+0000 7f7dc8c73740 1 Processor -- start 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b4506fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b4521720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5639b43ec330 con 0x5639b411b590 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5639b4449810 con 0x5639b413c270 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5639b41b6620 con 0x5639b41133b0 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.506+0000 7f7dc8c73740 10 client.0 _ll_register_callbacks cb 0x5639b43f1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54806/0 (socket says 172.21.15.37:54806) 2024-09-21T06:33:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 -- 172.21.15.37:0/2005643026 learned_addr learned my addr 172.21.15.37:0/2005643026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7dbb400640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b4521720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b4506fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 -- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 msgr2=0x5639b4506fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b4506fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 -- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 msgr2=0x5639b4521720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b4521720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.507+0000 7f7db3e00640 1 -- 172.21.15.37:0/2005643026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b41a7850 con 0x5639b411b590 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7db3e00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7da4005eb0 tx=0x7f7da4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb57ea80246c5801 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7d9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da403ccf0 con 0x5639b411b590 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7da4005020 con 0x5639b411b590 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dac062730 con 0x5639b411b590 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7da405a050 con 0x5639b411b590 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7dc8c73740 10 client.7068 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5639b421e6b0 con 0x5639b411b590 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.508+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dac062730 con 0x5639b411b590 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7f7da4005220 con 0x5639b411b590 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 10 client.7068 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 10 client.7068 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 20 client.7068 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b40fae40 0x5639b451fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 4 client.7068 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b4458520 con 0x5639b40fae40 2024-09-21T06:33:12.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.509+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b40fae40 0x5639b451fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.510+0000 7f7d9fe00640 10 client.7068 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:12.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.510+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b40fae40 0x5639b451fb70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584abde7c4c9fe3a server_cookie=dba44b58100f4139 in_seq=0 out_seq=0 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7d9fe00640 1 -- 172.21.15.37:0/2005643026 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639b4458520 con 0x5639b40fae40 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7d9fe00640 10 client.7068 handle_command_reply: tid=0 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 client.7068 shutdown 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 msgr2=0x5639b45211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7da4005eb0 tx=0x7f7da4005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 shutdown_connections 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b40fae40 0x5639b451fb70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b45211e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b4521720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/2005643026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b4506fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7d9fe00640 0 client.7068 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 >> 172.21.15.37:0/2005643026 conn(0x5639b41303c0 msgr2=0x5639b4504070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 shutdown_connections 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 1 -- 172.21.15.37:0/2005643026 wait complete. 2024-09-21T06:33:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.511+0000 7f7dc8c73740 20 client.7068 trim_cache size 0 max 16384 2024-09-21T06:33:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.531+0000 7f7dc8c73740 1 Processor -- start 2024-09-21T06:33:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.531+0000 7f7dc8c73740 1 -- start start 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b4505ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b451fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc2016c0 con 0x5639b413c270 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc201d90 con 0x5639b41133b0 2024-09-21T06:33:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc158af0 con 0x5639b411b590 2024-09-21T06:33:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dbb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dbb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54816/0 (socket says 172.21.15.37:54816) 2024-09-21T06:33:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.532+0000 7f7dbb400640 1 -- 172.21.15.37:0/1148627957 learned_addr learned my addr 172.21.15.37:0/1148627957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7db3e00640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b451fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b4505ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 -- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 msgr2=0x5639b451fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b451fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 -- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 msgr2=0x5639b4505ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b4505ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 -- 172.21.15.37:0/1148627957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b4527da0 con 0x5639b413c270 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7db3e00640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b451fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b4505ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7dbb400640 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7db40050e0 tx=0x7f7db40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c10d9efbc70b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7daaa00640 1 -- 172.21.15.37:0/1148627957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4006d40 con 0x5639b413c270 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7daaa00640 1 -- 172.21.15.37:0/1148627957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db401b020 con 0x5639b413c270 2024-09-21T06:33:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.533+0000 7f7daaa00640 1 -- 172.21.15.37:0/1148627957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4001d40 con 0x5639b413c270 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1148627957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 msgr2=0x5639b45200b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7db40050e0 tx=0x7f7db40055a0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1148627957 shutdown_connections 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b413c270 0x5639b45200b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b411b590 0x5639b451fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1148627957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b41133b0 0x5639b4505ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1148627957 >> 172.21.15.37:0/1148627957 conn(0x5639b41303c0 msgr2=0x5639b45051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1148627957 shutdown_connections 2024-09-21T06:33:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.534+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1148627957 wait complete. 2024-09-21T06:33:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.535+0000 7f7dc8c73740 1 -- start start 2024-09-21T06:33:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.535+0000 7f7dc8c73740 1 Processor -- start 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b45a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b451ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5639b453ace0 con 0x5639b411b590 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5639b45074e0 con 0x5639b413c270 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5639b4537360 con 0x5639b41133b0 2024-09-21T06:33:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dc8c73740 10 client.0 _ll_register_callbacks cb 0x7f7da4032c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54818/0 (socket says 172.21.15.37:54818) 2024-09-21T06:33:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 -- 172.21.15.37:0/1617444227 learned_addr learned my addr 172.21.15.37:0/1617444227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 -- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 msgr2=0x5639b45a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b45a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7db3e00640 1 -- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 msgr2=0x5639b451ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.537+0000 7f7dbb400640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b451ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7db3e00640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b451ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7db3e00640 1 -- 172.21.15.37:0/1617444227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b451e260 con 0x5639b411b590 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7dbb400640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b451ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7db3e00640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7da40057c0 tx=0x7f7da403cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e23700779d9e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7d9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4052690 con 0x5639b411b590 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7da4052830 con 0x5639b411b590 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db401b020 con 0x5639b411b590 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7da406d050 con 0x5639b411b590 2024-09-21T06:33:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7dc8c73740 10 client.7074 fetch_fsmap learned FSMap version 77 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.538+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5639b453b3d0 con 0x5639b411b590 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db401b020 con 0x5639b411b590 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 77) ==== 3542+0+0 (secure 0 0 0) 0x7f7da4070020 con 0x5639b411b590 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 10 client.7074 fetch_fsmap finished waiting for FSMap version 77 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 10 client.7074 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 20 client.7074 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b45ab3d0 0x5639b45a7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 4 client.7074 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "status"}) -- 0x5639b4453fd0 con 0x5639b45ab3d0 2024-09-21T06:33:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.539+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b45ab3d0 0x5639b45a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.540+0000 7f7dbaa00640 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b45ab3d0 0x5639b45a7230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48ab6e2abb291f2 server_cookie=f657c3d19e969de1 in_seq=0 out_seq=0 2024-09-21T06:33:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.540+0000 7f7d9fe00640 10 client.7074 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.540+0000 7f7d9fe00640 1 -- 172.21.15.37:0/1617444227 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5639b4453fd0 con 0x5639b45ab3d0 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.540+0000 7f7d9fe00640 10 client.7074 handle_command_reply: tid=0 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.540+0000 7f7dc8c73740 1 client.7074 shutdown 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 msgr2=0x5639b451e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7da40057c0 tx=0x7f7da403cd80 comp rx=0 tx=0).stop 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 shutdown_connections 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x5639b45ab3d0 0x5639b45a7230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639b411b590 0x5639b451e8e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639b413c270 0x5639b451ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7d9fe00640 0 client.7074 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 --2- 172.21.15.37:0/1617444227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639b41133b0 0x5639b45a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 >> 172.21.15.37:0/1617444227 conn(0x5639b41303c0 msgr2=0x5639b45365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 shutdown_connections 2024-09-21T06:33:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 1 -- 172.21.15.37:0/1617444227 wait complete. 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dc8c73740 20 client.7074 trim_cache size 0 max 16384 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6870, 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 12.709702764999999, 2024-09-21T06:33:12.546 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 77, 2024-09-21T06:33:12.547 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 80, 2024-09-21T06:33:12.547 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T06:33:12.547 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.679078017999998 2024-09-21T06:33:12.547 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 msgr2=0x7f7d9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 0x7f7d9807b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7da4034000 tx=0x7f7da4004730 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 msgr2=0x7f7dbc1f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.541+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e790 tx=0x7f7dac00ec50 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 shutdown_connections 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7d98078b90 0x7f7d9807b0a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dbc0d00e0 0x7f7dbc06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7dbc0b4a10 0x7f7dbc1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 --2- 172.21.15.37:0/785512476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dbc073b10 0x7f7dbc1f9270 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 >> 172.21.15.37:0/785512476 conn(0x7f7dbc0d1190 msgr2=0x7f7dbc1591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 shutdown_connections 2024-09-21T06:33:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.542+0000 7f7dbbe00640 1 -- 172.21.15.37:0/785512476 wait complete. 2024-09-21T06:33:12.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 -- 172.21.15.37:0/3950963125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1806d7f0 msgr2=0x7f2d18154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 --2- 172.21.15.37:0/3950963125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1806d7f0 0x7f2d18154ea0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b6de0 tx=0x7f2d1002aa30 comp rx=0 tx=0).stop 2024-09-21T06:33:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 -- 172.21.15.37:0/3950963125 shutdown_connections 2024-09-21T06:33:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 --2- 172.21.15.37:0/3950963125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1806d7f0 0x7f2d18154ea0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 --2- 172.21.15.37:0/3950963125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d181627f0 0x7f2d1806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 --2- 172.21.15.37:0/3950963125 >> v2:172.21.15.150:3300/0 conn(0x7f2d180b9e40 0x7f2d181622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 -- 172.21.15.37:0/3950963125 >> 172.21.15.37:0/3950963125 conn(0x7f2d180d1930 msgr2=0x7f2d180d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 -- 172.21.15.37:0/3950963125 shutdown_connections 2024-09-21T06:33:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.791+0000 7f2d17e00640 1 -- 172.21.15.37:0/3950963125 wait complete. 2024-09-21T06:33:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.793+0000 7f2d17e00640 1 Processor -- start 2024-09-21T06:33:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.793+0000 7f2d17e00640 1 -- start start 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.794+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d1806d7f0 0x7f2d1814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d180b9e40 0x7f2d18157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181555f0 con 0x7f2d181627f0 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1806e7f0 con 0x7f2d180b9e40 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2d18155770 con 0x7f2d1806d7f0 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54850/0 (socket says 172.21.15.37:54850) 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 -- 172.21.15.37:0/2054358423 learned_addr learned my addr 172.21.15.37:0/2054358423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d0e000640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d180b9e40 0x7f2d18157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d16a00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d1806d7f0 0x7f2d1814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 -- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d1806d7f0 msgr2=0x7f2d1814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d1806d7f0 0x7f2d1814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 -- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d180b9e40 msgr2=0x7f2d18157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d180b9e40 0x7f2d18157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.795+0000 7f2d17400640 1 -- 172.21.15.37:0/2054358423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d180ccae0 con 0x7f2d181627f0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d17400640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002b020 tx=0x7f2d10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10033050 con 0x7f2d181627f0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d10004450 con 0x7f2d181627f0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10038450 con 0x7f2d181627f0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d180ccdc0 con 0x7f2d181627f0 2024-09-21T06:33:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.796+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181608f0 con 0x7f2d181627f0 2024-09-21T06:33:12.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.797+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18160b50 con 0x7f2d181627f0 2024-09-21T06:33:12.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.799+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d10036070 con 0x7f2d181627f0 2024-09-21T06:33:12.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.800+0000 7f2d0f400640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 0x7f2cf407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:12.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.800+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2d100bc2e0 con 0x7f2d181627f0 2024-09-21T06:33:12.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.800+0000 7f2d16a00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 0x7f2cf407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.801+0000 7f2d16a00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 0x7f2cf407b0a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d0800b260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:12.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:12.804+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d10083f30 con 0x7f2d181627f0 2024-09-21T06:33:13.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.046+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d181622b0 con 0x7f2d181627f0 2024-09-21T06:33:13.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.047+0000 7f2d0f400640 1 -- 172.21.15.37:0/2054358423 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5269 (secure 0 0 0) 0x7f2d100874b0 con 0x7f2d181627f0 2024-09-21T06:33:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:13.050 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":77,"btime":"2024-09-21T06:33:10:816718+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:10.816711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6852,6858,6870]},"id":5}]} 2024-09-21T06:33:13.050 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 77 2024-09-21T06:33:13.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.049+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 msgr2=0x7f2cf407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:13.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.049+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 0x7f2cf407b0a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d0800b260 comp rx=0 tx=0).stop 2024-09-21T06:33:13.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.049+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 msgr2=0x7f2d181583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:13.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.049+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002b020 tx=0x7f2d10003f10 comp rx=0 tx=0).stop 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 shutdown_connections 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2cf4078b90 0x7f2cf407b0a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d181627f0 0x7f2d181583e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d180b9e40 0x7f2d18157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 --2- 172.21.15.37:0/2054358423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d1806d7f0 0x7f2d1814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 >> 172.21.15.37:0/2054358423 conn(0x7f2d180d1930 msgr2=0x7f2d180b8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 shutdown_connections 2024-09-21T06:33:13.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:13.050+0000 7f2d17e00640 1 -- 172.21.15.37:0/2054358423 wait complete. 2024-09-21T06:33:13.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:13.074 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:33:13.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:13.126 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:33:13.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:13.168 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:33:13.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:13.244 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:33:13.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:13.321 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:33:13.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:33:13.645 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:33:13.646 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:13.646 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:33:15.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:33:15.900 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:33:15.901 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:15.901 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:33:17.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:33:18.150 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:33:18.151 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:18.151 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:33:20.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:33:20.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:33:20.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:33:20.406 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:33:20.407 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:33:20.407 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:33:20.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:33:20.657 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:33:20.658 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:20.658 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:33:22.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:33:22.918 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:33:22.918 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:22.918 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:33:24.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:33:25.176 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T06:33:25.177 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:33:25.177 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:33:25.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:33:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:33:25.425 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:33:25.425 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:33:25.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 -- 172.21.15.37:0/1438853366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40b4f30 msgr2=0x7fccd41429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 --2- 172.21.15.37:0/1438853366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40b4f30 0x7fccd41429b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fccd40b3160 tx=0x7fccbc02a530 comp rx=0 tx=0).stop 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 -- 172.21.15.37:0/1438853366 shutdown_connections 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 --2- 172.21.15.37:0/1438853366 >> v2:172.21.15.150:3300/0 conn(0x7fccd40cc8d0 0x7fccd40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 --2- 172.21.15.37:0/1438853366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40b4f30 0x7fccd41429b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 -- 172.21.15.37:0/1438853366 >> 172.21.15.37:0/1438853366 conn(0x7fccd406f920 msgr2=0x7fccd40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 -- 172.21.15.37:0/1438853366 shutdown_connections 2024-09-21T06:33:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.650+0000 7fccd9000640 1 -- 172.21.15.37:0/1438853366 wait complete. 2024-09-21T06:33:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.653+0000 7fccd9000640 1 Processor -- start 2024-09-21T06:33:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.653+0000 7fccd9000640 1 -- start start 2024-09-21T06:33:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd40b4f30 0x7fccd415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 0x7fccd41f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ce150 con 0x7fccd40cc8d0 2024-09-21T06:33:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40cdfd0 con 0x7fccd40b4f30 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.654+0000 7fccd9000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fccd40ce2d0 con 0x7fccd41456e0 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53572/0 (socket says 172.21.15.37:53572) 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 -- 172.21.15.37:0/3547180956 learned_addr learned my addr 172.21.15.37:0/3547180956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fccd3400640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd40b4f30 0x7fccd415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 0x7fccd41f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 -- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 msgr2=0x7fccd41f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 0x7fccd41f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 -- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd40b4f30 msgr2=0x7fccd415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd40b4f30 0x7fccd415dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 -- 172.21.15.37:0/3547180956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd415e5c0 con 0x7fccd40cc8d0 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 0x7fccd41f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.655+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009780 tx=0x7fccbc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.656+0000 7fcccb400640 1 -- 172.21.15.37:0/3547180956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc031d60 con 0x7fccd40cc8d0 2024-09-21T06:33:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.656+0000 7fcccb400640 1 -- 172.21.15.37:0/3547180956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccbc033e10 con 0x7fccd40cc8d0 2024-09-21T06:33:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.656+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd415e8a0 con 0x7fccd40cc8d0 2024-09-21T06:33:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.656+0000 7fcccb400640 1 -- 172.21.15.37:0/3547180956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc005260 con 0x7fccd40cc8d0 2024-09-21T06:33:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.656+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd415ee60 con 0x7fccd40cc8d0 2024-09-21T06:33:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.658+0000 7fcccb400640 1 -- 172.21.15.37:0/3547180956 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fccbc005400 con 0x7fccd40cc8d0 2024-09-21T06:33:25.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.659+0000 7fcccb400640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 0x7fccb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.659+0000 7fcccb400640 1 -- 172.21.15.37:0/3547180956 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fccbc0bc040 con 0x7fccd40cc8d0 2024-09-21T06:33:25.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.659+0000 7fcce17cd740 1 Processor -- start 2024-09-21T06:33:25.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.660+0000 7fccd3400640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 0x7fccb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.660+0000 7fcce17cd740 1 -- start start 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf69390 0x55d18d357730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf404d0 0x55d18d33a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d18d2701c0 con 0x55d18cf486b0 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d18d283870 con 0x55d18cf69390 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d18d272ca0 con 0x55d18cf404d0 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fccd3400640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 0x7fccb007b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fccc4001720 tx=0x7fccc4007e90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53588/0 (socket says 172.21.15.37:53588) 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fccd3e00640 1 -- 172.21.15.37:0/2671507414 learned_addr learned my addr 172.21.15.37:0/2671507414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fcccbe00640 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf404d0 0x55d18d33a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.661+0000 7fccd3400640 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf69390 0x55d18d357730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 -- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf404d0 msgr2=0x55d18d33a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf404d0 0x55d18d33a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 -- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf69390 msgr2=0x55d18d357730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf69390 0x55d18d357730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 -- 172.21.15.37:0/2671507414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18d04b7d0 con 0x55d18cf486b0 2024-09-21T06:33:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccd3e00640 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009f20 tx=0x7fcccc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73874a3320154a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccc3e00640 1 -- 172.21.15.37:0/2671507414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc016070 con 0x55d18cf486b0 2024-09-21T06:33:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccc3e00640 1 -- 172.21.15.37:0/2671507414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc01b750 con 0x55d18cf486b0 2024-09-21T06:33:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.662+0000 7fccc3e00640 1 -- 172.21.15.37:0/2671507414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc01bba0 con 0x55d18cf486b0 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 -- 172.21.15.37:0/2671507414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 msgr2=0x55d18d33a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009f20 tx=0x7fcccc010800 comp rx=0 tx=0).stop 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 -- 172.21.15.37:0/2671507414 shutdown_connections 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf486b0 0x55d18d33a940 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf404d0 0x55d18d33a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 --2- 172.21.15.37:0/2671507414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf69390 0x55d18d357730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 -- 172.21.15.37:0/2671507414 >> 172.21.15.37:0/2671507414 conn(0x55d18cf5d4e0 msgr2=0x55d18d310620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 -- 172.21.15.37:0/2671507414 shutdown_connections 2024-09-21T06:33:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.663+0000 7fcce17cd740 1 -- 172.21.15.37:0/2671507414 wait complete. 2024-09-21T06:33:25.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.664+0000 7fcce17cd740 1 -- start start 2024-09-21T06:33:25.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.665+0000 7fcce17cd740 1 Processor -- start 2024-09-21T06:33:25.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d34f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d3647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d18d219410 con 0x55d18cf69390 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d18d276ab0 con 0x55d18cf404d0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d18cfe3740 con 0x55d18cf486b0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fcce17cd740 10 client.0 _ll_register_callbacks cb 0x55d18d1f6360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53590/0 (socket says 172.21.15.37:53590) 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.666+0000 7fccd3e00640 1 -- 172.21.15.37:0/1601672339 learned_addr learned my addr 172.21.15.37:0/1601672339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fcccbe00640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d3647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3400640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d34f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 -- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 msgr2=0x55d18d3647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d3647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 -- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 msgr2=0x55d18d34f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d34f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 -- 172.21.15.37:0/1601672339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18cfd4970 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3400640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d34f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccd3e00640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcccc015040 tx=0x7fcccc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277050d55fd50852 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc016040 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.667+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fcccc004d20 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcccc01b750 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fcccc02e050 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 10 client.7095 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55d18cfb26a0 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc01b750 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7fcccc031020 con 0x55d18cf69390 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 10 client.7095 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 10 client.7095 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-21T06:33:25.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 20 client.7095 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:25.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18cf27f60 0x55d18d34cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 4 client.7095 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:25.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.668+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18d2857c0 con 0x55d18cf27f60 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.669+0000 7fccd3400640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18cf27f60 0x55d18d34cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.669+0000 7fccd3400640 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18cf27f60 0x55d18d34cba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beba2a17e55d43c4 server_cookie=60cdc64047efba0a in_seq=0 out_seq=0 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.669+0000 7fccc0c00640 10 client.7095 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.670+0000 7fccc0c00640 1 -- 172.21.15.37:0/1601672339 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d18d2857c0 con 0x55d18cf27f60 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.670+0000 7fccc0c00640 10 client.7095 handle_command_reply: tid=0 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.670+0000 7fcce17cd740 1 client.7095 shutdown 2024-09-21T06:33:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 msgr2=0x55d18d328b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcccc015040 tx=0x7fcccc0092a0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 shutdown_connections 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18cf27f60 0x55d18d34cba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d328b00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d3647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 --2- 172.21.15.37:0/1601672339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d34f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 >> 172.21.15.37:0/1601672339 conn(0x55d18cf5d4e0 msgr2=0x55d18d358740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fccc0c00640 0 client.7095 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 shutdown_connections 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 1 -- 172.21.15.37:0/1601672339 wait complete. 2024-09-21T06:33:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.671+0000 7fcce17cd740 20 client.7095 trim_cache size 0 max 16384 2024-09-21T06:33:25.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.687+0000 7fcce17cd740 1 Processor -- start 2024-09-21T06:33:25.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.687+0000 7fcce17cd740 1 -- start start 2024-09-21T06:33:25.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d30a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d34d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccd42026d0 con 0x55d18cf69390 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd42018a0 con 0x55d18cf404d0 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fccd4201fa0 con 0x55d18cf486b0 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53594/0 (socket says 172.21.15.37:53594) 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.688+0000 7fccd3e00640 1 -- 172.21.15.37:0/3112367252 learned_addr learned my addr 172.21.15.37:0/3112367252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d34d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3400640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d30a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 -- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 msgr2=0x55d18d34d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d34d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 -- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 msgr2=0x55d18d30a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d30a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 -- 172.21.15.37:0/3112367252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18d34fb40 con 0x55d18cf69390 2024-09-21T06:33:25.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d34d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:25.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3400640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d30a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:25.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005c10 tx=0x7fcccc010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27bc2a1cc62a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccc3400640 1 -- 172.21.15.37:0/3112367252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc016040 con 0x55d18cf69390 2024-09-21T06:33:25.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccc3400640 1 -- 172.21.15.37:0/3112367252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc01b750 con 0x55d18cf69390 2024-09-21T06:33:25.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.689+0000 7fccc3400640 1 -- 172.21.15.37:0/3112367252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc004d80 con 0x55d18cf69390 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 -- 172.21.15.37:0/3112367252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 msgr2=0x55d18d336c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005c10 tx=0x7fcccc010e30 comp rx=0 tx=0).stop 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 -- 172.21.15.37:0/3112367252 shutdown_connections 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf69390 0x55d18d336c10 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf486b0 0x55d18d34d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3112367252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf404d0 0x55d18d30a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 -- 172.21.15.37:0/3112367252 >> 172.21.15.37:0/3112367252 conn(0x55d18cf5d4e0 msgr2=0x55d18d310090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 -- 172.21.15.37:0/3112367252 shutdown_connections 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.690+0000 7fcce17cd740 1 -- 172.21.15.37:0/3112367252 wait complete. 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.691+0000 7fcce17cd740 1 -- start start 2024-09-21T06:33:25.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.691+0000 7fcce17cd740 1 Processor -- start 2024-09-21T06:33:25.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 0x55d18d30a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf69390 0x55d18d3d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccd412f360 con 0x55d18cf404d0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d18d34d060 con 0x55d18cf486b0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d18d350590 con 0x55d18cf69390 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcce17cd740 10 client.0 _ll_register_callbacks cb 0x7fccbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53600/0 (socket says 172.21.15.37:53600) 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 -- 172.21.15.37:0/3657375636 learned_addr learned my addr 172.21.15.37:0/3657375636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3e00640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf69390 0x55d18d3d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 0x55d18d30a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 -- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf69390 msgr2=0x55d18d3d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf69390 0x55d18d3d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 -- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 msgr2=0x55d18d30a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 0x55d18d30a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fccd3400640 1 -- 172.21.15.37:0/3657375636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18d3d3b50 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.693+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 0x55d18d30a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fccd3400640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fccc400fd90 tx=0x7fccc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb7adcbe9096c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fccc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc402f060 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fccc4006890 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcccc01b750 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fccc4035050 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fcce17cd740 10 client.7101 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.694+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55d18d351bd0 con 0x55d18cf404d0 2024-09-21T06:33:25.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc01b750 con 0x55d18cf404d0 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7fccc4038020 con 0x55d18cf404d0 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 10 client.7101 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 10 client.7101 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 20 client.7101 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18d3d10e0 0x55d18d3d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 4 client.7101 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 --> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d18d281270 con 0x55d18d3d10e0 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.695+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18d3d10e0 0x55d18d3d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.696+0000 7fcccbe00640 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18d3d10e0 0x55d18d3d4060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fe2608a0f0af6f server_cookie=c25079f4d9aeb5fb in_seq=0 out_seq=0 2024-09-21T06:33:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.696+0000 7fccc0c00640 10 client.7101 ms_handle_connect on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fccc0c00640 1 -- 172.21.15.37:0/3657375636 <== mds.0 v2:172.21.15.150:6836/1559036255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d18d281270 con 0x55d18d3d10e0 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fccc0c00640 10 client.7101 handle_command_reply: tid=0 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 client.7101 shutdown 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 msgr2=0x55d18d34e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fccc400fd90 tx=0x7fccc4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 shutdown_connections 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:6836/1559036255,v1:172.21.15.150:6837/1559036255] conn(0x55d18d3d10e0 0x55d18d3d4060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d18cf69390 0x55d18d3d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d18cf486b0 0x55d18d30a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 --2- 172.21.15.37:0/3657375636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d18cf404d0 0x55d18d34e430 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 >> 172.21.15.37:0/3657375636 conn(0x55d18cf5d4e0 msgr2=0x55d18d311910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fccc0c00640 0 client.7101 ms_handle_reset on v2:172.21.15.150:6836/1559036255 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.711+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 shutdown_connections 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fcce17cd740 1 -- 172.21.15.37:0/3657375636 wait complete. 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fcce17cd740 20 client.7101 trim_cache size 0 max 16384 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:33:25.715 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 msgr2=0x7fccb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 0x7fccb007b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fccc4001720 tx=0x7fccc4007e90 comp rx=0 tx=0).stop 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 msgr2=0x7fccd41f8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009780 tx=0x7fccbc033c20 comp rx=0 tx=0).stop 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 shutdown_connections 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccb0078b90 0x7fccb007b0a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd40cc8d0 0x7fccd41f8b40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd41456e0 0x7fccd41f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 --2- 172.21.15.37:0/3547180956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd40b4f30 0x7fccd415dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.712+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 >> 172.21.15.37:0/3547180956 conn(0x7fccd406f920 msgr2=0x7fccd4072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.713+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 shutdown_connections 2024-09-21T06:33:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.713+0000 7fccd9000640 1 -- 172.21.15.37:0/3547180956 wait complete. 2024-09-21T06:33:25.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 -- 172.21.15.37:0/2987328369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 msgr2=0x7f34e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 --2- 172.21.15.37:0/2987328369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f34e8004800 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009b50 tx=0x7f34e8005830 comp rx=0 tx=0).stop 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3505800640 1 -- 172.21.15.37:0/2987328369 reap_dead start 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 -- 172.21.15.37:0/2987328369 shutdown_connections 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 -- 172.21.15.37:0/2987328369 >> 172.21.15.37:0/2987328369 conn(0x7f35000cd570 msgr2=0x7f35000cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 -- 172.21.15.37:0/2987328369 shutdown_connections 2024-09-21T06:33:25.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.966+0000 7f3506200640 1 -- 172.21.15.37:0/2987328369 wait complete. 2024-09-21T06:33:25.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.969+0000 7f3506200640 1 Processor -- start 2024-09-21T06:33:25.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.969+0000 7f3506200640 1 -- start start 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f350014fc60 0x7f35000b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35001f5230 0x7f35001f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f350015f8e0 con 0x7f34e8003ff0 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f350015f760 con 0x7f350014fc60 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3506200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f350015fa60 con 0x7f35001f5230 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.970+0000 7f3504e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53608/0 (socket says 172.21.15.37:53608) 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 -- 172.21.15.37:0/1488945432 learned_addr learned my addr 172.21.15.37:0/1488945432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f350014fc60 0x7f35000b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3505800640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35001f5230 0x7f35001f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 -- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35001f5230 msgr2=0x7f35001f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35001f5230 0x7f35001f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 -- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f350014fc60 msgr2=0x7f35000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f350014fc60 0x7f35000b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 -- 172.21.15.37:0/1488945432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35000b4ff0 con 0x7f34e8003ff0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.971+0000 7f3504e00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f34f4001eb0 tx=0x7f34f400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.972+0000 7f34fcc00640 1 -- 172.21.15.37:0/1488945432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4015070 con 0x7f34e8003ff0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.972+0000 7f34fcc00640 1 -- 172.21.15.37:0/1488945432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f4010980 con 0x7f34e8003ff0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.972+0000 7f34fcc00640 1 -- 172.21.15.37:0/1488945432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4010d50 con 0x7f34e8003ff0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.972+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35000b52d0 con 0x7f34e8003ff0 2024-09-21T06:33:25.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.972+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35000b5890 con 0x7f34e8003ff0 2024-09-21T06:33:25.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.974+0000 7f34fcc00640 1 -- 172.21.15.37:0/1488945432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34f4018070 con 0x7f34e8003ff0 2024-09-21T06:33:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.976+0000 7f34fcc00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.976+0000 7f34fcc00640 1 -- 172.21.15.37:0/1488945432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f34f409b300 con 0x7f34e8003ff0 2024-09-21T06:33:25.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.977+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.977+0000 7f350e9c9740 1 Processor -- start 2024-09-21T06:33:25.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.977+0000 7f350e9c9740 1 -- start start 2024-09-21T06:33:25.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.977+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 0x7f34e007b0d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f34ec035000 tx=0x7f34ec0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 0x55d608251730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d608234400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d60816a1c0 con 0x55d607e426b0 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d60817d870 con 0x55d607e63390 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d60816cca0 con 0x55d607e3a4d0 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53610/0 (socket says 172.21.15.37:53610) 2024-09-21T06:33:25.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.978+0000 7f3505800640 1 -- 172.21.15.37:0/1408128977 learned_addr learned my addr 172.21.15.37:0/1408128977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d608234400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3504e00640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 0x55d608251730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 -- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 msgr2=0x55d608234400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d608234400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 -- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 msgr2=0x55d608251730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 0x55d608251730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 -- 172.21.15.37:0/1408128977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d607f457d0 con 0x55d607e426b0 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3504e00640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 0x55d608251730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d608234400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f3505800640 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009f20 tx=0x7f34f80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1d6221e9f660 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f34f1600640 1 -- 172.21.15.37:0/1408128977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8012030 con 0x55d607e426b0 2024-09-21T06:33:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f34f1600640 1 -- 172.21.15.37:0/1408128977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f80121d0 con 0x55d607e426b0 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.979+0000 7f34f1600640 1 -- 172.21.15.37:0/1408128977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8003c30 con 0x55d607e426b0 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 -- 172.21.15.37:0/1408128977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 msgr2=0x55d608234940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009f20 tx=0x7f34f80128e0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 -- 172.21.15.37:0/1408128977 shutdown_connections 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e426b0 0x55d608234940 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d608234400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1408128977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e63390 0x55d608251730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 -- 172.21.15.37:0/1408128977 >> 172.21.15.37:0/1408128977 conn(0x55d607e574e0 msgr2=0x55d60820a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 -- 172.21.15.37:0/1408128977 shutdown_connections 2024-09-21T06:33:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.980+0000 7f350e9c9740 1 -- 172.21.15.37:0/1408128977 wait complete. 2024-09-21T06:33:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.981+0000 7f350e9c9740 1 -- start start 2024-09-21T06:33:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.982+0000 7f350e9c9740 1 Processor -- start 2024-09-21T06:33:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d60824bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d608248ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d608113410 con 0x55d607e63390 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d608170ab0 con 0x55d607e3a4d0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d607edd740 con 0x55d607e426b0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f350e9c9740 10 client.0 _ll_register_callbacks cb 0x55d6080f0360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.983+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53620/0 (socket says 172.21.15.37:53620) 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 -- 172.21.15.37:0/148198448 learned_addr learned my addr 172.21.15.37:0/148198448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3504e00640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d60824bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f34ffe00640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d608248ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 -- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 msgr2=0x55d608248ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d608248ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 -- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 msgr2=0x55d60824bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d60824bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 -- 172.21.15.37:0/148198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d607ece970 con 0x55d607e63390 2024-09-21T06:33:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f3505800640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f34f8012de0 tx=0x7f34f8010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cbd77e50358452 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f34e6000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8012030 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f80121d0 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.984+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f34f8004b20 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f34f802c050 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 10 client.7113 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55d607eac6a0 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f80121d0 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f34f802f020 con 0x55d607e63390 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 10 client.7113 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 10 client.7113 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.c 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 20 client.7113 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d607e21f60 0x55d608212c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 4 client.7113 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d60817f7c0 con 0x55d607e21f60 2024-09-21T06:33:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.985+0000 7f3504e00640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d607e21f60 0x55d608212c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.986+0000 7f3504e00640 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d607e21f60 0x55d608212c10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65951082248afc05 server_cookie=d62d9ff8420fd384 in_seq=0 out_seq=0 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.986+0000 7f34e6000640 10 client.7113 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f34e6000640 1 -- 172.21.15.37:0/148198448 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d60817f7c0 con 0x55d607e21f60 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f34e6000640 10 client.7113 handle_command_reply: tid=0 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 client.7113 shutdown 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 msgr2=0x55d60824e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f34f8012de0 tx=0x7f34f8010b50 comp rx=0 tx=0).stop 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 shutdown_connections 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d607e21f60 0x55d608212c10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824e820 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d608248ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 --2- 172.21.15.37:0/148198448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d60824bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f34e6000640 0 client.7113 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 >> 172.21.15.37:0/148198448 conn(0x55d607e574e0 msgr2=0x55d608223bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 shutdown_connections 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 1 -- 172.21.15.37:0/148198448 wait complete. 2024-09-21T06:33:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:25.987+0000 7f350e9c9740 20 client.7113 trim_cache size 0 max 16384 2024-09-21T06:33:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.003+0000 7f350e9c9740 1 Processor -- start 2024-09-21T06:33:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.003+0000 7f350e9c9740 1 -- start start 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d6082348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d60824b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3500202ac0 con 0x55d607e63390 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3500201c90 con 0x55d607e3a4d0 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.004+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3500202390 con 0x55d607e426b0 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53634/0 (socket says 172.21.15.37:53634) 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 -- 172.21.15.37:0/1108556719 learned_addr learned my addr 172.21.15.37:0/1108556719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3504e00640 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d6082348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f34ffe00640 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d60824b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 -- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 msgr2=0x55d60824b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d60824b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 -- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 msgr2=0x55d6082348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d6082348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.005+0000 7f3505800640 1 -- 172.21.15.37:0/1108556719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d60820b910 con 0x55d607e63390 2024-09-21T06:33:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.006+0000 7f3505800640 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009d70 tx=0x7f34f8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c5a9e3b9262e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.006+0000 7f34f0c00640 1 -- 172.21.15.37:0/1108556719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8035070 con 0x55d607e63390 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.006+0000 7f34f0c00640 1 -- 172.21.15.37:0/1108556719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f80094d0 con 0x55d607e63390 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.006+0000 7f34f0c00640 1 -- 172.21.15.37:0/1108556719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8026430 con 0x55d607e63390 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 -- 172.21.15.37:0/1108556719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 msgr2=0x55d60824bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009d70 tx=0x7f34f8010740 comp rx=0 tx=0).stop 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 -- 172.21.15.37:0/1108556719 shutdown_connections 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d60824bc10 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e426b0 0x55d60824b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 --2- 172.21.15.37:0/1108556719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e3a4d0 0x55d6082348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 -- 172.21.15.37:0/1108556719 >> 172.21.15.37:0/1108556719 conn(0x55d607e574e0 msgr2=0x55d608222270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 -- 172.21.15.37:0/1108556719 shutdown_connections 2024-09-21T06:33:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.007+0000 7f350e9c9740 1 -- 172.21.15.37:0/1108556719 wait complete. 2024-09-21T06:33:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.008+0000 7f350e9c9740 1 -- start start 2024-09-21T06:33:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.008+0000 7f350e9c9740 1 Processor -- start 2024-09-21T06:33:26.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d6082c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e426b0 0x55d6082cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35000ae370 con 0x55d607e63390 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d6082cfbd0 con 0x55d607e426b0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d6082517d0 con 0x55d607e3a4d0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.009+0000 7f350e9c9740 10 client.0 _ll_register_callbacks cb 0x7f34f8012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53636/0 (socket says 172.21.15.37:53636) 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 -- 172.21.15.37:0/2738923612 learned_addr learned my addr 172.21.15.37:0/2738923612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f34ffe00640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e426b0 0x55d6082cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3504e00640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d6082c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 -- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 msgr2=0x55d6082c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d6082c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 -- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e426b0 msgr2=0x55d6082cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e426b0 0x55d6082cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 -- 172.21.15.37:0/2738923612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d608251ab0 con 0x55d607e63390 2024-09-21T06:33:26.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f3505800640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009d40 tx=0x7f34f8027a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cf417e726df73f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.010+0000 7f34e6000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8035070 con 0x55d607e63390 2024-09-21T06:33:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f34f801c410 con 0x55d607e63390 2024-09-21T06:33:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f80094d0 con 0x55d607e63390 2024-09-21T06:33:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f34f8010bc0 con 0x55d607e63390 2024-09-21T06:33:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 10 client.7119 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55d608222480 con 0x55d607e63390 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f80094d0 con 0x55d607e63390 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f34f802f040 con 0x55d607e63390 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 10 client.7119 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 10 client.7119 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.c 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.011+0000 7f350e9c9740 20 client.7119 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d608207060 0x55d60824b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f350e9c9740 4 client.7119 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d60817b270 con 0x55d608207060 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f3504e00640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d608207060 0x55d60824b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f3504e00640 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d608207060 0x55d60824b6d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c12879b10bfc16c9 server_cookie=c436fe4ddc0a84a8 in_seq=0 out_seq=0 2024-09-21T06:33:26.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.012+0000 7f34e6000640 10 client.7119 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f34e6000640 1 -- 172.21.15.37:0/2738923612 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d60817b270 con 0x55d608207060 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f34e6000640 10 client.7119 handle_command_reply: tid=0 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 client.7119 shutdown 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 msgr2=0x55d608210b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009d40 tx=0x7f34f8027a80 comp rx=0 tx=0).stop 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 shutdown_connections 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55d608207060 0x55d60824b6d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d607e63390 0x55d608210b60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d607e426b0 0x55d6082cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f34e6000640 0 client.7119 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 --2- 172.21.15.37:0/2738923612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d607e3a4d0 0x55d6082c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.023+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 >> 172.21.15.37:0/2738923612 conn(0x55d607e574e0 msgr2=0x55d608260e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.024+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 shutdown_connections 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.024+0000 7f350e9c9740 1 -- 172.21.15.37:0/2738923612 wait complete. 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.024+0000 7f350e9c9740 20 client.7119 trim_cache size 0 max 16384 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:33:26.027 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:26.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.024+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 msgr2=0x7f34e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 0x7f34e007b0d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f34ec035000 tx=0x7f34ec0046c0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 msgr2=0x7f35000b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f34f4001eb0 tx=0x7f34f400ea20 comp rx=0 tx=0).stop 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 shutdown_connections 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35001f5230 0x7f35001f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f350014fc60 0x7f35000b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 --2- 172.21.15.37:0/1488945432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e8003ff0 0x7f35000b6090 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 >> 172.21.15.37:0/1488945432 conn(0x7f35000cd570 msgr2=0x7f35000ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 shutdown_connections 2024-09-21T06:33:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.025+0000 7f3506200640 1 -- 172.21.15.37:0/1488945432 wait complete. 2024-09-21T06:33:26.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2024-09-21T06:33:26.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 -- 172.21.15.37:0/3573977474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc0cc530 msgr2=0x7fa3bc15ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3573977474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc15ead0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc1624c0 tx=0x7fa3ac02a560 comp rx=0 tx=0).stop 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 -- 172.21.15.37:0/3573977474 shutdown_connections 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3573977474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc06e330 0x7fa3bc0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3573977474 >> v2:172.21.15.150:3300/0 conn(0x7fa3bc0af5b0 0x7fa3bc06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3573977474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc15ead0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 -- 172.21.15.37:0/3573977474 >> 172.21.15.37:0/3573977474 conn(0x7fa3bc0b1b90 msgr2=0x7fa3bc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 -- 172.21.15.37:0/3573977474 shutdown_connections 2024-09-21T06:33:26.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.278+0000 7fa3c1200640 1 -- 172.21.15.37:0/3573977474 wait complete. 2024-09-21T06:33:26.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.280+0000 7fa3c1200640 1 Processor -- start 2024-09-21T06:33:26.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.281+0000 7fa3c1200640 1 -- start start 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc0af5b0 0x7fa3bc1f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc1f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cadb0 con 0x7fa3bc06e330 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cac30 con 0x7fa3bc0cc530 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3c1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc0caf30 con 0x7fa3bc0af5b0 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53654/0 (socket says 172.21.15.37:53654) 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3109346838 learned_addr learned my addr 172.21.15.37:0/3109346838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3ba800640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc1f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.282+0000 7fa3b1400640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc0af5b0 0x7fa3bc1f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc0af5b0 msgr2=0x7fa3bc1f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc0af5b0 0x7fa3bc1f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3bc0cc530 msgr2=0x7fa3bc1f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc1f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3109346838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc0b3260 con 0x7fa3bc06e330 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac034040 tx=0x7fa3ac003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b2a00640 1 -- 172.21.15.37:0/3109346838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac03e030 con 0x7fa3bc06e330 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b2a00640 1 -- 172.21.15.37:0/3109346838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ac003e00 con 0x7fa3bc06e330 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3b2a00640 1 -- 172.21.15.37:0/3109346838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac035040 con 0x7fa3bc06e330 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.283+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc0b3540 con 0x7fa3bc06e330 2024-09-21T06:33:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.284+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc0b3b00 con 0x7fa3bc06e330 2024-09-21T06:33:26.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.286+0000 7fa3b2a00640 1 -- 172.21.15.37:0/3109346838 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3ac041030 con 0x7fa3bc06e330 2024-09-21T06:33:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.287+0000 7fa3b2a00640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 0x7fa39807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.287+0000 7fa3b2a00640 1 -- 172.21.15.37:0/3109346838 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa3ac0bb660 con 0x7fa3bc06e330 2024-09-21T06:33:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.287+0000 7fa3c8420740 1 Processor -- start 2024-09-21T06:33:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.288+0000 7fa3b1400640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 0x7fa39807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.288+0000 7fa3c8420740 1 -- start start 2024-09-21T06:33:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.288+0000 7fa3b1400640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 0x7fa39807b0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc1f3160 tx=0x7fa3a400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc034001c0 con 0x55dc030d04d0 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dc03413870 con 0x55dc030d86b0 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dc03402ca0 con 0x55dc030f9390 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53658/0 (socket says 172.21.15.37:53658) 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.289+0000 7fa3b1400640 1 -- 172.21.15.37:0/151207375 learned_addr learned my addr 172.21.15.37:0/151207375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3ba800640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 -- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 msgr2=0x55dc034e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 -- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 msgr2=0x55dc034ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 -- 172.21.15.37:0/151207375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc031db7d0 con 0x55dc030d04d0 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3b1400640 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4010f80 tx=0x7fa3a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe640274b648d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3aaa00640 1 -- 172.21.15.37:0/151207375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a401e620 con 0x55dc030d04d0 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3aaa00640 1 -- 172.21.15.37:0/151207375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3a401e7c0 con 0x55dc030d04d0 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.290+0000 7fa3aaa00640 1 -- 172.21.15.37:0/151207375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a401eb90 con 0x55dc030d04d0 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 -- 172.21.15.37:0/151207375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 msgr2=0x55dc034ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4010f80 tx=0x7fa3a4013000 comp rx=0 tx=0).stop 2024-09-21T06:33:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 -- 172.21.15.37:0/151207375 shutdown_connections 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034ca220 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 --2- 172.21.15.37:0/151207375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 -- 172.21.15.37:0/151207375 >> 172.21.15.37:0/151207375 conn(0x55dc030ed4e0 msgr2=0x55dc034a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 -- 172.21.15.37:0/151207375 shutdown_connections 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.291+0000 7fa3c8420740 1 -- 172.21.15.37:0/151207375 wait complete. 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.293+0000 7fa3c8420740 1 -- start start 2024-09-21T06:33:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.293+0000 7fa3c8420740 1 Processor -- start 2024-09-21T06:33:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc033a9410 con 0x55dc030d04d0 2024-09-21T06:33:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dc03406ab0 con 0x55dc030d86b0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dc03173740 con 0x55dc030f9390 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.294+0000 7fa3c8420740 10 client.0 _ll_register_callbacks cb 0x55dc03386360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53670/0 (socket says 172.21.15.37:53670) 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 -- 172.21.15.37:0/836576802 learned_addr learned my addr 172.21.15.37:0/836576802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b1400640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3ba800640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 -- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 msgr2=0x55dc034e0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 -- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 msgr2=0x55dc034e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 -- 172.21.15.37:0/836576802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc03164970 con 0x55dc030d04d0 2024-09-21T06:33:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0099c0 tx=0x7fa3ac082c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a75f5108281aba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.295+0000 7fa39f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:26.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac048040 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa3ac08c900 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a401e7c0 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa3ac0832b0 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 10 client.7131 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55dc031426a0 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3a401e7c0 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7fa3ac082510 con 0x55dc030d04d0 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 10 client.7131 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 10 client.7131 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.a 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 20 client.7131 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc030b7f60 0x55dc034f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 4 client.7131 mds_command: new command op to 6852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:26.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.296+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 --> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc034157c0 con 0x55dc030b7f60 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.297+0000 7fa3b1400640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc030b7f60 0x55dc034f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.297+0000 7fa3b1400640 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc030b7f60 0x55dc034f7b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=971ab2aed7299a20 server_cookie=a1faa91796fe9aa2 in_seq=0 out_seq=0 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.297+0000 7fa39f400640 10 client.7131 ms_handle_connect on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa39f400640 1 -- 172.21.15.37:0/836576802 <== mds.2 v2:172.21.15.37:6832/3081570660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc034157c0 con 0x55dc030b7f60 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa39f400640 10 client.7131 handle_command_reply: tid=0 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 client.7131 shutdown 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 msgr2=0x55dc034f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0099c0 tx=0x7fa3ac082c00 comp rx=0 tx=0).stop 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 shutdown_connections 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc030b7f60 0x55dc034f7b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030f9390 0x55dc034e0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030d86b0 0x55dc034e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 --2- 172.21.15.37:0/836576802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034f3e50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa39f400640 0 client.7131 ms_handle_reset on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 >> 172.21.15.37:0/836576802 conn(0x55dc030ed4e0 msgr2=0x55dc034e85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 shutdown_connections 2024-09-21T06:33:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.298+0000 7fa3c8420740 1 -- 172.21.15.37:0/836576802 wait complete. 2024-09-21T06:33:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.299+0000 7fa3c8420740 20 client.7131 trim_cache size 0 max 16384 2024-09-21T06:33:26.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.314+0000 7fa3c8420740 1 Processor -- start 2024-09-21T06:33:26.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.314+0000 7fa3c8420740 1 -- start start 2024-09-21T06:33:26.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc034f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc034f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc202600 con 0x55dc030d04d0 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc201830 con 0x55dc030f9390 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc201ed0 con 0x55dc030d86b0 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53686/0 (socket says 172.21.15.37:53686) 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3135427698 learned_addr learned my addr 172.21.15.37:0/3135427698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3ba800640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc034f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b1400640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc034f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 msgr2=0x55dc034f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc034f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 msgr2=0x55dc034f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc034f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b9e00640 1 -- 172.21.15.37:0/3135427698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc034e11f0 con 0x55dc030d04d0 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3b1400640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc034f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.316+0000 7fa3ba800640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc034f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.317+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02ab50 tx=0x7fa3ac005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f73eeb8aad785d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.317+0000 7fa3aa000640 1 -- 172.21.15.37:0/3135427698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac048070 con 0x55dc030d04d0 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.317+0000 7fa3aa000640 1 -- 172.21.15.37:0/3135427698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ac08cb90 con 0x55dc030d04d0 2024-09-21T06:33:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.317+0000 7fa3aa000640 1 -- 172.21.15.37:0/3135427698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac08c900 con 0x55dc030d04d0 2024-09-21T06:33:26.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 -- 172.21.15.37:0/3135427698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 msgr2=0x55dc034debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02ab50 tx=0x7fa3ac005f10 comp rx=0 tx=0).stop 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 -- 172.21.15.37:0/3135427698 shutdown_connections 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc034f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc034f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 --2- 172.21.15.37:0/3135427698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034debf0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 -- 172.21.15.37:0/3135427698 >> 172.21.15.37:0/3135427698 conn(0x55dc030ed4e0 msgr2=0x55dc034e85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 -- 172.21.15.37:0/3135427698 shutdown_connections 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.318+0000 7fa3c8420740 1 -- 172.21.15.37:0/3135427698 wait complete. 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.319+0000 7fa3c8420740 1 -- start start 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.319+0000 7fa3c8420740 1 Processor -- start 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc03563900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc03499fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc034dfda0 con 0x55dc030d04d0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc1f5fd0 con 0x55dc030f9390 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55dc0349d7b0 con 0x55dc030d86b0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.320+0000 7fa3c8420740 10 client.0 _ll_register_callbacks cb 0x7fa3a4005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53696/0 (socket says 172.21.15.37:53696) 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 -- 172.21.15.37:0/1454456495 learned_addr learned my addr 172.21.15.37:0/1454456495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3ba800640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc03499fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b1400640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc03563900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 -- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 msgr2=0x55dc03563900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc03563900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 -- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 msgr2=0x55dc03499fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc03499fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 -- 172.21.15.37:0/1454456495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc03496510 con 0x55dc030d04d0 2024-09-21T06:33:26.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa3b9e00640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02ab50 tx=0x7fa3ac07ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c5d70a5c3e58d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa39f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.321+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac048070 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa3ac081f80 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3ac08cb90 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa3ac07f150 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 10 client.7137 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55dc03496510 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ac08cb90 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7fa3ac082180 con 0x55dc030d04d0 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 10 client.7137 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 10 client.7137 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.a 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 20 client.7137 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:26.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc0349ebb0 0x55dc034f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 4 client.7137 mds_command: new command op to 6852 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:33:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 --> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc03411270 con 0x55dc0349ebb0 2024-09-21T06:33:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.322+0000 7fa3b1400640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc0349ebb0 0x55dc034f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.323+0000 7fa3b1400640 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc0349ebb0 0x55dc034f34f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e577c1859ca8fe9a server_cookie=e424c03740ff710d in_seq=0 out_seq=0 2024-09-21T06:33:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.323+0000 7fa39f400640 10 client.7137 ms_handle_connect on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa39f400640 1 -- 172.21.15.37:0/1454456495 <== mds.2 v2:172.21.15.37:6832/3081570660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc03411270 con 0x55dc0349ebb0 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa39f400640 10 client.7137 handle_command_reply: tid=0 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 client.7137 shutdown 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 msgr2=0x55dc034c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02ab50 tx=0x7fa3ac07ee30 comp rx=0 tx=0).stop 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 shutdown_connections 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:6832/3081570660,v1:172.21.15.37:6833/3081570660] conn(0x55dc0349ebb0 0x55dc034f34f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc030f9390 0x55dc03499fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc030d86b0 0x55dc03563900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 --2- 172.21.15.37:0/1454456495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc030d04d0 0x55dc034c70f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa39f400640 0 client.7137 ms_handle_reset on v2:172.21.15.37:6832/3081570660 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 >> 172.21.15.37:0/1454456495 conn(0x55dc030ed4e0 msgr2=0x55dc0349b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 shutdown_connections 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.335+0000 7fa3c8420740 1 -- 172.21.15.37:0/1454456495 wait complete. 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c8420740 20 client.7137 trim_cache size 0 max 16384 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:33:26.339 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 msgr2=0x7fa39807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 0x7fa39807b0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc1f3160 tx=0x7fa3a400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 msgr2=0x7fa3bc0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac034040 tx=0x7fa3ac003910 comp rx=0 tx=0).stop 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 shutdown_connections 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa398078b90 0x7fa39807b0a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3bc0cc530 0x7fa3bc1f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3bc0af5b0 0x7fa3bc1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 --2- 172.21.15.37:0/3109346838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3bc06e330 0x7fa3bc0b5a70 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 >> 172.21.15.37:0/3109346838 conn(0x7fa3bc0b1b90 msgr2=0x7fa3bc06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.336+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 shutdown_connections 2024-09-21T06:33:26.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.337+0000 7fa3c1200640 1 -- 172.21.15.37:0/3109346838 wait complete. 2024-09-21T06:33:26.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 -- 172.21.15.37:0/2748624249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 msgr2=0x7fdb940cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 --2- 172.21.15.37:0/2748624249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb940cc020 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdb94076360 tx=0x7fdb7c02a560 comp rx=0 tx=0).stop 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 -- 172.21.15.37:0/2748624249 shutdown_connections 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 --2- 172.21.15.37:0/2748624249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb9406ddf0 0x7fdb9406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 --2- 172.21.15.37:0/2748624249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb940cc020 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 --2- 172.21.15.37:0/2748624249 >> v2:172.21.15.150:3300/0 conn(0x7fdb94074cb0 0x7fdb94073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 -- 172.21.15.37:0/2748624249 >> 172.21.15.37:0/2748624249 conn(0x7fdb940cca40 msgr2=0x7fdb940cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 -- 172.21.15.37:0/2748624249 shutdown_connections 2024-09-21T06:33:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.588+0000 7fdb99800640 1 -- 172.21.15.37:0/2748624249 wait complete. 2024-09-21T06:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.590+0000 7fdb99800640 1 Processor -- start 2024-09-21T06:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.591+0000 7fdb99800640 1 -- start start 2024-09-21T06:33:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9406ddf0 0x7fdb940b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb94074cb0 0x7fdb9409bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9406f170 con 0x7fdb940cbc10 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9406eff0 con 0x7fdb9406ddf0 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb99800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9406f2f0 con 0x7fdb94074cb0 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb93400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb93400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53710/0 (socket says 172.21.15.37:53710) 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.592+0000 7fdb93400640 1 -- 172.21.15.37:0/1985663038 learned_addr learned my addr 172.21.15.37:0/1985663038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb92a00640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9406ddf0 0x7fdb940b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb8a000640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb94074cb0 0x7fdb9409bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 -- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb94074cb0 msgr2=0x7fdb9409bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb94074cb0 0x7fdb9409bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 -- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9406ddf0 msgr2=0x7fdb940b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9406ddf0 0x7fdb940b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 -- 172.21.15.37:0/1985663038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb94157e50 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.593+0000 7fdb93400640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00ea00 tx=0x7fdb8c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.594+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00cde0 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.594+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb8c004b20 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.594+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c005020 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.594+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb94158130 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.594+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9409e420 con 0x7fdb940cbc10 2024-09-21T06:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.595+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94157bc0 con 0x7fdb940cbc10 2024-09-21T06:33:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.596+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb8c0051c0 con 0x7fdb940cbc10 2024-09-21T06:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.597+0000 7fdb8b400640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 0x7fdb7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.597+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fdb8c09ba20 con 0x7fdb940cbc10 2024-09-21T06:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.598+0000 7fdb92a00640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 0x7fdb7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.598+0000 7fdb92a00640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 0x7fdb7007b0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415a730 tx=0x7fdb8400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.602+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb8c0635e0 con 0x7fdb940cbc10 2024-09-21T06:33:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.834+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb9415c3d0 con 0x7fdb940cbc10 2024-09-21T06:33:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.835+0000 7fdb8b400640 1 -- 172.21.15.37:0/1985663038 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5273 (secure 0 0 0) 0x7fdb8c068010 con 0x7fdb940cbc10 2024-09-21T06:33:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:26.837 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"btime":"2024-09-21T06:33:24:907118+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:24.856871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6852,6858,6870]},"id":5}]} 2024-09-21T06:33:26.838 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 79 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 msgr2=0x7fdb7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 0x7fdb7007b0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415a730 tx=0x7fdb8400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 msgr2=0x7fdb9409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00ea00 tx=0x7fdb8c00eec0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 shutdown_connections 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdb70078bc0 0x7fdb7007b0d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb940cbc10 0x7fdb9409c480 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb94074cb0 0x7fdb9409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 --2- 172.21.15.37:0/1985663038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9406ddf0 0x7fdb940b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 >> 172.21.15.37:0/1985663038 conn(0x7fdb940cca40 msgr2=0x7fdb9415e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 shutdown_connections 2024-09-21T06:33:26.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:26.837+0000 7fdb99800640 1 -- 172.21.15.37:0/1985663038 wait complete. 2024-09-21T06:33:26.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub start /d1/d2/d3 2024-09-21T06:33:27.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 -- 172.21.15.37:0/3715632176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 msgr2=0x7f201c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 --2- 172.21.15.37:0/3715632176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 0x7f201c00b9c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2030005fd0 tx=0x7f201c005830 comp rx=0 tx=0).stop 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f2036800640 1 -- 172.21.15.37:0/3715632176 reap_dead start 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 -- 172.21.15.37:0/3715632176 shutdown_connections 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 -- 172.21.15.37:0/3715632176 >> 172.21.15.37:0/3715632176 conn(0x7f20380cca40 msgr2=0x7f20380cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 -- 172.21.15.37:0/3715632176 shutdown_connections 2024-09-21T06:33:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.095+0000 7f203d000640 1 -- 172.21.15.37:0/3715632176 wait complete. 2024-09-21T06:33:27.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.098+0000 7f203d000640 1 Processor -- start 2024-09-21T06:33:27.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.098+0000 7f203d000640 1 -- start start 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 0x7f20380c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20380b4020 0x7f2038158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20380b30c0 0x7f2038159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f203806e680 con 0x7f201c005150 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f203806e500 con 0x7f20380b4020 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f203d000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f20380cde00 con 0x7f20380b30c0 2024-09-21T06:33:27.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f2035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 0x7f20380c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f2035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 0x7f20380c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53740/0 (socket says 172.21.15.37:53740) 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.099+0000 7f2035e00640 1 -- 172.21.15.37:0/422249390 learned_addr learned my addr 172.21.15.37:0/422249390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2036800640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20380b30c0 0x7f2038159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f202d400640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20380b4020 0x7f2038158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 -- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20380b30c0 msgr2=0x7f2038159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20380b30c0 0x7f2038159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 -- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20380b4020 msgr2=0x7f2038158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20380b4020 0x7f2038158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 -- 172.21.15.37:0/422249390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038150e70 con 0x7f201c005150 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f202d400640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20380b4020 0x7f2038158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f2035e00640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f201c005150 0x7f20380c9d70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2028011ae0 tx=0x7f2028009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.100+0000 7f202ea00640 1 -- 172.21.15.37:0/422249390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801e070 con 0x7f201c005150 2024-09-21T06:33:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.101+0000 7f202ea00640 1 -- 172.21.15.37:0/422249390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f202800fc70 con 0x7f201c005150 2024-09-21T06:33:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.101+0000 7f202ea00640 1 -- 172.21.15.37:0/422249390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028019410 con 0x7f201c005150 2024-09-21T06:33:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.101+0000 7f203d000640 1 -- 172.21.15.37:0/422249390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20380b5af0 con 0x7f201c005150 2024-09-21T06:33:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.101+0000 7f203d000640 1 -- 172.21.15.37:0/422249390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381f8b10 con 0x7f201c005150 2024-09-21T06:33:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.103+0000 7f202ea00640 1 -- 172.21.15.37:0/422249390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2028021070 con 0x7f201c005150 2024-09-21T06:33:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.104+0000 7f202ea00640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2014078b90 0x7f201407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.104+0000 7f202ea00640 1 -- 172.21.15.37:0/422249390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f202809b750 con 0x7f201c005150 2024-09-21T06:33:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.105+0000 7f202d400640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2014078b90 0x7f201407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.105+0000 7f2044223740 1 Processor -- start 2024-09-21T06:33:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.105+0000 7f2044223740 1 -- start start 2024-09-21T06:33:27.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.105+0000 7f202d400640 1 --2- 172.21.15.37:0/422249390 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2014078b90 0x7f201407b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2020001eb0 tx=0x7f20200017e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.106+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.106+0000 7f2044223740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.106+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ea2540 0x556123295060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.106+0000 7f2044223740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561231dd850 con 0x556122e9a360 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.106+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5561231ccc80 con 0x556122ea2540 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5561231d0a90 con 0x556122ec3220 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53746/0 (socket says 172.21.15.37:53746) 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 -- 172.21.15.37:0/1883675095 learned_addr learned my addr 172.21.15.37:0/1883675095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 -- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 msgr2=0x5561232b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 -- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ea2540 msgr2=0x556123295060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ea2540 0x556123295060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 -- 172.21.15.37:0/1883675095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556122ebf280 con 0x556122e9a360 2024-09-21T06:33:27.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.107+0000 7f202d400640 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2020010e00 tx=0x7f202000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78533f25e16276a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.108+0000 7f2026a00640 1 -- 172.21.15.37:0/1883675095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020012dd0 con 0x556122e9a360 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.108+0000 7f2026a00640 1 -- 172.21.15.37:0/1883675095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020024070 con 0x556122e9a360 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.108+0000 7f2026a00640 1 -- 172.21.15.37:0/1883675095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202001f660 con 0x556122e9a360 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.108+0000 7f2044223740 1 -- 172.21.15.37:0/1883675095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 msgr2=0x556123294b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.108+0000 7f2044223740 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2020010e00 tx=0x7f202000c000 comp rx=0 tx=0).stop 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 -- 172.21.15.37:0/1883675095 shutdown_connections 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123294b20 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ea2540 0x556123295060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 --2- 172.21.15.37:0/1883675095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 -- 172.21.15.37:0/1883675095 >> 172.21.15.37:0/1883675095 conn(0x556122eb7370 msgr2=0x55612326ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 -- 172.21.15.37:0/1883675095 shutdown_connections 2024-09-21T06:33:27.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.109+0000 7f2044223740 1 -- 172.21.15.37:0/1883675095 wait complete. 2024-09-21T06:33:27.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.110+0000 7f2044223740 1 -- start start 2024-09-21T06:33:27.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.110+0000 7f2044223740 1 Processor -- start 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556122f3d5d0 con 0x556122ea2540 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556123172550 con 0x556122e9a360 2024-09-21T06:33:27.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5561232c0dd0 con 0x556122ec3220 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.111+0000 7f2044223740 10 client.0 _ll_register_callbacks cb 0x5561231c6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53762/0 (socket says 172.21.15.37:53762) 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 -- 172.21.15.37:0/1148050976 learned_addr learned my addr 172.21.15.37:0/1148050976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f2036800640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f2035e00640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 -- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 msgr2=0x5561232beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 -- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 msgr2=0x556123291ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.112+0000 7f202d400640 1 -- 172.21.15.37:0/1148050976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556122fa7690 con 0x556122ea2540 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f202d400640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2020012ca0 tx=0x7f202001f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbad36ade34b247 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202001fbc0 con 0x556122ea2540 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f202001fd60 con 0x556122ea2540 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2020024070 con 0x556122ea2540 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f202003c050 con 0x556122ea2540 2024-09-21T06:33:27.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f2044223740 10 client.7155 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x556123007e20 con 0x556122ea2540 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020024070 con 0x556122ea2540 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.113+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f202003f020 con 0x556122ea2540 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 10 client.7155 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 10 client.7155 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 20 client.7155 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556122e82000 0x5561232c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 4 client.7155 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561231df7a0 con 0x556122e82000 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2035e00640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556122e82000 0x5561232c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f2035e00640 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556122e82000 0x5561232c20d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0415c06c4ffa58b server_cookie=16b95042e5a3d554 in_seq=0 out_seq=0 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.114+0000 7f201b400640 10 client.7155 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.115+0000 7f201b400640 1 -- 172.21.15.37:0/1148050976 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561231df7a0 con 0x556122e82000 2024-09-21T06:33:27.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.115+0000 7f201b400640 10 client.7155 handle_command_reply: tid=0 2024-09-21T06:33:27.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.115+0000 7f2044223740 1 client.7155 shutdown 2024-09-21T06:33:27.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.115+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 msgr2=0x5561232a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.115+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2020012ca0 tx=0x7f202001f690 comp rx=0 tx=0).stop 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 shutdown_connections 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556122e82000 0x5561232c20d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9ef0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x5561232beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 --2- 172.21.15.37:0/1148050976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f201b400640 0 client.7155 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 >> 172.21.15.37:0/1148050976 conn(0x556122eb7370 msgr2=0x556123263fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 shutdown_connections 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 1 -- 172.21.15.37:0/1148050976 wait complete. 2024-09-21T06:33:27.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.116+0000 7f2044223740 20 client.7155 trim_cache size 0 max 16384 2024-09-21T06:33:27.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.135+0000 7f2044223740 1 Processor -- start 2024-09-21T06:33:27.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.135+0000 7f2044223740 1 -- start start 2024-09-21T06:33:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x556123294d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2038201660 con 0x556122ea2540 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2038201d20 con 0x556122e9a360 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f20380a0570 con 0x556122ec3220 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f202d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53778/0 (socket says 172.21.15.37:53778) 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.136+0000 7f202d400640 1 -- 172.21.15.37:0/1704958507 learned_addr learned my addr 172.21.15.37:0/1704958507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f2035e00640 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f2036800640 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x556123294d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 -- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 msgr2=0x556123294d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x556123294d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 -- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 msgr2=0x556123291270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 -- 172.21.15.37:0/1704958507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556122f0c530 con 0x556122ea2540 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f202d400640 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2020010dd0 tx=0x7f202000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9d37ccd70256c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f2026000640 1 -- 172.21.15.37:0/1704958507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020045070 con 0x556122ea2540 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f2026000640 1 -- 172.21.15.37:0/1704958507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020024070 con 0x556122ea2540 2024-09-21T06:33:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.137+0000 7f2026000640 1 -- 172.21.15.37:0/1704958507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202001fcf0 con 0x556122ea2540 2024-09-21T06:33:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 -- 172.21.15.37:0/1704958507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 msgr2=0x5561232a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2020010dd0 tx=0x7f202000c000 comp rx=0 tx=0).stop 2024-09-21T06:33:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 -- 172.21.15.37:0/1704958507 shutdown_connections 2024-09-21T06:33:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122ea2540 0x5561232a9c40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ec3220 0x556123294d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 --2- 172.21.15.37:0/1704958507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 -- 172.21.15.37:0/1704958507 >> 172.21.15.37:0/1704958507 conn(0x556122eb7370 msgr2=0x5561232ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 -- 172.21.15.37:0/1704958507 shutdown_connections 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.138+0000 7f2044223740 1 -- 172.21.15.37:0/1704958507 wait complete. 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.139+0000 7f2044223740 1 -- start start 2024-09-21T06:33:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.139+0000 7f2044223740 1 Processor -- start 2024-09-21T06:33:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ea2540 0x5561232917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ec3220 0x5561232a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561232bf860 con 0x556122e9a360 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55612328e230 con 0x556122ec3220 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5561232bfe00 con 0x556122ea2540 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2044223740 10 client.0 _ll_register_callbacks cb 0x7f2020010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53784/0 (socket says 172.21.15.37:53784) 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 -- 172.21.15.37:0/3697237897 learned_addr learned my addr 172.21.15.37:0/3697237897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f202d400640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ea2540 0x5561232917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2036800640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ec3220 0x5561232a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 -- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ea2540 msgr2=0x5561232917b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ea2540 0x5561232917b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 -- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ec3220 msgr2=0x5561232a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ec3220 0x5561232a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.141+0000 7f2035e00640 1 -- 172.21.15.37:0/3697237897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556123294bc0 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f2035e00640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f20280630a0 tx=0x7f202805fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440ef02f513ac6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f201b400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202806c9f0 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2028062600 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2020024070 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f202806cc80 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f2044223740 10 client.7161 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55612332e7c0 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.142+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020024070 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f202804ace0 con 0x556122e9a360 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 10 client.7161 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 10 client.7161 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 20 client.7161 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556123265f80 0x55612332d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 4 client.7161 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) -- 0x5561231db250 con 0x556123265f80 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f202d400640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556123265f80 0x55612332d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f202d400640 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556123265f80 0x55612332d710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=205a1fa1f83d6daa server_cookie=5e44564d37ec0232 in_seq=0 out_seq=0 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.143+0000 7f201b400640 10 client.7161 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f201b400640 1 -- 172.21.15.37:0/3697237897 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x5561231db250 con 0x556123265f80 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f201b400640 10 client.7161 handle_command_reply: tid=0 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 client.7161 shutdown 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 msgr2=0x556123291270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f20280630a0 tx=0x7f202805fb20 comp rx=0 tx=0).stop 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 shutdown_connections 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556123265f80 0x55612332d710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556122ec3220 0x5561232a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556122ea2540 0x5561232917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 --2- 172.21.15.37:0/3697237897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556122e9a360 0x556123291270 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 >> 172.21.15.37:0/3697237897 conn(0x556122eb7370 msgr2=0x5561232a9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f201b400640 0 client.7161 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 shutdown_connections 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 1 -- 172.21.15.37:0/3697237897 wait complete. 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.144+0000 7f2044223740 20 client.7161 trim_cache size 0 max 16384 2024-09-21T06:33:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:Error EXDEV: Not rank 0 2024-09-21T06:33:27.160 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-21T06:33:27.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub abort 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 -- 172.21.15.37:0/1552919496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c5006e500 msgr2=0x7f2c500cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 --2- 172.21.15.37:0/1552919496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c5006e500 0x7f2c500cef00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2c500b2c70 tx=0x7f2c4802aac0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 -- 172.21.15.37:0/1552919496 shutdown_connections 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 --2- 172.21.15.37:0/1552919496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c5006e500 0x7f2c500cef00 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 --2- 172.21.15.37:0/1552919496 >> v2:172.21.15.150:3300/0 conn(0x7f2c50156a50 0x7f2c50074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 -- 172.21.15.37:0/1552919496 >> 172.21.15.37:0/1552919496 conn(0x7f2c500b1b90 msgr2=0x7f2c500b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 -- 172.21.15.37:0/1552919496 shutdown_connections 2024-09-21T06:33:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.388+0000 7f2c56000640 1 -- 172.21.15.37:0/1552919496 wait complete. 2024-09-21T06:33:27.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.391+0000 7f2c56000640 1 Processor -- start 2024-09-21T06:33:27.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.391+0000 7f2c56000640 1 -- start start 2024-09-21T06:33:27.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50074b60 0x7f2c501f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c50156a50 0x7f2c500b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c500b35f0 0x7f2c5009fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500cf7d0 con 0x7f2c50074b60 2024-09-21T06:33:27.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500cf650 con 0x7f2c50156a50 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c56000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500cf950 con 0x7f2c500b35f0 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.392+0000 7f2c4f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50074b60 0x7f2c501f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50074b60 0x7f2c501f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53798/0 (socket says 172.21.15.37:53798) 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 -- 172.21.15.37:0/1683517199 learned_addr learned my addr 172.21.15.37:0/1683517199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c500b35f0 0x7f2c5009fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c50156a50 0x7f2c500b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 -- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c500b35f0 msgr2=0x7f2c5009fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c500b35f0 0x7f2c5009fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 -- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c50156a50 msgr2=0x7f2c500b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c50156a50 0x7f2c500b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 -- 172.21.15.37:0/1683517199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c500cbc70 con 0x7f2c50074b60 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c50156a50 0x7f2c500b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.393+0000 7f2c4f400640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50074b60 0x7f2c501f6240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e790 tx=0x7f2c4000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.394+0000 7f2c47400640 1 -- 172.21.15.37:0/1683517199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000eea0 con 0x7f2c50074b60 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.394+0000 7f2c47400640 1 -- 172.21.15.37:0/1683517199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c4000cdb0 con 0x7f2c50074b60 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.394+0000 7f2c47400640 1 -- 172.21.15.37:0/1683517199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40010690 con 0x7f2c50074b60 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.394+0000 7f2c56000640 1 -- 172.21.15.37:0/1683517199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5009ee90 con 0x7f2c50074b60 2024-09-21T06:33:27.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.394+0000 7f2c56000640 1 -- 172.21.15.37:0/1683517199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5009f410 con 0x7f2c50074b60 2024-09-21T06:33:27.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.396+0000 7f2c47400640 1 -- 172.21.15.37:0/1683517199 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2c40010830 con 0x7f2c50074b60 2024-09-21T06:33:27.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.397+0000 7f2c47400640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c30078bc0 0x7f2c3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.397+0000 7f2c47400640 1 -- 172.21.15.37:0/1683517199 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2c4009b970 con 0x7f2c50074b60 2024-09-21T06:33:27.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.398+0000 7f2c5d083740 1 Processor -- start 2024-09-21T06:33:27.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.398+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c30078bc0 0x7f2c3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.398+0000 7f2c5d083740 1 -- start start 2024-09-21T06:33:27.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.398+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/1683517199 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c30078bc0 0x7f2c3007b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c501f2c70 tx=0x7f2c3c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566479a99f0 con 0x55664766e3e0 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556647998e20 con 0x55664768f0c0 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.399+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55664799cc30 con 0x556647666200 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53806/0 (socket says 172.21.15.37:53806) 2024-09-21T06:33:27.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 -- 172.21.15.37:0/21571542 learned_addr learned my addr 172.21.15.37:0/21571542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4f400640 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 -- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 msgr2=0x556647a61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a61020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 -- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 msgr2=0x556647a7e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a7e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 -- 172.21.15.37:0/21571542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664768b120 con 0x55664766e3e0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.400+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c4802b0b0 tx=0x7f2c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc126d056b20b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c3b400640 1 -- 172.21.15.37:0/21571542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48045070 con 0x55664766e3e0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c3b400640 1 -- 172.21.15.37:0/21571542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c4802cab0 con 0x55664766e3e0 2024-09-21T06:33:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c3b400640 1 -- 172.21.15.37:0/21571542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48003970 con 0x55664766e3e0 2024-09-21T06:33:27.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c5d083740 1 -- 172.21.15.37:0/21571542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 msgr2=0x556647a61560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c5d083740 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c4802b0b0 tx=0x7f2c48004a40 comp rx=0 tx=0).stop 2024-09-21T06:33:27.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c5d083740 1 -- 172.21.15.37:0/21571542 shutdown_connections 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.401+0000 7f2c5d083740 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a61560 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.402+0000 7f2c5d083740 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.402+0000 7f2c5d083740 1 --2- 172.21.15.37:0/21571542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.402+0000 7f2c5d083740 1 -- 172.21.15.37:0/21571542 >> 172.21.15.37:0/21571542 conn(0x556647683210 msgr2=0x556647a37240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.402+0000 7f2c5d083740 1 -- 172.21.15.37:0/21571542 shutdown_connections 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.402+0000 7f2c5d083740 1 -- 172.21.15.37:0/21571542 wait complete. 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.403+0000 7f2c5d083740 1 -- start start 2024-09-21T06:33:27.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.403+0000 7f2c5d083740 1 Processor -- start 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556647709470 con 0x556647666200 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55664793e7d0 con 0x55664768f0c0 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556647a76ec0 con 0x55664766e3e0 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.404+0000 7f2c5d083740 10 client.0 _ll_register_callbacks cb 0x5566479447e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53812/0 (socket says 172.21.15.37:53812) 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 -- 172.21.15.37:0/2978775581 learned_addr learned my addr 172.21.15.37:0/2978775581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 -- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 msgr2=0x556647a8b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 -- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 msgr2=0x556647a763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 -- 172.21.15.37:0/2978775581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556647771500 con 0x556647666200 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c4f400640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2c4005f490 tx=0x7f2c40063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3df2bbc7209a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.405+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40067f60 con 0x556647666200 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2c4007ae70 con 0x556647666200 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4802cab0 con 0x556647666200 2024-09-21T06:33:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f2c40082050 con 0x556647666200 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 10 client.7176 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x5566477d1c90 con 0x556647666200 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c4802cab0 con 0x556647666200 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f2c40085020 con 0x556647666200 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 10 client.7176 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 10 client.7176 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 20 client.7176 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55664764dea0 0x556647a7b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 4 client.7176 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.406+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566479ab940 con 0x55664764dea0 2024-09-21T06:33:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.407+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55664764dea0 0x556647a7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.407+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55664764dea0 0x556647a7b030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e7a3e679d460573 server_cookie=9bf99919cf80fb0d in_seq=0 out_seq=0 2024-09-21T06:33:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.407+0000 7f2c2fe00640 10 client.7176 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c2fe00640 1 -- 172.21.15.37:0/2978775581 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566479ab940 con 0x55664764dea0 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c2fe00640 10 client.7176 handle_command_reply: tid=0 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 client.7176 shutdown 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 msgr2=0x556647a4f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2c4005f490 tx=0x7f2c40063000 comp rx=0 tx=0).stop 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 shutdown_connections 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55664764dea0 0x556647a7b030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2978775581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556647666200 0x556647a4f790 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c2fe00640 0 client.7176 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 >> 172.21.15.37:0/2978775581 conn(0x556647683210 msgr2=0x556647a37310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 shutdown_connections 2024-09-21T06:33:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.408+0000 7f2c5d083740 1 -- 172.21.15.37:0/2978775581 wait complete. 2024-09-21T06:33:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.409+0000 7f2c5d083740 20 client.7176 trim_cache size 0 max 16384 2024-09-21T06:33:27.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.427+0000 7f2c5d083740 1 Processor -- start 2024-09-21T06:33:27.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.427+0000 7f2c5d083740 1 -- start start 2024-09-21T06:33:27.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50201880 con 0x55664766e3e0 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50201f80 con 0x55664768f0c0 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500b57f0 con 0x556647666200 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c4ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c4ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53816/0 (socket says 172.21.15.37:53816) 2024-09-21T06:33:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.428+0000 7f2c4ea00640 1 -- 172.21.15.37:0/2350571963 learned_addr learned my addr 172.21.15.37:0/2350571963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 -- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 msgr2=0x556647a7ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 -- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 msgr2=0x556647a38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 -- 172.21.15.37:0/2350571963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566476d83d0 con 0x55664766e3e0 2024-09-21T06:33:27.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005290 tx=0x7f2c3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca4230ca57a740 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c3aa00640 1 -- 172.21.15.37:0/2350571963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00d550 con 0x55664766e3e0 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c3aa00640 1 -- 172.21.15.37:0/2350571963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c3c00d710 con 0x55664766e3e0 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.429+0000 7f2c3aa00640 1 -- 172.21.15.37:0/2350571963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00db60 con 0x55664766e3e0 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 -- 172.21.15.37:0/2350571963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 msgr2=0x556647a37f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005290 tx=0x7f2c3c011000 comp rx=0 tx=0).stop 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 -- 172.21.15.37:0/2350571963 shutdown_connections 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664766e3e0 0x556647a37f20 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55664768f0c0 0x556647a38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 --2- 172.21.15.37:0/2350571963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556647666200 0x556647a7ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 -- 172.21.15.37:0/2350571963 >> 172.21.15.37:0/2350571963 conn(0x556647683210 msgr2=0x556647a8e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 -- 172.21.15.37:0/2350571963 shutdown_connections 2024-09-21T06:33:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.430+0000 7f2c5d083740 1 -- 172.21.15.37:0/2350571963 wait complete. 2024-09-21T06:33:27.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.431+0000 7f2c5d083740 1 -- start start 2024-09-21T06:33:27.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.431+0000 7f2c5d083740 1 Processor -- start 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 0x556647afb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556647a4ff70 con 0x55664768f0c0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556647a78d50 con 0x556647666200 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556647a79bc0 con 0x55664766e3e0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c5d083740 10 client.0 _ll_register_callbacks cb 0x7f2c40062e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53830/0 (socket says 172.21.15.37:53830) 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 -- 172.21.15.37:0/334984834 learned_addr learned my addr 172.21.15.37:0/334984834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4ea00640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4f400640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 0x556647afb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 -- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 msgr2=0x556647a8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 -- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 msgr2=0x556647afb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 0x556647afb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4fe00640 1 -- 172.21.15.37:0/334984834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556647a75c10 con 0x55664768f0c0 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.433+0000 7f2c4f400640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 0x556647afb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c4fe00640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2c48043040 tx=0x7f2c48009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b9910caafb966 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48045040 con 0x55664768f0c0 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2c4802ccb0 con 0x55664768f0c0 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c3c00d710 con 0x55664768f0c0 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f2c48059050 con 0x55664768f0c0 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c5d083740 10 client.7182 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.434+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x556647a918e0 con 0x55664768f0c0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c3c00d710 con 0x55664768f0c0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f2c4805c020 con 0x55664768f0c0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 10 client.7182 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 10 client.7182 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 20 client.7182 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556647affad0 0x556647a9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 4 client.7182 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x5566479a73f0 con 0x556647affad0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.435+0000 7f2c4f400640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556647affad0 0x556647a9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c2fe00640 10 client.7182 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c4f400640 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556647affad0 0x556647a9d980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd0d700a263ae3b6 server_cookie=6364204df77aeebe in_seq=0 out_seq=0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c2fe00640 1 -- 172.21.15.37:0/334984834 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x5566479a73f0 con 0x556647affad0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c2fe00640 10 client.7182 handle_command_reply: tid=0 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 client.7182 shutdown 2024-09-21T06:33:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 msgr2=0x556647a8e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2c48043040 tx=0x7f2c48009800 comp rx=0 tx=0).stop 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 shutdown_connections 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x556647affad0 0x556647a9d980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55664768f0c0 0x556647a8e970 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55664766e3e0 0x556647a8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c2fe00640 0 client.7182 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 --2- 172.21.15.37:0/334984834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556647666200 0x556647afb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 >> 172.21.15.37:0/334984834 conn(0x556647683210 msgr2=0x556647a37f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 shutdown_connections 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 1 -- 172.21.15.37:0/334984834 wait complete. 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.436+0000 7f2c5d083740 20 client.7182 trim_cache size 0 max 16384 2024-09-21T06:33:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:Error EXDEV: Not rank 0 2024-09-21T06:33:27.454 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-21T06:33:27.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub pause 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 -- 172.21.15.37:0/3905834973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 msgr2=0x7f54f00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 --2- 172.21.15.37:0/3905834973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 0x7f54f00b4f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009860 tx=0x7f54e002a5a0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 -- 172.21.15.37:0/3905834973 shutdown_connections 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 --2- 172.21.15.37:0/3905834973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 0x7f54f00b4f90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 -- 172.21.15.37:0/3905834973 >> 172.21.15.37:0/3905834973 conn(0x7f54f006f920 msgr2=0x7f54f00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 -- 172.21.15.37:0/3905834973 shutdown_connections 2024-09-21T06:33:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.680+0000 7f54f4c00640 1 -- 172.21.15.37:0/3905834973 wait complete. 2024-09-21T06:33:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.682+0000 7f54f4c00640 1 Processor -- start 2024-09-21T06:33:27.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.683+0000 7f54f4c00640 1 -- start start 2024-09-21T06:33:27.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 0x7f54f0151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f0156c60 0x7f54f014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54f0152180 0x7f54f014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54f006d950 con 0x7f54f0152180 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54f006d7d0 con 0x7f54f0156c60 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54f4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f54f006dad0 con 0x7f54f006e370 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54f0152180 0x7f54f014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54f0152180 0x7f54f014bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53844/0 (socket says 172.21.15.37:53844) 2024-09-21T06:33:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.684+0000 7f54efe00640 1 -- 172.21.15.37:0/363893970 learned_addr learned my addr 172.21.15.37:0/363893970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54ef400640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 0x7f54f0151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54e7e00640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f0156c60 0x7f54f014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 -- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 msgr2=0x7f54f0151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54f006e370 0x7f54f0151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 -- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f0156c60 msgr2=0x7f54f014b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f0156c60 0x7f54f014b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 -- 172.21.15.37:0/363893970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f0150a80 con 0x7f54f0152180 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54efe00640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54f0152180 0x7f54f014bb20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c780 tx=0x7f54e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54e7400640 1 -- 172.21.15.37:0/363893970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003b70 con 0x7f54f0152180 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54e7400640 1 -- 172.21.15.37:0/363893970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e8003d10 con 0x7f54f0152180 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.685+0000 7f54e7400640 1 -- 172.21.15.37:0/363893970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e800aab0 con 0x7f54f0152180 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.686+0000 7f54f4c00640 1 -- 172.21.15.37:0/363893970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f0150d60 con 0x7f54f0152180 2024-09-21T06:33:27.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.686+0000 7f54f4c00640 1 -- 172.21.15.37:0/363893970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f014d600 con 0x7f54f0152180 2024-09-21T06:33:27.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.688+0000 7f54e7400640 1 -- 172.21.15.37:0/363893970 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54e800ac50 con 0x7f54f0152180 2024-09-21T06:33:27.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.689+0000 7f54e7400640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f54cc078bc0 0x7f54cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.689+0000 7f54e7400640 1 -- 172.21.15.37:0/363893970 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f54e809ac90 con 0x7f54f0152180 2024-09-21T06:33:27.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.689+0000 7f54ef400640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f54cc078bc0 0x7f54cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.690+0000 7f54fd2b6740 1 Processor -- start 2024-09-21T06:33:27.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.690+0000 7f54fd2b6740 1 -- start start 2024-09-21T06:33:27.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.690+0000 7f54ef400640 1 --2- 172.21.15.37:0/363893970 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f54cc078bc0 0x7f54cc07b0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f54e0034000 tx=0x7f54e0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118ae200 0x563411cc50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 0x563411ca6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563411bf19f0 con 0x5634118d70c0 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563411be0e20 con 0x5634118b63e0 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563411be4c30 con 0x5634118ae200 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53856/0 (socket says 172.21.15.37:53856) 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54e7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118ae200 0x563411cc50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54ef400640 1 -- 172.21.15.37:0/244880346 learned_addr learned my addr 172.21.15.37:0/244880346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.691+0000 7f54efe00640 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 0x563411ca6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 -- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118ae200 msgr2=0x563411cc50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118ae200 0x563411cc50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 -- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 msgr2=0x563411ca6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 0x563411ca6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 -- 172.21.15.37:0/244880346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634118d3120 con 0x5634118d70c0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54efe00640 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 0x563411ca6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.692+0000 7f54ef400640 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009950 tx=0x7f54e003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17b2867b34a5dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.693+0000 7f54df400640 1 -- 172.21.15.37:0/244880346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0049420 con 0x5634118d70c0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.693+0000 7f54df400640 1 -- 172.21.15.37:0/244880346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e00496b0 con 0x5634118d70c0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.693+0000 7f54df400640 1 -- 172.21.15.37:0/244880346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0049a60 con 0x5634118d70c0 2024-09-21T06:33:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.693+0000 7f54fd2b6740 1 -- 172.21.15.37:0/244880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 msgr2=0x563411c99510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.693+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009950 tx=0x7f54e003e070 comp rx=0 tx=0).stop 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 -- 172.21.15.37:0/244880346 shutdown_connections 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118b63e0 0x563411ca6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118ae200 0x563411cc50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/244880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411c99510 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 -- 172.21.15.37:0/244880346 >> 172.21.15.37:0/244880346 conn(0x5634118cb210 msgr2=0x563411c7f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 -- 172.21.15.37:0/244880346 shutdown_connections 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.694+0000 7f54fd2b6740 1 -- 172.21.15.37:0/244880346 wait complete. 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.695+0000 7f54fd2b6740 1 -- start start 2024-09-21T06:33:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.695+0000 7f54fd2b6740 1 Processor -- start 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411c7ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c7d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563411951470 con 0x5634118ae200 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563411b867d0 con 0x5634118d70c0 2024-09-21T06:33:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563411c987b0 con 0x5634118b63e0 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.696+0000 7f54fd2b6740 10 client.0 _ll_register_callbacks cb 0x563411b8c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53866/0 (socket says 172.21.15.37:53866) 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 -- 172.21.15.37:0/3064250165 learned_addr learned my addr 172.21.15.37:0/3064250165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54e7e00640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411c7ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54efe00640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c7d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 -- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 msgr2=0x563411c7ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411c7ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 -- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 msgr2=0x563411c7d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c7d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 -- 172.21.15.37:0/3064250165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634119b9500 con 0x5634118ae200 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54ef400640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005eb0 tx=0x7f54e0049550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed312839b7ced28e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.697+0000 7f54d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e005f060 con 0x5634118ae200 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f54e0049c70 con 0x5634118ae200 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e00496b0 con 0x5634118ae200 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f54e0065050 con 0x5634118ae200 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54fd2b6740 10 client.7194 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x563411a19c90 con 0x5634118ae200 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e00496b0 con 0x5634118ae200 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.698+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f54e0068020 con 0x5634118ae200 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 10 client.7194 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 10 client.7194 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 20 client.7194 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411895ea0 0x563411cdb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 4 client.7194 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563411bf3940 con 0x563411895ea0 2024-09-21T06:33:27.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54e7e00640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411895ea0 0x563411cdb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54e7e00640 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411895ea0 0x563411cdb920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eef81e958b631ef9 server_cookie=5a3ab1d39df8466f in_seq=0 out_seq=0 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.699+0000 7f54d3e00640 10 client.7194 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.700+0000 7f54d3e00640 1 -- 172.21.15.37:0/3064250165 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563411bf3940 con 0x563411895ea0 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.700+0000 7f54d3e00640 10 client.7194 handle_command_reply: tid=0 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.700+0000 7f54fd2b6740 1 client.7194 shutdown 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.700+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 msgr2=0x563411c97bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.700+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005eb0 tx=0x7f54e0049550 comp rx=0 tx=0).stop 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 shutdown_connections 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411895ea0 0x563411cdb920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c7d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411c7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/3064250165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411c97bd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54d3e00640 0 client.7194 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 >> 172.21.15.37:0/3064250165 conn(0x5634118cb210 msgr2=0x563411cc6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 shutdown_connections 2024-09-21T06:33:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 1 -- 172.21.15.37:0/3064250165 wait complete. 2024-09-21T06:33:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.701+0000 7f54fd2b6740 20 client.7194 trim_cache size 0 max 16384 2024-09-21T06:33:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.719+0000 7f54fd2b6740 1 Processor -- start 2024-09-21T06:33:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.719+0000 7f54fd2b6740 1 -- start start 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118ae200 0x563411c7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411cc6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0074a40 con 0x5634118d70c0 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01fd9d0 con 0x5634118ae200 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f54f014ed00 con 0x5634118b63e0 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53876/0 (socket says 172.21.15.37:53876) 2024-09-21T06:33:27.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 -- 172.21.15.37:0/4078823476 learned_addr learned my addr 172.21.15.37:0/4078823476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 -- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 msgr2=0x563411cc6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411cc6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 -- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118ae200 msgr2=0x563411c7b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118ae200 0x563411c7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.721+0000 7f54efe00640 1 -- 172.21.15.37:0/4078823476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634119203d0 con 0x5634118d70c0 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54efe00640 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54e800a430 tx=0x7f54e8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a4686faef9914e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54dea00640 1 -- 172.21.15.37:0/4078823476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8088020 con 0x5634118d70c0 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54dea00640 1 -- 172.21.15.37:0/4078823476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e8062f40 con 0x5634118d70c0 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54dea00640 1 -- 172.21.15.37:0/4078823476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8063dd0 con 0x5634118d70c0 2024-09-21T06:33:27.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54fd2b6740 1 -- 172.21.15.37:0/4078823476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 msgr2=0x563411cc73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.722+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54e800a430 tx=0x7f54e8001250 comp rx=0 tx=0).stop 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 -- 172.21.15.37:0/4078823476 shutdown_connections 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118d70c0 0x563411cc73d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411cc6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/4078823476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118ae200 0x563411c7b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 -- 172.21.15.37:0/4078823476 >> 172.21.15.37:0/4078823476 conn(0x5634118cb210 msgr2=0x563411c7d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 -- 172.21.15.37:0/4078823476 shutdown_connections 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.723+0000 7f54fd2b6740 1 -- 172.21.15.37:0/4078823476 wait complete. 2024-09-21T06:33:27.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.724+0000 7f54fd2b6740 1 -- start start 2024-09-21T06:33:27.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.724+0000 7f54fd2b6740 1 Processor -- start 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411d41d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c9f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563411cbf460 con 0x5634118ae200 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563411cbd0d0 con 0x5634118d70c0 2024-09-21T06:33:27.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563411cbdc70 con 0x5634118b63e0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.725+0000 7f54fd2b6740 10 client.0 _ll_register_callbacks cb 0x7f54e80017b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53882/0 (socket says 172.21.15.37:53882) 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 -- 172.21.15.37:0/913895062 learned_addr learned my addr 172.21.15.37:0/913895062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54e7e00640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411d41d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54efe00640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c9f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 -- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 msgr2=0x563411d41d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411d41d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 -- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 msgr2=0x563411c9f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c9f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 -- 172.21.15.37:0/913895062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563411d47d40 con 0x5634118ae200 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54ef400640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f54e002a5e0 tx=0x7f54e00096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34168bbd1806d45c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.726+0000 7f54d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0049dc0 con 0x5634118ae200 2024-09-21T06:33:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f54e0051400 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e8062f40 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f54e0077050 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 10 client.7200 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x563411cc4ec0 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e8062f40 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7f54e007a020 con 0x5634118ae200 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 10 client.7200 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 10 client.7200 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 20 client.7200 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411c7e570 0x563411d4b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 4 client.7200 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.727+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x563411bef3f0 con 0x563411c7e570 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54e7e00640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411c7e570 0x563411d4b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54e7e00640 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411c7e570 0x563411d4b330 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28231fe8937bdf2d server_cookie=4f640586a1bec469 in_seq=0 out_seq=0 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54d3e00640 10 client.7200 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54d3e00640 1 -- 172.21.15.37:0/913895062 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x563411bef3f0 con 0x563411c7e570 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54d3e00640 10 client.7200 handle_command_reply: tid=0 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.728+0000 7f54fd2b6740 1 client.7200 shutdown 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 msgr2=0x563411cec800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f54e002a5e0 tx=0x7f54e00096f0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 shutdown_connections 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x563411c7e570 0x563411d4b330 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634118d70c0 0x563411c9f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5634118b63e0 0x563411d41d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54d3e00640 0 client.7200 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 --2- 172.21.15.37:0/913895062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634118ae200 0x563411cec800 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 >> 172.21.15.37:0/913895062 conn(0x5634118cb210 msgr2=0x563411c98f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 shutdown_connections 2024-09-21T06:33:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 1 -- 172.21.15.37:0/913895062 wait complete. 2024-09-21T06:33:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.729+0000 7f54fd2b6740 20 client.7200 trim_cache size 0 max 16384 2024-09-21T06:33:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:Error EXDEV: Not rank 0 2024-09-21T06:33:27.744 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-21T06:33:27.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub resume 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 -- 172.21.15.37:0/175919472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc406cde0 msgr2=0x7effc4155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 --2- 172.21.15.37:0/175919472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc406cde0 0x7effc4155560 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7effb4009860 tx=0x7effb402a540 comp rx=0 tx=0).stop 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 -- 172.21.15.37:0/175919472 shutdown_connections 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 --2- 172.21.15.37:0/175919472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effc40ccfe0 0x7effc40af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 --2- 172.21.15.37:0/175919472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc406cde0 0x7effc4155560 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 -- 172.21.15.37:0/175919472 >> 172.21.15.37:0/175919472 conn(0x7effc406f920 msgr2=0x7effc40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 -- 172.21.15.37:0/175919472 shutdown_connections 2024-09-21T06:33:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.972+0000 7effc3e00640 1 -- 172.21.15.37:0/175919472 wait complete. 2024-09-21T06:33:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.975+0000 7effc3e00640 1 Processor -- start 2024-09-21T06:33:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.975+0000 7effc3e00640 1 -- start start 2024-09-21T06:33:27.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effc406cde0 0x7effc4153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effc40aef90 0x7effc415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc40ce7d0 0x7effc40cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effc415f490 con 0x7effc40ce7d0 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7effc415f310 con 0x7effc406cde0 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7effc415f610 con 0x7effc40aef90 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc40ce7d0 0x7effc40cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.976+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc40ce7d0 0x7effc40cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53896/0 (socket says 172.21.15.37:53896) 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 -- 172.21.15.37:0/3741916426 learned_addr learned my addr 172.21.15.37:0/3741916426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc2000640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effc40aef90 0x7effc415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc2a00640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effc406cde0 0x7effc4153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 -- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effc40aef90 msgr2=0x7effc415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effc40aef90 0x7effc415ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 -- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effc406cde0 msgr2=0x7effc4153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effc406cde0 0x7effc4153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 -- 172.21.15.37:0/3741916426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc40cf120 con 0x7effc40ce7d0 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.977+0000 7effc3400640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc40ce7d0 0x7effc40cebe0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7effbc00eab0 tx=0x7effbc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.978+0000 7effbb400640 1 -- 172.21.15.37:0/3741916426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00cea0 con 0x7effc40ce7d0 2024-09-21T06:33:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.978+0000 7effc3e00640 1 -- 172.21.15.37:0/3741916426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc415b5b0 con 0x7effc40ce7d0 2024-09-21T06:33:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.978+0000 7effbb400640 1 -- 172.21.15.37:0/3741916426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effbc004b20 con 0x7effc40ce7d0 2024-09-21T06:33:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.978+0000 7effbb400640 1 -- 172.21.15.37:0/3741916426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc005020 con 0x7effc40ce7d0 2024-09-21T06:33:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.978+0000 7effc3e00640 1 -- 172.21.15.37:0/3741916426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc415ff20 con 0x7effc40ce7d0 2024-09-21T06:33:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.980+0000 7effbb400640 1 -- 172.21.15.37:0/3741916426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effbc0051c0 con 0x7effc40ce7d0 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.981+0000 7effbb400640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7effa4078b90 0x7effa407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.981+0000 7effbb400640 1 -- 172.21.15.37:0/3741916426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7effbc09bb30 con 0x7effc40ce7d0 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.982+0000 7effd1001740 1 Processor -- start 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.982+0000 7effc2a00640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7effa4078b90 0x7effa407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.982+0000 7effd1001740 1 -- start start 2024-09-21T06:33:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.982+0000 7effc2a00640 1 --2- 172.21.15.37:0/3741916426 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7effa4078b90 0x7effa407b0a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7effb4034000 tx=0x7effb4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 0x55cd7246b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7244e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cd72337c10 con 0x55cd7205b6b0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cd723969f0 con 0x55cd720534d0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cd72385e20 con 0x55cd7207c390 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.983+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 0x55cd7246b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7244e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 0x55cd7246b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:43190/0 (socket says 172.21.15.37:43190) 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53902/0 (socket says 172.21.15.37:53902) 2024-09-21T06:33:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc2a00640 1 -- 172.21.15.37:0/2742826457 learned_addr learned my addr 172.21.15.37:0/2742826457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 -- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 msgr2=0x55cd7246b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 0x55cd7246b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 -- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 msgr2=0x55cd7244e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7244e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 -- 172.21.15.37:0/2742826457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7215e7d0 con 0x55cd7205b6b0 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effc3400640 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7effbc099e00 tx=0x7effbc062f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779267db7b791ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.984+0000 7effaf400640 1 -- 172.21.15.37:0/2742826457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc06cfe0 con 0x55cd7205b6b0 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effaf400640 1 -- 172.21.15.37:0/2742826457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effbc06cb50 con 0x55cd7205b6b0 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effaf400640 1 -- 172.21.15.37:0/2742826457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc06c040 con 0x55cd7205b6b0 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 -- 172.21.15.37:0/2742826457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 msgr2=0x55cd7244ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7effbc099e00 tx=0x7effbc062f00 comp rx=0 tx=0).stop 2024-09-21T06:33:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 -- 172.21.15.37:0/2742826457 shutdown_connections 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7205b6b0 0x55cd7244ea70 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7244e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 --2- 172.21.15.37:0/2742826457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7207c390 0x55cd7246b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 -- 172.21.15.37:0/2742826457 >> 172.21.15.37:0/2742826457 conn(0x55cd720704e0 msgr2=0x55cd72424750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.985+0000 7effd1001740 1 -- 172.21.15.37:0/2742826457 shutdown_connections 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.986+0000 7effd1001740 1 -- 172.21.15.37:0/2742826457 wait complete. 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.987+0000 7effd1001740 1 -- start start 2024-09-21T06:33:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.987+0000 7effd1001740 1 Processor -- start 2024-09-21T06:33:27.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7243c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 0x55cd72469890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cd7232b790 con 0x55cd7207c390 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cd72389c30 con 0x55cd720534d0 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cd720f6740 con 0x55cd7205b6b0 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.988+0000 7effd1001740 10 client.0 _ll_register_callbacks cb 0x55cd7233b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53908/0 (socket says 172.21.15.37:53908) 2024-09-21T06:33:27.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 -- 172.21.15.37:0/2429584569 learned_addr learned my addr 172.21.15.37:0/2429584569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc2000640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 0x55cd72469890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc2a00640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7243c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 -- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 msgr2=0x55cd72469890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 0x55cd72469890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 -- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 msgr2=0x55cd7243c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7243c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 -- 172.21.15.37:0/2429584569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd720c56a0 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc2000640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 0x55cd72469890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc2a00640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7243c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effc3400640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7effbc004af0 tx=0x7effbc05fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ac2f4c73fc38bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.989+0000 7effa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc082070 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7effbc062c00 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effbc06cb50 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7effbc075050 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 10 client.7215 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55cd721bef60 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effbc06cb50 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7effbc078020 con 0x55cd7207c390 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 10 client.7215 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 10 client.7215 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 20 client.7215 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd7203af60 0x55cd72467870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 4 client.7215 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.990+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd72398940 con 0x55cd7203af60 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.991+0000 7effc2a00640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd7203af60 0x55cd72467870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.991+0000 7effc2a00640 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd7203af60 0x55cd72467870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c0a0565dba9f55 server_cookie=6010d70d62eb7b8a in_seq=0 out_seq=0 2024-09-21T06:33:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.991+0000 7effa3e00640 10 client.7215 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effa3e00640 1 -- 172.21.15.37:0/2429584569 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd72398940 con 0x55cd7203af60 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effa3e00640 10 client.7215 handle_command_reply: tid=0 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 client.7215 shutdown 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 msgr2=0x55cd72423ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7effbc004af0 tx=0x7effbc05fed0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 shutdown_connections 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd7203af60 0x55cd72467870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd72423ad0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd7205b6b0 0x55cd72469890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 --2- 172.21.15.37:0/2429584569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd720534d0 0x55cd7243c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effa3e00640 0 client.7215 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.992+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 >> 172.21.15.37:0/2429584569 conn(0x55cd720704e0 msgr2=0x55cd7244b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.993+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 shutdown_connections 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.993+0000 7effd1001740 1 -- 172.21.15.37:0/2429584569 wait complete. 2024-09-21T06:33:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:27.993+0000 7effd1001740 20 client.7215 trim_cache size 0 max 16384 2024-09-21T06:33:28.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.011+0000 7effd1001740 1 Processor -- start 2024-09-21T06:33:28.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.011+0000 7effd1001740 1 -- start start 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd724254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd7241f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effc4201550 con 0x55cd7207c390 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7effc4201c50 con 0x55cd7205b6b0 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7effc4200620 con 0x55cd720534d0 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53922/0 (socket says 172.21.15.37:53922) 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 -- 172.21.15.37:0/3849385861 learned_addr learned my addr 172.21.15.37:0/3849385861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 -- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 msgr2=0x55cd724254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd724254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc2000640 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd7241f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 -- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 msgr2=0x55cd7241f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd7241f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.013+0000 7effc3400640 1 -- 172.21.15.37:0/3849385861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7215cad0 con 0x55cd7207c390 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.014+0000 7effc2000640 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd7241f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.014+0000 7effc3400640 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7effbc00ea80 tx=0x7effbc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6407d70b6d507907 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.014+0000 7effaea00640 1 -- 172.21.15.37:0/3849385861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc082040 con 0x55cd7207c390 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.014+0000 7effaea00640 1 -- 172.21.15.37:0/3849385861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effbc06cb50 con 0x55cd7207c390 2024-09-21T06:33:28.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.014+0000 7effaea00640 1 -- 172.21.15.37:0/3849385861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc05f630 con 0x55cd7207c390 2024-09-21T06:33:28.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 -- 172.21.15.37:0/3849385861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 msgr2=0x55cd7243dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7effbc00ea80 tx=0x7effbc009200 comp rx=0 tx=0).stop 2024-09-21T06:33:28.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 -- 172.21.15.37:0/3849385861 shutdown_connections 2024-09-21T06:33:28.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7243dc40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd7241f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 --2- 172.21.15.37:0/3849385861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd724254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 -- 172.21.15.37:0/3849385861 >> 172.21.15.37:0/3849385861 conn(0x55cd720704e0 msgr2=0x55cd7241a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 -- 172.21.15.37:0/3849385861 shutdown_connections 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.015+0000 7effd1001740 1 -- 172.21.15.37:0/3849385861 wait complete. 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.016+0000 7effd1001740 1 -- start start 2024-09-21T06:33:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.016+0000 7effd1001740 1 Processor -- start 2024-09-21T06:33:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd72468770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd724e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cd72467ec0 con 0x55cd7207c390 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cd7232b790 con 0x55cd7205b6b0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cd72479480 con 0x55cd720534d0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effd1001740 10 client.0 _ll_register_callbacks cb 0x7effb0001220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd72468770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd724e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc2a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd72468770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:43220/0 (socket says 172.21.15.37:43220) 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd724e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:37966/0 (socket says 172.21.15.37:37966) 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc2a00640 1 -- 172.21.15.37:0/1474562189 learned_addr learned my addr 172.21.15.37:0/1474562189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.018+0000 7effc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53930/0 (socket says 172.21.15.37:53930) 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 -- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 msgr2=0x55cd72468770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd72468770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 -- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 msgr2=0x55cd724e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd724e74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 -- 172.21.15.37:0/1474562189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7243dce0 con 0x55cd7207c390 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effc3400640 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7effbc07e000 tx=0x7effbc0099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7993e95bd7fd21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc082040 con 0x55cd7207c390 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7effbc06a550 con 0x55cd7207c390 2024-09-21T06:33:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effbc06cb50 con 0x55cd7207c390 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7effbc05f630 con 0x55cd7207c390 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effd1001740 10 client.7221 fetch_fsmap learned FSMap version 79 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.019+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x55cd724e7f50 con 0x55cd7207c390 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effbc06cb50 con 0x55cd7207c390 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 79) ==== 3554+0+0 (secure 0 0 0) 0x7effbc078070 con 0x55cd7207c390 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 10 client.7221 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 10 client.7221 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 20 client.7221 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd72422830 0x55cd7247b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 4 client.7221 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 --> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55cd723943f0 con 0x55cd72422830 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.020+0000 7effc2a00640 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd72422830 0x55cd7247b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effc2a00640 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd72422830 0x55cd7247b660 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27a8356189f0fea9 server_cookie=136fb13bc5fb44ef in_seq=0 out_seq=0 2024-09-21T06:33:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effa3e00640 10 client.7221 ms_handle_connect on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effa3e00640 1 -- 172.21.15.37:0/1474562189 <== mds.1 v2:172.21.15.37:6834/2799311131 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x55cd723943f0 con 0x55cd72422830 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effa3e00640 10 client.7221 handle_command_reply: tid=0 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 client.7221 shutdown 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 msgr2=0x55cd7241d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7effbc07e000 tx=0x7effbc0099d0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 shutdown_connections 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:6834/2799311131,v1:172.21.15.37:6835/2799311131] conn(0x55cd72422830 0x55cd7247b660 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd7207c390 0x55cd7241d810 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd7205b6b0 0x55cd724e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 --2- 172.21.15.37:0/1474562189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cd720534d0 0x55cd72468770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 >> 172.21.15.37:0/1474562189 conn(0x55cd720704e0 msgr2=0x55cd72424890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effa3e00640 0 client.7221 ms_handle_reset on v2:172.21.15.37:6834/2799311131 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 shutdown_connections 2024-09-21T06:33:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 1 -- 172.21.15.37:0/1474562189 wait complete. 2024-09-21T06:33:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.021+0000 7effd1001740 20 client.7221 trim_cache size 0 max 16384 2024-09-21T06:33:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:Error EXDEV: Not rank 0 2024-09-21T06:33:28.040 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-21T06:33:28.041 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:33:28.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:33:28.127 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:33:28.214 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:33:28.214 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:33:28.214 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:28.214 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:28.249 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:28.249 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:28.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 -- 172.21.15.37:0/659250488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8156a50 msgr2=0x7fe3c8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 --2- 172.21.15.37:0/659250488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8156a50 0x7fe3c8074620 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009830 tx=0x7fe3b802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 -- 172.21.15.37:0/659250488 shutdown_connections 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 --2- 172.21.15.37:0/659250488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8156a50 0x7fe3c8074620 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 -- 172.21.15.37:0/659250488 >> 172.21.15.37:0/659250488 conn(0x7fe3c80b1b90 msgr2=0x7fe3c80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 -- 172.21.15.37:0/659250488 shutdown_connections 2024-09-21T06:33:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.591+0000 7fe3cd800640 1 -- 172.21.15.37:0/659250488 wait complete. 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.594+0000 7fe3cd800640 1 Processor -- start 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.594+0000 7fe3cd800640 1 -- start start 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8156a50 0x7fe3c815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3c815c560 0x7fe3c815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf790 con 0x7fe3c8074b60 2024-09-21T06:33:28.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf610 con 0x7fe3c8156a50 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3cd800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf910 con 0x7fe3c815c560 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3c6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3c6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53948/0 (socket says 172.21.15.37:53948) 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.595+0000 7fe3c6a00640 1 -- 172.21.15.37:0/1892496264 learned_addr learned my addr 172.21.15.37:0/1892496264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c7400640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3c815c560 0x7fe3c815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3bfe00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8156a50 0x7fe3c815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 -- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3c815c560 msgr2=0x7fe3c815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3c815c560 0x7fe3c815ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 -- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8156a50 msgr2=0x7fe3c815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8156a50 0x7fe3c815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 -- 172.21.15.37:0/1892496264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c81f73f0 con 0x7fe3c8074b60 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.596+0000 7fe3c6a00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.597+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8031d60 con 0x7fe3c8074b60 2024-09-21T06:33:28.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.597+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3b80330d0 con 0x7fe3c8074b60 2024-09-21T06:33:28.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.597+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8033520 con 0x7fe3c8074b60 2024-09-21T06:33:28.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.597+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81f6230 con 0x7fe3c8074b60 2024-09-21T06:33:28.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.597+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81f6b50 con 0x7fe3c8074b60 2024-09-21T06:33:28.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.598+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c8074620 con 0x7fe3c8074b60 2024-09-21T06:33:28.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.599+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3b80035f0 con 0x7fe3c8074b60 2024-09-21T06:33:28.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.601+0000 7fe3bea00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 0x7fe3a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:28.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.601+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe3b80bc2a0 con 0x7fe3c8074b60 2024-09-21T06:33:28.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.601+0000 7fe3bfe00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 0x7fe3a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:28.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.602+0000 7fe3bfe00640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 0x7fe3a407b0d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80709f0 tx=0x7fe3b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:28.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.605+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3b803f030 con 0x7fe3c8074b60 2024-09-21T06:33:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.812+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3c8070d00 con 0x7fe3c8074b60 2024-09-21T06:33:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.813+0000 7fe3bea00640 1 -- 172.21.15.37:0/1892496264 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7fe3b8083ef0 con 0x7fe3c8074b60 2024-09-21T06:33:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:33:28.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 msgr2=0x7fe3a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 0x7fe3a407b0d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80709f0 tx=0x7fe3b0009040 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 msgr2=0x7fe3c81f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8033d60 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 shutdown_connections 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3a4078bc0 0x7fe3a407b0d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3c815c560 0x7fe3c815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8156a50 0x7fe3c815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 --2- 172.21.15.37:0/1892496264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3c8074b60 0x7fe3c81f5cf0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 >> 172.21.15.37:0/1892496264 conn(0x7fe3c80b1b90 msgr2=0x7fe3c80c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 shutdown_connections 2024-09-21T06:33:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:28.815+0000 7fe3cd800640 1 -- 172.21.15.37:0/1892496264 wait complete. 2024-09-21T06:33:28.839 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:33:28.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:28.839 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:28.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:33:28.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:28.904 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:33:28.940 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:33:28.940 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:33:28.941 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:28.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:33:28.981 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:33:28.981 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:28.981 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:29.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:33:29.016 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:33:29.016 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:33:29.016 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:29.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:33:29.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 -- 172.21.15.37:0/551153173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89940cbc10 msgr2=0x7f89940cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 --2- 172.21.15.37:0/551153173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89940cbc10 0x7f89940cc020 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f897c009830 tx=0x7f897c02a510 comp rx=0 tx=0).stop 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 -- 172.21.15.37:0/551153173 shutdown_connections 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 --2- 172.21.15.37:0/551153173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89940cbc10 0x7f89940cc020 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 --2- 172.21.15.37:0/551153173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8994074cb0 0x7f8994073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 -- 172.21.15.37:0/551153173 >> 172.21.15.37:0/551153173 conn(0x7f89940cca40 msgr2=0x7f89940cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 -- 172.21.15.37:0/551153173 shutdown_connections 2024-09-21T06:33:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.283+0000 7f899b200640 1 -- 172.21.15.37:0/551153173 wait complete. 2024-09-21T06:33:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.285+0000 7f899b200640 1 Processor -- start 2024-09-21T06:33:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.285+0000 7f899b200640 1 -- start start 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f899406ddf0 0x7f89940cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89940cbc10 0x7f899409c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f899406f360 con 0x7f8994074cb0 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f899406f1e0 con 0x7f899406ddf0 2024-09-21T06:33:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f899b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f899406f4e0 con 0x7f89940cbc10 2024-09-21T06:33:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53960/0 (socket says 172.21.15.37:53960) 2024-09-21T06:33:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 -- 172.21.15.37:0/2512274479 learned_addr learned my addr 172.21.15.37:0/2512274479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f8999200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89940cbc10 0x7f899409c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f8993e00640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f899406ddf0 0x7f89940cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 -- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89940cbc10 msgr2=0x7f899409c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89940cbc10 0x7f899409c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 -- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f899406ddf0 msgr2=0x7f89940cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f899406ddf0 0x7f89940cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.287+0000 7f898b400640 1 -- 172.21.15.37:0/2512274479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89941f9ca0 con 0x7f8994074cb0 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f898b400640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f897c009950 tx=0x7f897c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c005c90 con 0x7f8994074cb0 2024-09-21T06:33:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f897c031e50 con 0x7f8994074cb0 2024-09-21T06:33:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c005260 con 0x7f8994074cb0 2024-09-21T06:33:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89941f9120 con 0x7f8994074cb0 2024-09-21T06:33:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.288+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89941f96e0 con 0x7f8994074cb0 2024-09-21T06:33:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.289+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899406fa00 con 0x7f8994074cb0 2024-09-21T06:33:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.291+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f897c005400 con 0x7f8994074cb0 2024-09-21T06:33:29.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.292+0000 7f8990c00640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 0x7f897007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.292+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f897c0bbf20 con 0x7f8994074cb0 2024-09-21T06:33:29.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.292+0000 7f8993e00640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 0x7f897007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.293+0000 7f8993e00640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 0x7f897007b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f89840095f0 tx=0x7f8984005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:29.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.296+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f897c03f030 con 0x7f8994074cb0 2024-09-21T06:33:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.535+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f899415a0d0 con 0x7f8994074cb0 2024-09-21T06:33:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.535+0000 7f8990c00640 1 -- 172.21.15.37:0/2512274479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5273 (secure 0 0 0) 0x7f897c083b70 con 0x7f8994074cb0 2024-09-21T06:33:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:29.538 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"btime":"2024-09-21T06:33:24:907118+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:24.856871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6852,6858,6870]},"id":5}]} 2024-09-21T06:33:29.538 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 79 2024-09-21T06:33:29.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 msgr2=0x7f897007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 0x7f897007b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f89840095f0 tx=0x7f8984005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 msgr2=0x7f89941f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f897c009950 tx=0x7f897c033c20 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 shutdown_connections 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8970078bc0 0x7f897007b0d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8994074cb0 0x7f89941f4e90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89940cbc10 0x7f899409c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 --2- 172.21.15.37:0/2512274479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f899406ddf0 0x7f89940cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 >> 172.21.15.37:0/2512274479 conn(0x7f89940cca40 msgr2=0x7f899415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 shutdown_connections 2024-09-21T06:33:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.538+0000 7f899b200640 1 -- 172.21.15.37:0/2512274479 wait complete. 2024-09-21T06:33:29.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 -- 172.21.15.37:0/608052564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950074d90 msgr2=0x7fc95006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 --2- 172.21.15.37:0/608052564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950074d90 0x7fc95006f710 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc950070dc0 tx=0x7fc94002ade0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 -- 172.21.15.37:0/608052564 shutdown_connections 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 --2- 172.21.15.37:0/608052564 >> v2:172.21.15.150:3300/0 conn(0x7fc95006e330 0x7fc9500cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 --2- 172.21.15.37:0/608052564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc950070030 0x7fc95006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 --2- 172.21.15.37:0/608052564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950074d90 0x7fc95006f710 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 -- 172.21.15.37:0/608052564 >> 172.21.15.37:0/608052564 conn(0x7fc9500cca40 msgr2=0x7fc9500cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 -- 172.21.15.37:0/608052564 shutdown_connections 2024-09-21T06:33:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.788+0000 7fc956c00640 1 -- 172.21.15.37:0/608052564 wait complete. 2024-09-21T06:33:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.791+0000 7fc956c00640 1 Processor -- start 2024-09-21T06:33:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.791+0000 7fc956c00640 1 -- start start 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc95006e330 0x7fc9500c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 0x7fc9501f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc95015ad50 con 0x7fc950070030 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc95015abd0 con 0x7fc950074d90 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc956c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc950070e80 con 0x7fc95006e330 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc94f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc94f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53980/0 (socket says 172.21.15.37:53980) 2024-09-21T06:33:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.792+0000 7fc94f400640 1 -- 172.21.15.37:0/46225483 learned_addr learned my addr 172.21.15.37:0/46225483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94fe00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc95006e330 0x7fc9500c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc954c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 0x7fc9501f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 -- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc95006e330 msgr2=0x7fc9500c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc95006e330 0x7fc9500c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 -- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 msgr2=0x7fc9501f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 0x7fc9501f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 -- 172.21.15.37:0/46225483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9501f47e0 con 0x7fc950070030 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc954c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 0x7fc9501f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc94f400640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c970 tx=0x7fc93c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.793+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c007bb0 con 0x7fc950070030 2024-09-21T06:33:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.794+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc93c007e40 con 0x7fc950070030 2024-09-21T06:33:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.794+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9500c8880 con 0x7fc950070030 2024-09-21T06:33:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.794+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00f6b0 con 0x7fc950070030 2024-09-21T06:33:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.794+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9500c8e40 con 0x7fc950070030 2024-09-21T06:33:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.795+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9501f4550 con 0x7fc950070030 2024-09-21T06:33:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.796+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc93c00f850 con 0x7fc950070030 2024-09-21T06:33:29.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.797+0000 7fc947e00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 0x7fc93007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:29.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.797+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc93c09b6e0 con 0x7fc950070030 2024-09-21T06:33:29.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.797+0000 7fc94fe00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 0x7fc93007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:29.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.798+0000 7fc94fe00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 0x7fc93007b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc940034000 tx=0x7fc940004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:29.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:29.801+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc93c014030 con 0x7fc950070030 2024-09-21T06:33:30.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.047+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc95006ddf0 con 0x7fc950070030 2024-09-21T06:33:30.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.047+0000 7fc947e00640 1 -- 172.21.15.37:0/46225483 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5275 (secure 0 0 0) 0x7fc93c0638f0 con 0x7fc950070030 2024-09-21T06:33:30.053 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:30.054 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"btime":"2024-09-21T06:33:29:927104+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:29.856592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870,6858,6852]},"id":5}]} 2024-09-21T06:33:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 80 2024-09-21T06:33:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 msgr2=0x7fc93007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 0x7fc93007b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc940034000 tx=0x7fc940004730 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 msgr2=0x7fc9501f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c970 tx=0x7fc93c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 shutdown_connections 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc930078b90 0x7fc93007b0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc950070030 0x7fc9501f7a50 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc950074d90 0x7fc9501f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 --2- 172.21.15.37:0/46225483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc95006e330 0x7fc9500c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 >> 172.21.15.37:0/46225483 conn(0x7fc9500cca40 msgr2=0x7fc9500cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 shutdown_connections 2024-09-21T06:33:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.050+0000 7fc956c00640 1 -- 172.21.15.37:0/46225483 wait complete. 2024-09-21T06:33:30.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.301+0000 7f2585a00640 1 -- 172.21.15.37:0/1601817659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2564003a50 msgr2=0x7f2564003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.301+0000 7f2585a00640 1 --2- 172.21.15.37:0/1601817659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2564003a50 0x7f2564003ec0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f257000c310 tx=0x7f2564005830 comp rx=0 tx=0).stop 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.302+0000 7f2585a00640 1 -- 172.21.15.37:0/1601817659 shutdown_connections 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.302+0000 7f2585a00640 1 --2- 172.21.15.37:0/1601817659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2564003a50 0x7f2564003ec0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.302+0000 7f2585a00640 1 -- 172.21.15.37:0/1601817659 >> 172.21.15.37:0/1601817659 conn(0x7f258006f940 msgr2=0x7f25800b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:30.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.302+0000 7f2585a00640 1 -- 172.21.15.37:0/1601817659 shutdown_connections 2024-09-21T06:33:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.302+0000 7f2585a00640 1 -- 172.21.15.37:0/1601817659 wait complete. 2024-09-21T06:33:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.305+0000 7f2585a00640 1 Processor -- start 2024-09-21T06:33:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.305+0000 7f2585a00640 1 -- start start 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2564003a50 0x7f2580144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258013d510 0x7f258013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2580160dd0 con 0x7f25800d0170 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2580160c50 con 0x7f2564003a50 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2580160f50 con 0x7f258013d510 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53994/0 (socket says 172.21.15.37:53994) 2024-09-21T06:33:30.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 -- 172.21.15.37:0/4204609020 learned_addr learned my addr 172.21.15.37:0/4204609020 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:30.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257fe00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2564003a50 0x7f2580144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f2585000640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258013d510 0x7f258013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 -- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258013d510 msgr2=0x7f258013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258013d510 0x7f258013d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 -- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2564003a50 msgr2=0x7f2580144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2564003a50 0x7f2580144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.306+0000 7f257f400640 1 -- 172.21.15.37:0/4204609020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801f5230 con 0x7f25800d0170 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f257f400640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f256c003ed0 tx=0x7f256c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c015070 con 0x7f25800d0170 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f256c011840 con 0x7f25800d0170 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c011c70 con 0x7f25800d0170 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2580141bf0 con 0x7f25800d0170 2024-09-21T06:33:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.307+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801f59e0 con 0x7f25800d0170 2024-09-21T06:33:30.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.308+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25800ccc50 con 0x7f25800d0170 2024-09-21T06:33:30.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.309+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f256c018070 con 0x7f25800d0170 2024-09-21T06:33:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.310+0000 7f2577e00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 0x7f256007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.311+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f256c09ba60 con 0x7f25800d0170 2024-09-21T06:33:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.311+0000 7f257fe00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 0x7f256007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.312+0000 7f257fe00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 0x7f256007b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2570033000 tx=0x7f2570038040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.315+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f256c0636b0 con 0x7f25800d0170 2024-09-21T06:33:30.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.515+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f258013e330 con 0x7f25800d0170 2024-09-21T06:33:30.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.518+0000 7f2577e00640 1 -- 172.21.15.37:0/4204609020 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16957 (secure 0 0 0) 0x7f258013e330 con 0x7f25800d0170 2024-09-21T06:33:30.519 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:30.519 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:00.819168+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:32:57.301028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T06:32:58.299171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 msgr2=0x7f256007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 0x7f256007b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2570033000 tx=0x7f2570038040 comp rx=0 tx=0).stop 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 msgr2=0x7f25800d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f256c003ed0 tx=0x7f256c001630 comp rx=0 tx=0).stop 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 shutdown_connections 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2560078b90 0x7f256007b0a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25800d0170 0x7f25800d0580 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258013d510 0x7f258013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 --2- 172.21.15.37:0/4204609020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2564003a50 0x7f2580144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.519+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 >> 172.21.15.37:0/4204609020 conn(0x7f258006f940 msgr2=0x7f25800b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:30.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.520+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 shutdown_connections 2024-09-21T06:33:30.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.520+0000 7f2585a00640 1 -- 172.21.15.37:0/4204609020 wait complete. 2024-09-21T06:33:30.543 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:33:30.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 -- 172.21.15.37:0/3050132049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 msgr2=0x7fc4dc06def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 --2- 172.21.15.37:0/3050132049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc06def0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4005fd0 tx=0x7fc4d402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 -- 172.21.15.37:0/3050132049 shutdown_connections 2024-09-21T06:33:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 --2- 172.21.15.37:0/3050132049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc06def0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 --2- 172.21.15.37:0/3050132049 >> v2:172.21.15.150:3300/0 conn(0x7fc4dc152530 0x7fc4dc0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 -- 172.21.15.37:0/3050132049 >> 172.21.15.37:0/3050132049 conn(0x7fc4dc06f940 msgr2=0x7fc4dc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 -- 172.21.15.37:0/3050132049 shutdown_connections 2024-09-21T06:33:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.770+0000 7fc4e2400640 1 -- 172.21.15.37:0/3050132049 wait complete. 2024-09-21T06:33:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.772+0000 7fc4e2400640 1 Processor -- start 2024-09-21T06:33:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.772+0000 7fc4e2400640 1 -- start start 2024-09-21T06:33:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 0x7fc4dc145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc0cf5b0 con 0x7fc4dc152530 2024-09-21T06:33:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc0cf430 con 0x7fc4dc073dd0 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc0cf730 con 0x7fc4dc0b39d0 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 0x7fc4dc145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4d3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54002/0 (socket says 172.21.15.37:54002) 2024-09-21T06:33:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 -- 172.21.15.37:0/4058201749 learned_addr learned my addr 172.21.15.37:0/4058201749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 -- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 msgr2=0x7fc4dc15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 -- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 msgr2=0x7fc4dc145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 0x7fc4dc145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 -- 172.21.15.37:0/4058201749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc1513d0 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1000640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 0x7fc4dc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e1a00640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc4d402aea0 tx=0x7fc4d40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4031da0 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d4003a60 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc1510f0 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.775+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40330d0 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.776+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc14ef60 con 0x7fc4dc152530 2024-09-21T06:33:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.776+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4dc0cfb50 con 0x7fc4dc152530 2024-09-21T06:33:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.779+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4d4033270 con 0x7fc4dc152530 2024-09-21T06:33:30.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.779+0000 7fc4d9600640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 0x7fc4b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:30.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.779+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc4d40bbf40 con 0x7fc4dc152530 2024-09-21T06:33:30.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.779+0000 7fc4e1000640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 0x7fc4b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:30.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.781+0000 7fc4e1000640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 0x7fc4b807b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc007f20 tx=0x7fc4cc007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:30.783+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc4d404a030 con 0x7fc4dc152530 2024-09-21T06:33:31.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.022+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4dc150e80 con 0x7fc4dc152530 2024-09-21T06:33:31.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.023+0000 7fc4d9600640 1 -- 172.21.15.37:0/4058201749 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5275 (secure 0 0 0) 0x7fc4d403f030 con 0x7fc4dc152530 2024-09-21T06:33:31.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:31.025 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"btime":"2024-09-21T06:33:29:927104+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:29.856592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870,6858,6852]},"id":5}]} 2024-09-21T06:33:31.026 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 80 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 msgr2=0x7fc4b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 0x7fc4b807b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc007f20 tx=0x7fc4cc007eb0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 msgr2=0x7fc4dc15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc4d402aea0 tx=0x7fc4d40316f0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 shutdown_connections 2024-09-21T06:33:31.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc4b8078b90 0x7fc4b807b0a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4dc152530 0x7fc4dc15a940 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4dc0b39d0 0x7fc4dc15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 --2- 172.21.15.37:0/4058201749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4dc073dd0 0x7fc4dc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 >> 172.21.15.37:0/4058201749 conn(0x7fc4dc06f940 msgr2=0x7fc4dc06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 shutdown_connections 2024-09-21T06:33:31.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.026+0000 7fc4e2400640 1 -- 172.21.15.37:0/4058201749 wait complete. 2024-09-21T06:33:31.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.275+0000 7fb68d600640 1 -- 172.21.15.37:0/659647940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 msgr2=0x7fb66c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.275+0000 7fb68d600640 1 --2- 172.21.15.37:0/659647940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb66c00b9c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb680001d90 tx=0x7fb66c005830 comp rx=0 tx=0).stop 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.275+0000 7fb68d600640 1 -- 172.21.15.37:0/659647940 shutdown_connections 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.275+0000 7fb68d600640 1 --2- 172.21.15.37:0/659647940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb66c00b9c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.275+0000 7fb68d600640 1 -- 172.21.15.37:0/659647940 >> 172.21.15.37:0/659647940 conn(0x7fb6880cca40 msgr2=0x7fb6880cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.276+0000 7fb68d600640 1 -- 172.21.15.37:0/659647940 shutdown_connections 2024-09-21T06:33:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.276+0000 7fb68d600640 1 -- 172.21.15.37:0/659647940 wait complete. 2024-09-21T06:33:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.278+0000 7fb68d600640 1 Processor -- start 2024-09-21T06:33:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.278+0000 7fb68d600640 1 -- start start 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb688153e40 0x7fb688154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6881e5db0 0x7fb6881e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb68806e180 con 0x7fb66c005250 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb68806e000 con 0x7fb6881e5db0 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.279+0000 7fb68d600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880cbc10 con 0x7fb688153e40 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54018/0 (socket says 172.21.15.37:54018) 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 -- 172.21.15.37:0/4188425205 learned_addr learned my addr 172.21.15.37:0/4188425205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb687400640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6881e5db0 0x7fb6881e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb67fe00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb688153e40 0x7fb688154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 -- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb688153e40 msgr2=0x7fb688154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb688153e40 0x7fb688154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 -- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6881e5db0 msgr2=0x7fb6881e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6881e5db0 0x7fb6881e61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 -- 172.21.15.37:0/4188425205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68809e2e0 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.280+0000 7fb686a00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb6780066b0 tx=0x7fb6780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.281+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678015070 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.281+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb678011920 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.281+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6881e6700 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.281+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678011d70 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.281+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68809e860 con 0x7fb66c005250 2024-09-21T06:33:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.282+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6880cc170 con 0x7fb66c005250 2024-09-21T06:33:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.283+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb678018070 con 0x7fb66c005250 2024-09-21T06:33:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.284+0000 7fb67ea00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.284+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb67809bce0 con 0x7fb66c005250 2024-09-21T06:33:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.285+0000 7fb67fe00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.285+0000 7fb67fe00640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6881e6a40 tx=0x7fb670005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.288+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb678063f00 con 0x7fb66c005250 2024-09-21T06:33:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.525+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb68815e560 con 0x7fb66c005250 2024-09-21T06:33:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.526+0000 7fb67ea00640 1 -- 172.21.15.37:0/4188425205 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5275 (secure 0 0 0) 0x7fb678063800 con 0x7fb66c005250 2024-09-21T06:33:31.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:31.527 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"btime":"2024-09-21T06:33:29:927104+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:32:59.345604+0000","modified":"2024-09-21T06:33:29.856592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6870,"mds_1":6858,"mds_2":6852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6852":{"gid":6852,"name":"a","rank":2,"incarnation":76,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6833/3081570660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3081570660},{"type":"v1","addr":"172.21.15.37:6833","nonce":3081570660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6858":{"gid":6858,"name":"c","rank":1,"incarnation":74,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/2799311131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2799311131},{"type":"v1","addr":"172.21.15.37:6835","nonce":2799311131}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6870":{"gid":6870,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6837/1559036255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1559036255},{"type":"v1","addr":"172.21.15.150:6837","nonce":1559036255}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6870,"qdb_cluster":[6870,6858,6852]},"id":5}]} 2024-09-21T06:33:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 80 2024-09-21T06:33:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 msgr2=0x7fb66407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6881e6a40 tx=0x7fb670005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 msgr2=0x7fb6881f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb6780066b0 tx=0x7fb6780039c0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 shutdown_connections 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6881e5db0 0x7fb6881e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb688153e40 0x7fb688154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 --2- 172.21.15.37:0/4188425205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb66c005250 0x7fb6881f86c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 >> 172.21.15.37:0/4188425205 conn(0x7fb6880cca40 msgr2=0x7fb6880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 shutdown_connections 2024-09-21T06:33:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.528+0000 7fb68d600640 1 -- 172.21.15.37:0/4188425205 wait complete. 2024-09-21T06:33:31.551 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 -- 172.21.15.37:0/2811687829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 msgr2=0x7f44e406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 --2- 172.21.15.37:0/2811687829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e406ddf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009830 tx=0x7f44d002a460 comp rx=0 tx=0).stop 2024-09-21T06:33:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 -- 172.21.15.37:0/2811687829 shutdown_connections 2024-09-21T06:33:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 --2- 172.21.15.37:0/2811687829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e406ddf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 -- 172.21.15.37:0/2811687829 >> 172.21.15.37:0/2811687829 conn(0x7f44e40cca40 msgr2=0x7f44e40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 -- 172.21.15.37:0/2811687829 shutdown_connections 2024-09-21T06:33:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.777+0000 7f44ea000640 1 -- 172.21.15.37:0/2811687829 wait complete. 2024-09-21T06:33:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.779+0000 7f44ea000640 1 Processor -- start 2024-09-21T06:33:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.779+0000 7f44ea000640 1 -- start start 2024-09-21T06:33:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e40c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 0x7f44e41f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44e406f670 con 0x7f44e406e330 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44e406f4f0 con 0x7f44e4073dd0 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44ea000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f44e406f7f0 con 0x7f44e4074cb0 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44e3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44e3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54028/0 (socket says 172.21.15.37:54028) 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44e3400640 1 -- 172.21.15.37:0/783668283 learned_addr learned my addr 172.21.15.37:0/783668283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.781+0000 7f44e2a00640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e40c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3e00640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 0x7f44e41f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 -- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 msgr2=0x7f44e41f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 0x7f44e41f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 -- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 msgr2=0x7f44e40c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e40c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 -- 172.21.15.37:0/783668283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40c9e60 con 0x7f44e406e330 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3e00640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 0x7f44e41f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44e3400640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c550 tx=0x7f44d400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4016020 con 0x7f44e406e330 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d4012920 con 0x7f44e406e330 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4012d30 con 0x7f44e406e330 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.782+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e409bc90 con 0x7f44e406e330 2024-09-21T06:33:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.783+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e409c250 con 0x7f44e406e330 2024-09-21T06:33:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.784+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e406ddf0 con 0x7f44e406e330 2024-09-21T06:33:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.785+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f44d4019020 con 0x7f44e406e330 2024-09-21T06:33:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.786+0000 7f44db400640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 0x7f44c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.786+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f44d409ae90 con 0x7f44e406e330 2024-09-21T06:33:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.786+0000 7f44e2a00640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 0x7f44c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.787+0000 7f44e2a00640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 0x7f44c407b0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f44d0034000 tx=0x7f44d002c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.790+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f44d4020020 con 0x7f44e406e330 2024-09-21T06:33:31.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.995+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44e41580f0 con 0x7f44e406e330 2024-09-21T06:33:31.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.996+0000 7f44db400640 1 -- 172.21.15.37:0/783668283 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16957 (secure 0 0 0) 0x7f44d409c090 con 0x7f44e406e330 2024-09-21T06:33:31.998 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:31.998 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:00.819168+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:32:57.301028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T06:32:58.299171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 msgr2=0x7f44c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 0x7f44c407b0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f44d0034000 tx=0x7f44d002c040 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 msgr2=0x7f44e40c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c550 tx=0x7f44d400ca10 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 shutdown_connections 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f44c4078bc0 0x7f44c407b0d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44e4074cb0 0x7f44e41f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e4073dd0 0x7f44e40c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 --2- 172.21.15.37:0/783668283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e406e330 0x7f44e40c7ed0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 >> 172.21.15.37:0/783668283 conn(0x7f44e40cca40 msgr2=0x7f44e4068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:31.999+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 shutdown_connections 2024-09-21T06:33:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.000+0000 7f44ea000640 1 -- 172.21.15.37:0/783668283 wait complete. 2024-09-21T06:33:32.027 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 -- 172.21.15.37:0/1893117016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee94156a50 msgr2=0x7fee94074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 --2- 172.21.15.37:0/1893117016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee94156a50 0x7fee94074620 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fee84009830 tx=0x7fee8402a510 comp rx=0 tx=0).stop 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 -- 172.21.15.37:0/1893117016 shutdown_connections 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 --2- 172.21.15.37:0/1893117016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee9406e500 0x7fee940cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 --2- 172.21.15.37:0/1893117016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee94156a50 0x7fee94074620 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 -- 172.21.15.37:0/1893117016 >> 172.21.15.37:0/1893117016 conn(0x7fee940b1b90 msgr2=0x7fee940b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.254+0000 7fee99000640 1 -- 172.21.15.37:0/1893117016 shutdown_connections 2024-09-21T06:33:32.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.255+0000 7fee99000640 1 -- 172.21.15.37:0/1893117016 wait complete. 2024-09-21T06:33:32.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.257+0000 7fee99000640 1 Processor -- start 2024-09-21T06:33:32.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.257+0000 7fee99000640 1 -- start start 2024-09-21T06:33:32.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee94074b60 0x7fee940b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee94156a50 0x7fee94159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee940cf7d0 con 0x7fee941f66a0 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee940cf650 con 0x7fee94074b60 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee99000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fee940cf950 con 0x7fee94156a50 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.258+0000 7fee92800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54046/0 (socket says 172.21.15.37:54046) 2024-09-21T06:33:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 -- 172.21.15.37:0/631248403 learned_addr learned my addr 172.21.15.37:0/631248403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee91e00640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee94074b60 0x7fee940b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee91400640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee94156a50 0x7fee94159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 -- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee94156a50 msgr2=0x7fee94159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee94156a50 0x7fee94159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 -- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee94074b60 msgr2=0x7fee940b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee94074b60 0x7fee940b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 -- 172.21.15.37:0/631248403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9415a030 con 0x7fee941f66a0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.259+0000 7fee92800640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9c0 tx=0x7fee8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.260+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c004d20 con 0x7fee941f66a0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.260+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee8c00cb20 con 0x7fee941f66a0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.260+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c005020 con 0x7fee941f66a0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.260+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee940b5160 con 0x7fee941f66a0 2024-09-21T06:33:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.260+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee940b5720 con 0x7fee941f66a0 2024-09-21T06:33:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.261+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee940cef00 con 0x7fee941f66a0 2024-09-21T06:33:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.262+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee8c0051c0 con 0x7fee941f66a0 2024-09-21T06:33:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.263+0000 7fee8a000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 0x7fee7407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.263+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fee8c09b840 con 0x7fee941f66a0 2024-09-21T06:33:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.264+0000 7fee91e00640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 0x7fee7407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.264+0000 7fee91e00640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 0x7fee7407b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee84034000 tx=0x7fee84004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.268+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee9415a030 con 0x7fee941f66a0 2024-09-21T06:33:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.506+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fee940b58f0 con 0x7fee941f66a0 2024-09-21T06:33:32.508 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:33:32.507+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:33:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.509+0000 7fee8a000640 1 -- 172.21.15.37:0/631248403 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7fee8c06c020 con 0x7fee941f66a0 2024-09-21T06:33:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 msgr2=0x7fee7407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 0x7fee7407b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee84034000 tx=0x7fee84004730 comp rx=0 tx=0).stop 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 msgr2=0x7fee941f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9c0 tx=0x7fee8c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 shutdown_connections 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fee74078bc0 0x7fee7407b0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee941f66a0 0x7fee941f6ab0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee94156a50 0x7fee94159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 --2- 172.21.15.37:0/631248403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee94074b60 0x7fee940b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 >> 172.21.15.37:0/631248403 conn(0x7fee940b1b90 msgr2=0x7fee940afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 shutdown_connections 2024-09-21T06:33:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.513+0000 7fee99000640 1 -- 172.21.15.37:0/631248403 wait complete. 2024-09-21T06:33:32.525 INFO:tasks.ceph.mds.b.smithi150.stderr: -9663> 2024-09-21T06:33:01.808+0000 7fd256000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:33:32.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:33:32.652 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:33:32.652 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:33:32.653 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:33:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 -- 172.21.15.37:0/2843557599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c040d2760 msgr2=0x7f2c04162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 --2- 172.21.15.37:0/2843557599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c040d2760 0x7f2c04162350 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005fd0 tx=0x7f2bf402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 -- 172.21.15.37:0/2843557599 shutdown_connections 2024-09-21T06:33:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 --2- 172.21.15.37:0/2843557599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c04162890 0x7f2c04154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 --2- 172.21.15.37:0/2843557599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c040d2760 0x7f2c04162350 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 -- 172.21.15.37:0/2843557599 >> 172.21.15.37:0/2843557599 conn(0x7f2c040d4360 msgr2=0x7f2c040d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 -- 172.21.15.37:0/2843557599 shutdown_connections 2024-09-21T06:33:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.776+0000 7f2c03e00640 1 -- 172.21.15.37:0/2843557599 wait complete. 2024-09-21T06:33:32.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.778+0000 7f2c03e00640 1 Processor -- start 2024-09-21T06:33:32.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.778+0000 7f2c03e00640 1 -- start start 2024-09-21T06:33:32.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c040d2760 0x7f2c040ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c04162890 0x7f2c041f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0415f290 con 0x7f2c04155460 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040cfee0 con 0x7f2c04162890 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0415f410 con 0x7f2c040d2760 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2bfbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2bfbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54092/0 (socket says 172.21.15.37:54092) 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2bfbe00640 1 -- 172.21.15.37:0/3677007887 learned_addr learned my addr 172.21.15.37:0/3677007887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c02a00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c040d2760 0x7f2c040ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.780+0000 7f2c03400640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c04162890 0x7f2c041f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 -- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c040d2760 msgr2=0x7f2c040ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c040d2760 0x7f2c040ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 -- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c04162890 msgr2=0x7f2c041f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:32.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c04162890 0x7f2c041f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:32.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 -- 172.21.15.37:0/3677007887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c040b8430 con 0x7f2c04155460 2024-09-21T06:33:32.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfbe00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9f0 tx=0x7f2bec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00cd80 con 0x7f2c04155460 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bec004020 con 0x7f2c04155460 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec004470 con 0x7f2c04155460 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c040b8710 con 0x7f2c04155460 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.781+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c040b8cd0 con 0x7f2c04155460 2024-09-21T06:33:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.783+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c040ce5b0 con 0x7f2c04155460 2024-09-21T06:33:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.784+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2bec0035f0 con 0x7f2c04155460 2024-09-21T06:33:32.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.785+0000 7f2bfaa00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 0x7f2be007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:32.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.785+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6359+0+0 (secure 0 0 0) 0x7f2bec09baf0 con 0x7f2c04155460 2024-09-21T06:33:32.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.785+0000 7f2c02a00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 0x7f2be007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.786+0000 7f2c02a00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 0x7f2be007b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4033000 tx=0x7f2bf4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:32.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:32.789+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2bec063530 con 0x7f2c04155460 2024-09-21T06:33:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.026+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2c0415f9a0 con 0x7f2c04155460 2024-09-21T06:33:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.511+0000 7f2bfaa00640 1 -- 172.21.15.37:0/3677007887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) ==== 104+0+0 (secure 0 0 0) 0x7f2bec067f60 con 0x7f2c04155460 2024-09-21T06:33:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 msgr2=0x7f2be007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 0x7f2be007b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4033000 tx=0x7f2bf4004730 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 msgr2=0x7f2c04148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9f0 tx=0x7f2bec00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 shutdown_connections 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2be0078b90 0x7f2be007b0a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c04155460 0x7f2c04148e00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c04162890 0x7f2c041f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 --2- 172.21.15.37:0/3677007887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c040d2760 0x7f2c040ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 >> 172.21.15.37:0/3677007887 conn(0x7f2c040d4360 msgr2=0x7f2c0415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 shutdown_connections 2024-09-21T06:33:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.514+0000 7f2c03e00640 1 -- 172.21.15.37:0/3677007887 wait complete. 2024-09-21T06:33:33.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 -- 172.21.15.37:0/1267429777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf8160c20 msgr2=0x7f5bf806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 --2- 172.21.15.37:0/1267429777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf8160c20 0x7f5bf806d820 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0005fd0 tx=0x7f5bf002aa90 comp rx=0 tx=0).stop 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 -- 172.21.15.37:0/1267429777 shutdown_connections 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 --2- 172.21.15.37:0/1267429777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf8160c20 0x7f5bf806d820 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 --2- 172.21.15.37:0/1267429777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160810 0x7f5bf806d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 -- 172.21.15.37:0/1267429777 >> 172.21.15.37:0/1267429777 conn(0x7f5bf80ccac0 msgr2=0x7f5bf80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:33.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 -- 172.21.15.37:0/1267429777 shutdown_connections 2024-09-21T06:33:33.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.774+0000 7f5c00200640 1 -- 172.21.15.37:0/1267429777 wait complete. 2024-09-21T06:33:33.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.776+0000 7f5c00200640 1 Processor -- start 2024-09-21T06:33:33.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.776+0000 7f5c00200640 1 -- start start 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 0x7f5bf81f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160c20 0x7f5bf81f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf806e0f0 con 0x7f5bf80ca690 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf806df70 con 0x7f5bf8160810 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5c00200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf806e270 con 0x7f5bf8160c20 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5bfd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5bfd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54116/0 (socket says 172.21.15.37:54116) 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.778+0000 7f5bfd800640 1 -- 172.21.15.37:0/1022271336 learned_addr learned my addr 172.21.15.37:0/1022271336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfe200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160c20 0x7f5bf81f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 -- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160c20 msgr2=0x7f5bf81f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160c20 0x7f5bf81f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 -- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 msgr2=0x7f5bf81f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfce00640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 0x7f5bf81f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 0x7f5bf81f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 -- 172.21.15.37:0/1022271336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80ca200 con 0x7f5bf80ca690 2024-09-21T06:33:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfce00640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 0x7f5bf81f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bfd800640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c460 tx=0x7f5bec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0052d0 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bec005490 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80c71f0 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec012870 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.779+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf80c77b0 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.780+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf8160100 con 0x7f5bf80ca690 2024-09-21T06:33:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.782+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5bec012a10 con 0x7f5bf80ca690 2024-09-21T06:33:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.783+0000 7f5bf5600640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 0x7f5bd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.783+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6359+0+0 (secure 0 0 0) 0x7f5bec09ab90 con 0x7f5bf80ca690 2024-09-21T06:33:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.783+0000 7f5bfce00640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 0x7f5bd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.784+0000 7f5bfce00640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 0x7f5bd807b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8158a00 tx=0x7f5be4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.787+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bec062ca0 con 0x7f5bf80ca690 2024-09-21T06:33:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:33.988+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5bf80b5f20 con 0x7f5bf80ca690 2024-09-21T06:33:34.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.958+0000 7f5bf5600640 1 -- 172.21.15.37:0/1022271336 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f5bec09a030 con 0x7f5bf80ca690 2024-09-21T06:33:34.960 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:33:34.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 msgr2=0x7f5bd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:34.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 0x7f5bd807b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8158a00 tx=0x7f5be4005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 msgr2=0x7f5bf814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c460 tx=0x7f5bec00c920 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 shutdown_connections 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bd8078b90 0x7f5bd807b0a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bf8160c20 0x7f5bf81f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf8160810 0x7f5bf81f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 --2- 172.21.15.37:0/1022271336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf80ca690 0x7f5bf814f280 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.963+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 >> 172.21.15.37:0/1022271336 conn(0x7f5bf80ccac0 msgr2=0x7f5bf80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:34.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.964+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 shutdown_connections 2024-09-21T06:33:34.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:34.964+0000 7f5c00200640 1 -- 172.21.15.37:0/1022271336 wait complete. 2024-09-21T06:33:34.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.235+0000 7f3477200640 1 -- 172.21.15.37:0/1442939421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34780b49b0 msgr2=0x7f34780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.235+0000 7f3477200640 1 --2- 172.21.15.37:0/1442939421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34780b49b0 0x7f34780b4dc0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3470005fd0 tx=0x7f347002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 -- 172.21.15.37:0/1442939421 shutdown_connections 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 --2- 172.21.15.37:0/1442939421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34780b49b0 0x7f34780b4dc0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 --2- 172.21.15.37:0/1442939421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3478155570 0x7f34780b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 -- 172.21.15.37:0/1442939421 >> 172.21.15.37:0/1442939421 conn(0x7f34780d0460 msgr2=0x7f347806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 -- 172.21.15.37:0/1442939421 shutdown_connections 2024-09-21T06:33:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.236+0000 7f3477200640 1 -- 172.21.15.37:0/1442939421 wait complete. 2024-09-21T06:33:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.238+0000 7f3477200640 1 Processor -- start 2024-09-21T06:33:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.238+0000 7f3477200640 1 -- start start 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.239+0000 7f3477200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34780b49b0 0x7f34781520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3477200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3478155570 0x7f34780afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3477200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3477200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34780ae180 con 0x7f347815f720 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3477200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34780ae000 con 0x7f3478155570 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3477200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f34780ae300 con 0x7f34780b49b0 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38402/0 (socket says 172.21.15.37:38402) 2024-09-21T06:33:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 -- 172.21.15.37:0/3915683972 learned_addr learned my addr 172.21.15.37:0/3915683972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 -- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34780b49b0 msgr2=0x7f34781520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34780b49b0 0x7f34781520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 -- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3478155570 msgr2=0x7f34780afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3478155570 0x7f34780afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.240+0000 7f3476800640 1 -- 172.21.15.37:0/3915683972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34780b0750 con 0x7f347815f720 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f3476800640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3470009cd0 tx=0x7f34700316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470031990 con 0x7f347815f720 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3470031c20 con 0x7f347815f720 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34781e78d0 con 0x7f347815f720 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700330d0 con 0x7f347815f720 2024-09-21T06:33:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.241+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781e7e50 con 0x7f347815f720 2024-09-21T06:33:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.242+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347815f1f0 con 0x7f347815f720 2024-09-21T06:33:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.244+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3470033270 con 0x7f347815f720 2024-09-21T06:33:35.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.244+0000 7f346e000640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 0x7f345807b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.245+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5921+0+0 (secure 0 0 0) 0x7f34700bc320 con 0x7f347815f720 2024-09-21T06:33:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.245+0000 7f3475e00640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 0x7f345807b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.246+0000 7f3475e00640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 0x7f345807b090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34680046b0 tx=0x7f34680012b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:35.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.249+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f347815f1f0 con 0x7f347815f720 2024-09-21T06:33:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.446+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f34780b09e0 con 0x7f347815f720 2024-09-21T06:33:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.965+0000 7f346e000640 1 -- 172.21.15.37:0/3915683972 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f347003f030 con 0x7f347815f720 2024-09-21T06:33:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:33:35.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 msgr2=0x7f345807b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 0x7f345807b090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34680046b0 tx=0x7f34680012b0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 msgr2=0x7f34781f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3470009cd0 tx=0x7f34700316d0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 shutdown_connections 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3458078b80 0x7f345807b090 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34680046b0 tx=0x7f34680012b0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f347815f720 0x7f34781f5930 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3478155570 0x7f34780afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 --2- 172.21.15.37:0/3915683972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34780b49b0 0x7f34781520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.969+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 >> 172.21.15.37:0/3915683972 conn(0x7f34780d0460 msgr2=0x7f3478158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.970+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 shutdown_connections 2024-09-21T06:33:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:35.970+0000 7f3477200640 1 -- 172.21.15.37:0/3915683972 wait complete. 2024-09-21T06:33:35.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:33:36.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 -- 172.21.15.37:0/660260952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cf6d0 msgr2=0x7f0d9c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 --2- 172.21.15.37:0/660260952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c0b3110 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c15d900 tx=0x7f0d9402a820 comp rx=0 tx=0).stop 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 -- 172.21.15.37:0/660260952 shutdown_connections 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 --2- 172.21.15.37:0/660260952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c0b3110 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 --2- 172.21.15.37:0/660260952 >> v2:172.21.15.150:3300/0 conn(0x7f0d9c1605f0 0x7f0d9c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 -- 172.21.15.37:0/660260952 >> 172.21.15.37:0/660260952 conn(0x7f0d9c0d05c0 msgr2=0x7f0d9c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 -- 172.21.15.37:0/660260952 shutdown_connections 2024-09-21T06:33:36.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.225+0000 7f0da0a00640 1 -- 172.21.15.37:0/660260952 wait complete. 2024-09-21T06:33:36.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.227+0000 7f0da0a00640 1 Processor -- start 2024-09-21T06:33:36.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.227+0000 7f0da0a00640 1 -- start start 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 0x7f0d9c0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0b3c80 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0b3b00 con 0x7f0d9c0cf6d0 2024-09-21T06:33:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0da0a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c0b3e00 con 0x7f0d9c1f29e0 2024-09-21T06:33:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0d9b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0d9b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38412/0 (socket says 172.21.15.37:38412) 2024-09-21T06:33:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0d9b400640 1 -- 172.21.15.37:0/3989055511 learned_addr learned my addr 172.21.15.37:0/3989055511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.229+0000 7f0d92a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9be00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 0x7f0d9c0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 -- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 msgr2=0x7f0d9c0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 0x7f0d9c0cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 -- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d9c0cf6d0 msgr2=0x7f0d9c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 -- 172.21.15.37:0/3989055511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c137050 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9be00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 0x7f0d9c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d9b400640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00c3a0 tx=0x7f0d8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c011790 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c011cc0 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c137550 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.230+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c137b10 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.231+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c0b42b0 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.233+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d8c01c460 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.234+0000 7f0d93e00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 0x7f0d7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.234+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f0d8c09ab00 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.234+0000 7f0d92a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 0x7f0d7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.235+0000 7f0d92a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 0x7f0d7807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:36.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.238+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d8c062f60 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.434+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0d9c15bb20 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.439+0000 7f0d93e00640 1 -- 172.21.15.37:0/3989055511 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f0d8c09ae20 con 0x7f0d9c0cc0e0 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 msgr2=0x7f0d7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 0x7f0d7807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84005850 comp rx=0 tx=0).stop 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 msgr2=0x7f0d9c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00c3a0 tx=0x7f0d8c00c860 comp rx=0 tx=0).stop 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 shutdown_connections 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0d78078b90 0x7f0d7807b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d9c1f29e0 0x7f0d9c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d9c0cf6d0 0x7f0d9c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 --2- 172.21.15.37:0/3989055511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d9c0cc0e0 0x7f0d9c06da30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 >> 172.21.15.37:0/3989055511 conn(0x7f0d9c0d05c0 msgr2=0x7f0d9c15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:36.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 shutdown_connections 2024-09-21T06:33:36.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.441+0000 7f0da0a00640 1 -- 172.21.15.37:0/3989055511 wait complete. 2024-09-21T06:33:36.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-21T06:33:36.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.694+0000 7f209ec00640 1 -- 172.21.15.37:0/1644311846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f207c003fd0 msgr2=0x7f207c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.694+0000 7f209ec00640 1 --2- 172.21.15.37:0/1644311846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f207c003fd0 0x7f207c004800 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2084009b50 tx=0x7f207c005830 comp rx=0 tx=0).stop 2024-09-21T06:33:36.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.694+0000 7f209cc00640 1 -- 172.21.15.37:0/1644311846 reap_dead start 2024-09-21T06:33:36.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.695+0000 7f209ec00640 1 -- 172.21.15.37:0/1644311846 shutdown_connections 2024-09-21T06:33:36.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.695+0000 7f209ec00640 1 -- 172.21.15.37:0/1644311846 >> 172.21.15.37:0/1644311846 conn(0x7f20980ccf00 msgr2=0x7f20980cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:36.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.695+0000 7f209ec00640 1 -- 172.21.15.37:0/1644311846 shutdown_connections 2024-09-21T06:33:36.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.695+0000 7f209ec00640 1 -- 172.21.15.37:0/1644311846 wait complete. 2024-09-21T06:33:36.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.697+0000 7f209ec00640 1 Processor -- start 2024-09-21T06:33:36.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.697+0000 7f209ec00640 1 -- start start 2024-09-21T06:33:36.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f207c003fd0 0x7f209815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 0x7f209815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20980b1dc0 con 0x7f20981f41e0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20980b1f40 con 0x7f20981595c0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.698+0000 7f209ec00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f20980b20c0 con 0x7f207c003fd0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38444/0 (socket says 172.21.15.37:38444) 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 -- 172.21.15.37:0/3452005562 learned_addr learned my addr 172.21.15.37:0/3452005562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f2097400640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 0x7f209815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f2097e00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f207c003fd0 0x7f209815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 -- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f207c003fd0 msgr2=0x7f209815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f207c003fd0 0x7f209815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 -- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 msgr2=0x7f209815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 0x7f209815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 -- 172.21.15.37:0/3452005562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981f6eb0 con 0x7f20981f41e0 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f2097400640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 0x7f209815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.699+0000 7f209cc00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f20980b4bd0 tx=0x7f2090006710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.700+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209000e810 con 0x7f20981f41e0 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.700+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209000eaa0 con 0x7f20981f41e0 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.700+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090011460 con 0x7f20981f41e0 2024-09-21T06:33:36.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.700+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981f7190 con 0x7f20981f41e0 2024-09-21T06:33:36.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.700+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981f7750 con 0x7f20981f41e0 2024-09-21T06:33:36.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.701+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20980c7190 con 0x7f20981f41e0 2024-09-21T06:33:36.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.702+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2090019020 con 0x7f20981f41e0 2024-09-21T06:33:36.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.703+0000 7f208fe00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 0x7f207807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:36.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.703+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f209009ac70 con 0x7f20981f41e0 2024-09-21T06:33:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.704+0000 7f2097e00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 0x7f207807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.704+0000 7f2097e00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 0x7f207807b0d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2088009b10 tx=0x7f2088005f30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.708+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2090020020 con 0x7f20981f41e0 2024-09-21T06:33:36.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:36.991+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7f20980d0c80 con 0x7f20981f41e0 2024-09-21T06:33:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.973+0000 7f208fe00640 1 -- 172.21.15.37:0/3452005562 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v302) ==== 141+0+0 (secure 0 0 0) 0x7f20980b2710 con 0x7f20981f41e0 2024-09-21T06:33:37.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 msgr2=0x7f207807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 0x7f207807b0d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2088009b10 tx=0x7f2088005f30 comp rx=0 tx=0).stop 2024-09-21T06:33:37.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 msgr2=0x7f209806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f20980b4bd0 tx=0x7f2090006710 comp rx=0 tx=0).stop 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 shutdown_connections 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2078078bc0 0x7f207807b0d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20981f41e0 0x7f209806edf0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20981595c0 0x7f209815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 --2- 172.21.15.37:0/3452005562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f207c003fd0 0x7f209815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 >> 172.21.15.37:0/3452005562 conn(0x7f20980ccf00 msgr2=0x7f20980b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 shutdown_connections 2024-09-21T06:33:37.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:37.977+0000 7f209ec00640 1 -- 172.21.15.37:0/3452005562 wait complete. 2024-09-21T06:33:38.004 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:33:38.004 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:33:38.004 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 -- 172.21.15.37:0/3715295291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cccb0 msgr2=0x7f9e00156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 --2- 172.21.15.37:0/3715295291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cccb0 0x7f9e00156520 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005fd0 tx=0x7f9dec02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 -- 172.21.15.37:0/3715295291 shutdown_connections 2024-09-21T06:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 --2- 172.21.15.37:0/3715295291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cccb0 0x7f9e00156520 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 -- 172.21.15.37:0/3715295291 >> 172.21.15.37:0/3715295291 conn(0x7f9e000ceaf0 msgr2=0x7f9e000cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 -- 172.21.15.37:0/3715295291 shutdown_connections 2024-09-21T06:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.235+0000 7f9e06a00640 1 -- 172.21.15.37:0/3715295291 wait complete. 2024-09-21T06:33:38.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.237+0000 7f9e06a00640 1 Processor -- start 2024-09-21T06:33:38.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.237+0000 7f9e06a00640 1 -- start start 2024-09-21T06:33:38.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9e000cccb0 0x7f9e001557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e00156a60 0x7f9e000a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0015d420 con 0x7f9e000cc8a0 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00071320 con 0x7f9e00156a60 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0015d5a0 con 0x7f9e000cccb0 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e05600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e05600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38460/0 (socket says 172.21.15.37:38460) 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e05600640 1 -- 172.21.15.37:0/823180837 learned_addr learned my addr 172.21.15.37:0/823180837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e04c00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9e000cccb0 0x7f9e001557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.239+0000 7f9e06000640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e00156a60 0x7f9e000a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 -- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9e000cccb0 msgr2=0x7f9e001557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9e000cccb0 0x7f9e001557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 -- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e00156a60 msgr2=0x7f9e000a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e00156a60 0x7f9e000a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 -- 172.21.15.37:0/823180837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0015bbf0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e05600640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9df400c3a0 tx=0x7f9df400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4017070 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9df40053c0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4003850 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e001578b0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.240+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e00157e70 con 0x7f9e000cc8a0 2024-09-21T06:33:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.242+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e000694e0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.243+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9df401a070 con 0x7f9e000cc8a0 2024-09-21T06:33:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.244+0000 7f9dfd600640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 0x7f9de007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.244+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f9df409a0d0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.245+0000 7f9e04c00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 0x7f9de007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.246+0000 7f9e04c00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 0x7f9de007b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9dec033000 tx=0x7f9dec004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.249+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e000694e0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.446+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e00156520 con 0x7f9e000cc8a0 2024-09-21T06:33:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.447+0000 7f9dfd600640 1 -- 172.21.15.37:0/823180837 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13580 (secure 0 0 0) 0x7f9df40625c0 con 0x7f9e000cc8a0 2024-09-21T06:33:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:38.448 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:35.959512+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6832/3081570660":"2024-09-22T06:33:32.507153+0000","172.21.15.37:6833/3081570660":"2024-09-22T06:33:32.507153+0000","172.21.15.150:6837/1559036255":"2024-09-22T06:33:32.507176+0000","172.21.15.150:6836/1559036255":"2024-09-22T06:33:32.507176+0000","172.21.15.37:6835/2799311131":"2024-09-22T06:33:32.507171+0000","172.21.15.37:6834/2799311131":"2024-09-22T06:33:32.507171+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 msgr2=0x7f9de007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 0x7f9de007b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9dec033000 tx=0x7f9dec004730 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 msgr2=0x7f9e0015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9df400c3a0 tx=0x7f9df400c860 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 shutdown_connections 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9de0078b90 0x7f9de007b0a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e00156a60 0x7f9e000a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9e000cccb0 0x7f9e001557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 --2- 172.21.15.37:0/823180837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e000cc8a0 0x7f9e0015b2f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 >> 172.21.15.37:0/823180837 conn(0x7f9e000ceaf0 msgr2=0x7f9e0015e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 shutdown_connections 2024-09-21T06:33:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.449+0000 7f9e06a00640 1 -- 172.21.15.37:0/823180837 wait complete. 2024-09-21T06:33:38.476 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:33:38.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.706+0000 7f0f44400640 1 -- 172.21.15.37:0/3123356892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 msgr2=0x7f0f24004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.706+0000 7f0f44400640 1 --2- 172.21.15.37:0/3123356892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f24004060 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009b50 tx=0x7f0f24005830 comp rx=0 tx=0).stop 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.706+0000 7f0f42400640 1 -- 172.21.15.37:0/3123356892 reap_dead start 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.707+0000 7f0f44400640 1 -- 172.21.15.37:0/3123356892 shutdown_connections 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.707+0000 7f0f44400640 1 -- 172.21.15.37:0/3123356892 >> 172.21.15.37:0/3123356892 conn(0x7f0f3c0cca40 msgr2=0x7f0f3c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.707+0000 7f0f44400640 1 -- 172.21.15.37:0/3123356892 shutdown_connections 2024-09-21T06:33:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.707+0000 7f0f44400640 1 -- 172.21.15.37:0/3123356892 wait complete. 2024-09-21T06:33:38.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.709+0000 7f0f44400640 1 Processor -- start 2024-09-21T06:33:38.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.709+0000 7f0f44400640 1 -- start start 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f3c15a010 0x7f0f3c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f3c0b37e0 0x7f0f3c1f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0af5a0 con 0x7f0f24003c10 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0af420 con 0x7f0f3c15a010 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.710+0000 7f0f44400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0af720 con 0x7f0f3c0b37e0 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38474/0 (socket says 172.21.15.37:38474) 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 -- 172.21.15.37:0/806718910 learned_addr learned my addr 172.21.15.37:0/806718910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f42400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f3c0b37e0 0x7f0f3c1f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41000640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f3c15a010 0x7f0f3c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 -- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f3c0b37e0 msgr2=0x7f0f3c1f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f3c0b37e0 0x7f0f3c1f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 -- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f3c15a010 msgr2=0x7f0f3c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f3c15a010 0x7f0f3c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 -- 172.21.15.37:0/806718910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0b3bf0 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.711+0000 7f0f41a00640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009c70 tx=0x7f0f30033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.712+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30038040 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.712+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f3002b7e0 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.712+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c0a1180 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.712+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30004280 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.712+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c0a0a00 con 0x7f0f24003c10 2024-09-21T06:33:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.713+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c0cde00 con 0x7f0f24003c10 2024-09-21T06:33:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.714+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f30045020 con 0x7f0f24003c10 2024-09-21T06:33:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.715+0000 7f0f39600640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.715+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f0f300bb640 con 0x7f0f24003c10 2024-09-21T06:33:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.715+0000 7f0f41000640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.717+0000 7f0f41000640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0f28001860 tx=0x7f0f280011c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.720+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f3c0b3bf0 con 0x7f0f24003c10 2024-09-21T06:33:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.918+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f3c0a0ea0 con 0x7f0f24003c10 2024-09-21T06:33:38.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.919+0000 7f0f39600640 1 -- 172.21.15.37:0/806718910 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f0f3002b9b0 con 0x7f0f24003c10 2024-09-21T06:33:38.920 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:33:38.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 msgr2=0x7f0f1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0f28001860 tx=0x7f0f280011c0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 msgr2=0x7f0f3c0b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:38.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009c70 tx=0x7f0f30033670 comp rx=0 tx=0).stop 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 shutdown_connections 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f3c0b37e0 0x7f0f3c1f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f3c15a010 0x7f0f3c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 --2- 172.21.15.37:0/806718910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f24003c10 0x7f0f3c0b4860 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 >> 172.21.15.37:0/806718910 conn(0x7f0f3c0cca40 msgr2=0x7f0f3c074770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 shutdown_connections 2024-09-21T06:33:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:38.921+0000 7f0f44400640 1 -- 172.21.15.37:0/806718910 wait complete. 2024-09-21T06:33:38.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-21T06:33:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 -- 172.21.15.37:0/148074600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1804074cb0 msgr2=0x7f1804073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 --2- 172.21.15.37:0/148074600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1804074cb0 0x7f1804073dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009860 tx=0x7f17f402a5a0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 -- 172.21.15.37:0/148074600 shutdown_connections 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 --2- 172.21.15.37:0/148074600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1804074cb0 0x7f1804073dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 -- 172.21.15.37:0/148074600 >> 172.21.15.37:0/148074600 conn(0x7f18040cca40 msgr2=0x7f18040cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 -- 172.21.15.37:0/148074600 shutdown_connections 2024-09-21T06:33:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.169+0000 7f1808c00640 1 -- 172.21.15.37:0/148074600 wait complete. 2024-09-21T06:33:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.172+0000 7f1808c00640 1 Processor -- start 2024-09-21T06:33:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.172+0000 7f1808c00640 1 -- start start 2024-09-21T06:33:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18040cbc10 0x7f180409e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1804157710 0x7f180409e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f180406f670 con 0x7f1804074cb0 2024-09-21T06:33:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f180406f4f0 con 0x7f1804157710 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1808c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f180406f7f0 con 0x7f18040cbc10 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1801e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1801e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38490/0 (socket says 172.21.15.37:38490) 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.173+0000 7f1801e00640 1 -- 172.21.15.37:0/2857635073 learned_addr learned my addr 172.21.15.37:0/2857635073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f17fbe00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18040cbc10 0x7f180409e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1802800640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1804157710 0x7f180409e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 -- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18040cbc10 msgr2=0x7f180409e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18040cbc10 0x7f180409e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 -- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1804157710 msgr2=0x7f180409e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1804157710 0x7f180409e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 -- 172.21.15.37:0/2857635073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1804154d80 con 0x7f1804074cb0 2024-09-21T06:33:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.174+0000 7f1801e00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009980 tx=0x7f17f4033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.175+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4031b50 con 0x7f1804074cb0 2024-09-21T06:33:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.175+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17f4031de0 con 0x7f1804074cb0 2024-09-21T06:33:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.175+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1804155060 con 0x7f1804074cb0 2024-09-21T06:33:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.175+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4033330 con 0x7f1804074cb0 2024-09-21T06:33:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.175+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180409ddb0 con 0x7f1804074cb0 2024-09-21T06:33:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.176+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180415f070 con 0x7f1804074cb0 2024-09-21T06:33:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.177+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17f40334d0 con 0x7f1804074cb0 2024-09-21T06:33:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.179+0000 7f17fa000640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 0x7f17e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.179+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f17f40bbee0 con 0x7f1804074cb0 2024-09-21T06:33:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.179+0000 7f17fbe00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 0x7f17e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.180+0000 7f17fbe00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 0x7f17e007b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1804051160 tx=0x7f17ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.183+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f180415f070 con 0x7f1804074cb0 2024-09-21T06:33:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.454+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7f1804073dd0 con 0x7f1804074cb0 2024-09-21T06:33:39.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.981+0000 7f17fa000640 1 -- 172.21.15.37:0/2857635073 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v304) ==== 157+0+0 (secure 0 0 0) 0x7f17f403f030 con 0x7f1804074cb0 2024-09-21T06:33:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 msgr2=0x7f17e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 0x7f17e007b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1804051160 tx=0x7f17ec009040 comp rx=0 tx=0).stop 2024-09-21T06:33:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 msgr2=0x7f18041571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009980 tx=0x7f17f4033ba0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 shutdown_connections 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f17e0078b90 0x7f17e007b0a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1804157710 0x7f180409e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18040cbc10 0x7f180409e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 --2- 172.21.15.37:0/2857635073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1804074cb0 0x7f18041571d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 >> 172.21.15.37:0/2857635073 conn(0x7f18040cca40 msgr2=0x7f18040cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 shutdown_connections 2024-09-21T06:33:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:39.984+0000 7f1808c00640 1 -- 172.21.15.37:0/2857635073 wait complete. 2024-09-21T06:33:40.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:33:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.241+0000 7f6f75200640 1 -- 172.21.15.37:0/625187140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f54004800 msgr2=0x7f6f54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.241+0000 7f6f75200640 1 --2- 172.21.15.37:0/625187140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f54004800 0x7f6f54004c10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6f700cd8c0 tx=0x7f6f54005830 comp rx=0 tx=0).stop 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.241+0000 7f6f6e800640 1 -- 172.21.15.37:0/625187140 reap_dead start 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.242+0000 7f6f75200640 1 -- 172.21.15.37:0/625187140 shutdown_connections 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.242+0000 7f6f75200640 1 -- 172.21.15.37:0/625187140 >> 172.21.15.37:0/625187140 conn(0x7f6f700cca40 msgr2=0x7f6f700cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.242+0000 7f6f75200640 1 -- 172.21.15.37:0/625187140 shutdown_connections 2024-09-21T06:33:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.242+0000 7f6f75200640 1 -- 172.21.15.37:0/625187140 wait complete. 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.244+0000 7f6f75200640 1 Processor -- start 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.244+0000 7f6f75200640 1 -- start start 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f54004800 0x7f6f700cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 0x7f6f700ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7006ede0 con 0x7f6f7015e3e0 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7006e710 con 0x7f6f701f6540 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f75200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7006ef60 con 0x7f6f54004800 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38502/0 (socket says 172.21.15.37:38502) 2024-09-21T06:33:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 -- 172.21.15.37:0/1232365142 learned_addr learned my addr 172.21.15.37:0/1232365142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f67e00640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 0x7f6f700ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6de00640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f54004800 0x7f6f700cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 -- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f54004800 msgr2=0x7f6f700cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f54004800 0x7f6f700cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 -- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 msgr2=0x7f6f700ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 0x7f6f700ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f6e800640 1 -- 172.21.15.37:0/1232365142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f700cdc30 con 0x7f6f7015e3e0 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.246+0000 7f6f67e00640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 0x7f6f700ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f6e800640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6f68005fd0 tx=0x7f6f6800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004b20 con 0x7f6f7015e3e0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f68004cc0 con 0x7f6f7015e3e0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68003bd0 con 0x7f6f7015e3e0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f700cdf10 con 0x7f6f7015e3e0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.247+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f700ce4d0 con 0x7f6f7015e3e0 2024-09-21T06:33:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.248+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7006ddf0 con 0x7f6f7015e3e0 2024-09-21T06:33:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.250+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f68004030 con 0x7f6f7015e3e0 2024-09-21T06:33:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.251+0000 7f6f66000640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 0x7f6f4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.251+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f6f6809b130 con 0x7f6f7015e3e0 2024-09-21T06:33:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.251+0000 7f6f6de00640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 0x7f6f4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.252+0000 7f6f6de00640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 0x7f6f4c07b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6f7015a730 tx=0x7f6f60006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.255+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f68063630 con 0x7f6f7015e3e0 2024-09-21T06:33:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.452+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f6f701f6950 con 0x7f6f7015e3e0 2024-09-21T06:33:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.452+0000 7f6f66000640 1 -- 172.21.15.37:0/1232365142 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f6f68062f30 con 0x7f6f7015e3e0 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:33:40.454 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:33:40.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:33:40.456 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 msgr2=0x7f6f4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 0x7f6f4c07b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6f7015a730 tx=0x7f6f60006b10 comp rx=0 tx=0).stop 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 msgr2=0x7f6f700cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6f68005fd0 tx=0x7f6f6800f3c0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 shutdown_connections 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c078b90 0x7f6f4c07b0a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015e3e0 0x7f6f700cef30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f701f6540 0x7f6f700ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 --2- 172.21.15.37:0/1232365142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f54004800 0x7f6f700cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 >> 172.21.15.37:0/1232365142 conn(0x7f6f700cca40 msgr2=0x7f6f7015a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 shutdown_connections 2024-09-21T06:33:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.455+0000 7f6f75200640 1 -- 172.21.15.37:0/1232365142 wait complete. 2024-09-21T06:33:40.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:33:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 -- 172.21.15.37:0/1435629135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0155630 msgr2=0x7f31d00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 --2- 172.21.15.37:0/1435629135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0155630 0x7f31d00cecc0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005fd0 tx=0x7f31c802aa90 comp rx=0 tx=0).stop 2024-09-21T06:33:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 -- 172.21.15.37:0/1435629135 shutdown_connections 2024-09-21T06:33:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 --2- 172.21.15.37:0/1435629135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0155630 0x7f31d00cecc0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 --2- 172.21.15.37:0/1435629135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 0x7f31d0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 -- 172.21.15.37:0/1435629135 >> 172.21.15.37:0/1435629135 conn(0x7f31d00d4360 msgr2=0x7f31d00d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 -- 172.21.15.37:0/1435629135 shutdown_connections 2024-09-21T06:33:40.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.711+0000 7f31d4c00640 1 -- 172.21.15.37:0/1435629135 wait complete. 2024-09-21T06:33:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.713+0000 7f31d4c00640 1 Processor -- start 2024-09-21T06:33:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.713+0000 7f31d4c00640 1 -- start start 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 0x7f31d015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0155630 0x7f31d01407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31d00cf790 con 0x7f31d0162890 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31d00cf610 con 0x7f31d0155630 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31d4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f31d00cf910 con 0x7f31d00d2760 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38526/0 (socket says 172.21.15.37:38526) 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31cfe00640 1 -- 172.21.15.37:0/2751748328 learned_addr learned my addr 172.21.15.37:0/2751748328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31cf400640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 0x7f31d015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.715+0000 7f31c7e00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0155630 0x7f31d01407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 -- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 msgr2=0x7f31d015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 0x7f31d015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 -- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0155630 msgr2=0x7f31d01407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0155630 0x7f31d01407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 -- 172.21.15.37:0/2751748328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d00ca190 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31cfe00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009d90 tx=0x7f31c8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8044070 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31c802e7f0 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80044b0 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d00ca470 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.716+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d01402a0 con 0x7f31d0162890 2024-09-21T06:33:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.718+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d00cfea0 con 0x7f31d0162890 2024-09-21T06:33:40.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.719+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31c8047070 con 0x7f31d0162890 2024-09-21T06:33:40.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.720+0000 7f31c7400640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 0x7f31ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.720+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f31c80bbbc0 con 0x7f31d0162890 2024-09-21T06:33:40.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.720+0000 7f31cf400640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 0x7f31ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.721+0000 7f31cf400640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 0x7f31ac07b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f31c000bb50 tx=0x7f31c000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.724+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31c8032030 con 0x7f31d0162890 2024-09-21T06:33:40.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.922+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f31d00c9f20 con 0x7f31d0162890 2024-09-21T06:33:40.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.927+0000 7f31c7400640 1 -- 172.21.15.37:0/2751748328 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f31c80839d0 con 0x7f31d0162890 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 msgr2=0x7f31ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 0x7f31ac07b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f31c000bb50 tx=0x7f31c000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 msgr2=0x7f31d0140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009d90 tx=0x7f31c8003f10 comp rx=0 tx=0).stop 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 shutdown_connections 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f31ac078b90 0x7f31ac07b0a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d0162890 0x7f31d0140d00 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0155630 0x7f31d01407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 --2- 172.21.15.37:0/2751748328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d00d2760 0x7f31d015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 >> 172.21.15.37:0/2751748328 conn(0x7f31d00d4360 msgr2=0x7f31d015fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.929+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 shutdown_connections 2024-09-21T06:33:40.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:40.930+0000 7f31d4c00640 1 -- 172.21.15.37:0/2751748328 wait complete. 2024-09-21T06:33:40.952 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:33:40.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:33:41.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:33:41.137 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:33:41.137 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:41.137 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:41.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:33:41.173 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:33:41.173 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:41.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:33:41.210 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:41.210 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:41.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:33:41.249 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:33:41.249 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:41.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:33:41.287 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:33:41.288 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.514+0000 7fd237600640 1 -- 172.21.15.37:0/3734337780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd23006e500 msgr2=0x7fd2300cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.514+0000 7fd237600640 1 --2- 172.21.15.37:0/3734337780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd23006e500 0x7fd2300cc020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd228005fd0 tx=0x7fd22802aaa0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 -- 172.21.15.37:0/3734337780 shutdown_connections 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 --2- 172.21.15.37:0/3734337780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd23006e500 0x7fd2300cc020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 --2- 172.21.15.37:0/3734337780 >> v2:172.21.15.150:3300/0 conn(0x7fd230074cb0 0x7fd230073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 -- 172.21.15.37:0/3734337780 >> 172.21.15.37:0/3734337780 conn(0x7fd2300cca40 msgr2=0x7fd2300cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 -- 172.21.15.37:0/3734337780 shutdown_connections 2024-09-21T06:33:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.515+0000 7fd237600640 1 -- 172.21.15.37:0/3734337780 wait complete. 2024-09-21T06:33:41.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.517+0000 7fd237600640 1 Processor -- start 2024-09-21T06:33:41.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.517+0000 7fd237600640 1 -- start start 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.518+0000 7fd237600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd230074cb0 0x7fd2301592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.518+0000 7fd237600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2300cbc10 0x7fd23009cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd237600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd237600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006f3a0 con 0x7fd230159aa0 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd237600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006f220 con 0x7fd230074cb0 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd237600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd23006f520 con 0x7fd2300cbc10 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38530/0 (socket says 172.21.15.37:38530) 2024-09-21T06:33:41.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 -- 172.21.15.37:0/231724817 learned_addr learned my addr 172.21.15.37:0/231724817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:41.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd22fe00640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2300cbc10 0x7fd23009cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:41.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd234c00640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd230074cb0 0x7fd2301592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 -- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2300cbc10 msgr2=0x7fd23009cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2300cbc10 0x7fd23009cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 -- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd230074cb0 msgr2=0x7fd2301592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd230074cb0 0x7fd2301592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.519+0000 7fd235600640 1 -- 172.21.15.37:0/231724817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2301597e0 con 0x7fd230159aa0 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd235600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd228009d90 tx=0x7fd228003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228044070 con 0x7fd230159aa0 2024-09-21T06:33:41.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2301e5eb0 con 0x7fd230159aa0 2024-09-21T06:33:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd228003e00 con 0x7fd230159aa0 2024-09-21T06:33:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2280044b0 con 0x7fd230159aa0 2024-09-21T06:33:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.520+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23009b550 con 0x7fd230159aa0 2024-09-21T06:33:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.521+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23006fa40 con 0x7fd230159aa0 2024-09-21T06:33:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.523+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd228047070 con 0x7fd230159aa0 2024-09-21T06:33:41.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.523+0000 7fd22cc00640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 0x7fd21007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.524+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7fd2280bbb20 con 0x7fd230159aa0 2024-09-21T06:33:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.524+0000 7fd234c00640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 0x7fd21007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:41.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.525+0000 7fd234c00640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 0x7fd21007b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd2240095f0 tx=0x7fd224005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.528+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd228032030 con 0x7fd230159aa0 2024-09-21T06:33:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.763+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2301e5c40 con 0x7fd230159aa0 2024-09-21T06:33:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.764+0000 7fd22cc00640 1 -- 172.21.15.37:0/231724817 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+3917 (secure 0 0 0) 0x7fd2280098b0 con 0x7fd230159aa0 2024-09-21T06:33:41.765 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:41.765 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"btime":"2024-09-21T06:33:33:509768+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6428,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3026866329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3026866329},{"type":"v1","addr":"172.21.15.150:6837","nonce":3026866329}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":6631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3775852903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3775852903},{"type":"v1","addr":"172.21.15.150:6835","nonce":3775852903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7018,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1220006132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1220006132},{"type":"v1","addr":"172.21.15.37:6835","nonce":1220006132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3642987931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3642987931},{"type":"v1","addr":"172.21.15.37:6834","nonce":3642987931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[]} 2024-09-21T06:33:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 82 2024-09-21T06:33:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 msgr2=0x7fd21007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 0x7fd21007b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd2240095f0 tx=0x7fd224005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 msgr2=0x7fd23009d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd228009d90 tx=0x7fd228003c10 comp rx=0 tx=0).stop 2024-09-21T06:33:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 shutdown_connections 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd210078b90 0x7fd21007b0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd230159aa0 0x7fd23009d4c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2300cbc10 0x7fd23009cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 --2- 172.21.15.37:0/231724817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd230074cb0 0x7fd2301592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 >> 172.21.15.37:0/231724817 conn(0x7fd2300cca40 msgr2=0x7fd2300a0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 shutdown_connections 2024-09-21T06:33:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:41.766+0000 7fd237600640 1 -- 172.21.15.37:0/231724817 wait complete. 2024-09-21T06:33:41.793 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:33:41.793 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:33:41.793 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:33:41.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:33:41.793 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:33:41.794 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:33:41.794 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:33:41.794 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:33:41.794 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:33:41.794 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:33:41.794 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:33:41.794 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:33:41.795 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:33:41.795 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:33:41.795 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:33:41.795 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:33:41.821 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:33:41.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:33:41.822 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:33:41.822 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:33:41.823 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:33:41.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:33:41.825 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:33:41.825 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:33:41.867 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:33:41.868 DEBUG:teuthology.parallel:result is None 2024-09-21T06:33:41.868 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:33:41.868 DEBUG:teuthology.parallel:result is None 2024-09-21T06:33:41.870 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:33:41.870 DEBUG:teuthology.parallel:result is None 2024-09-21T06:33:41.871 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:33:41.871 DEBUG:teuthology.parallel:result is None 2024-09-21T06:33:41.872 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:33:42.068 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:33:42.070 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:33:42.070 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:33:42.072 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:33:42.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 -- 172.21.15.37:0/3429510192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40cd050 msgr2=0x7f4be40b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 --2- 172.21.15.37:0/3429510192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40cd050 0x7f4be40b5030 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4be40c58c0 tx=0x7f4bd402a600 comp rx=0 tx=0).stop 2024-09-21T06:33:42.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 -- 172.21.15.37:0/3429510192 shutdown_connections 2024-09-21T06:33:42.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 --2- 172.21.15.37:0/3429510192 >> v2:172.21.15.150:3300/0 conn(0x7f4be415c8c0 0x7f4be40b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 --2- 172.21.15.37:0/3429510192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40cd050 0x7f4be40b5030 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 -- 172.21.15.37:0/3429510192 >> 172.21.15.37:0/3429510192 conn(0x7f4be40d2620 msgr2=0x7f4be40b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:42.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 -- 172.21.15.37:0/3429510192 shutdown_connections 2024-09-21T06:33:42.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.107+0000 7f4be9c00640 1 -- 172.21.15.37:0/3429510192 wait complete. 2024-09-21T06:33:42.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.109+0000 7f4be9c00640 1 Processor -- start 2024-09-21T06:33:42.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.109+0000 7f4be9c00640 1 -- start start 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4be40cd050 0x7f4be41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4be415c8c0 0x7f4be41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4be40719a0 con 0x7f4be40b5ab0 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4071820 con 0x7f4be40cd050 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4be40b5ec0 con 0x7f4be415c8c0 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38572/0 (socket says 172.21.15.37:38572) 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be3e00640 1 -- 172.21.15.37:0/2809065399 learned_addr learned my addr 172.21.15.37:0/2809065399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4bdb400640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4be40cd050 0x7f4be41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.111+0000 7f4be9200640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4be415c8c0 0x7f4be41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 -- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4be415c8c0 msgr2=0x7f4be41589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4be415c8c0 0x7f4be41589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 -- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4be40cd050 msgr2=0x7f4be41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4be40cd050 0x7f4be41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 -- 172.21.15.37:0/2809065399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be40c6980 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be3e00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4034040 tx=0x7f4bd4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd403e030 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd4003d10 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4035040 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.112+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be40c5d70 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.113+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be406fb20 con 0x7f4be40b5ab0 2024-09-21T06:33:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.114+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be4075780 con 0x7f4be40b5ab0 2024-09-21T06:33:42.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.115+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4bd4041030 con 0x7f4be40b5ab0 2024-09-21T06:33:42.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.116+0000 7f4be0c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 0x7f4bc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.116+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f4bd40bb430 con 0x7f4be40b5ab0 2024-09-21T06:33:42.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.116+0000 7f4bdb400640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 0x7f4bc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.117+0000 7f4bdb400640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 0x7f4bc007b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009b30 tx=0x7f4bcc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.121+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4be40c6980 con 0x7f4be40b5ab0 2024-09-21T06:33:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:42.331+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4be40c6490 con 0x7f4be40b5ab0 2024-09-21T06:33:43.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.002+0000 7f4be0c00640 1 -- 172.21.15.37:0/2809065399 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) ==== 97+0+0 (secure 0 0 0) 0x7f4bd402ae20 con 0x7f4be40b5ab0 2024-09-21T06:33:43.004 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:33:43.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 msgr2=0x7f4bc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 0x7f4bc007b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009b30 tx=0x7f4bcc005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:43.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 msgr2=0x7f4be40c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4034040 tx=0x7f4bd4003820 comp rx=0 tx=0).stop 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 shutdown_connections 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4bc0078b90 0x7f4bc007b0a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4be415c8c0 0x7f4be41589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4be40cd050 0x7f4be41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 --2- 172.21.15.37:0/2809065399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4be40b5ab0 0x7f4be40c80c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 >> 172.21.15.37:0/2809065399 conn(0x7f4be40d2620 msgr2=0x7f4be415bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 shutdown_connections 2024-09-21T06:33:43.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.007+0000 7f4be9c00640 1 -- 172.21.15.37:0/2809065399 wait complete. 2024-09-21T06:33:43.042 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 -- 172.21.15.37:0/2368040457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e4806e500 msgr2=0x7f0e480cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 --2- 172.21.15.37:0/2368040457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e4806e500 0x7f0e480cc020 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005fd0 tx=0x7f0e4002aa90 comp rx=0 tx=0).stop 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 -- 172.21.15.37:0/2368040457 shutdown_connections 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 --2- 172.21.15.37:0/2368040457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e4806e500 0x7f0e480cc020 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 --2- 172.21.15.37:0/2368040457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e48074cb0 0x7f0e48073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 -- 172.21.15.37:0/2368040457 >> 172.21.15.37:0/2368040457 conn(0x7f0e480cca40 msgr2=0x7f0e480cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 -- 172.21.15.37:0/2368040457 shutdown_connections 2024-09-21T06:33:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.270+0000 7f0e4f000640 1 -- 172.21.15.37:0/2368040457 wait complete. 2024-09-21T06:33:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.273+0000 7f0e4f000640 1 Processor -- start 2024-09-21T06:33:43.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.273+0000 7f0e4f000640 1 -- start start 2024-09-21T06:33:43.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 0x7f0e480b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e480cbc10 0x7f0e481f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4806f310 con 0x7f0e481e6830 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4806f190 con 0x7f0e48074cb0 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.274+0000 7f0e4f000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4806f490 con 0x7f0e480cbc10 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38590/0 (socket says 172.21.15.37:38590) 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e47e00640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 0x7f0e480b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 -- 172.21.15.37:0/1941076579 learned_addr learned my addr 172.21.15.37:0/1941076579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e47400640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e480cbc10 0x7f0e481f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 -- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e480cbc10 msgr2=0x7f0e481f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e480cbc10 0x7f0e481f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 -- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 msgr2=0x7f0e480b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 0x7f0e480b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 -- 172.21.15.37:0/1941076579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480cdc30 con 0x7f0e481e6830 2024-09-21T06:33:43.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e47e00640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 0x7f0e480b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:33:43.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.275+0000 7f0e4d000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009d90 tx=0x7f0e40003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.276+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40044070 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.276+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e40004250 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.276+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40004660 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.276+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e480cdf10 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.276+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e480ce4d0 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.277+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e481e7290 con 0x7f0e481e6830 2024-09-21T06:33:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.278+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e40047070 con 0x7f0e481e6830 2024-09-21T06:33:43.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.279+0000 7f0e3fe00640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 0x7f0e2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.279+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f0e400bbad0 con 0x7f0e481e6830 2024-09-21T06:33:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.280+0000 7f0e47e00640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 0x7f0e2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.281+0000 7f0e47e00640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 0x7f0e2807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0e4815a730 tx=0x7f0e38009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.284+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e40032030 con 0x7f0e481e6830 2024-09-21T06:33:43.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.554+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0e4815f870 con 0x7f0e481e6830 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.555+0000 7f0e3fe00640 1 -- 172.21.15.37:0/1941076579 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f0e40083b00 con 0x7f0e481e6830 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:33:43.556 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:33:43.557 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.558 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.560 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:33:43.562 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:33:43.564 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:33:43.565 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:33:43.566 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.567 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:33:43.569 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 msgr2=0x7f0e2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 0x7f0e2807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0e4815a730 tx=0x7f0e38009060 comp rx=0 tx=0).stop 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 msgr2=0x7f0e481e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009d90 tx=0x7f0e40003d00 comp rx=0 tx=0).stop 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 shutdown_connections 2024-09-21T06:33:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e28078b90 0x7f0e2807b0a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e481e6830 0x7f0e481e6c40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e480cbc10 0x7f0e481f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 --2- 172.21.15.37:0/1941076579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e48074cb0 0x7f0e480b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 >> 172.21.15.37:0/1941076579 conn(0x7f0e480cca40 msgr2=0x7f0e4809acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 shutdown_connections 2024-09-21T06:33:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.557+0000 7f0e4f000640 1 -- 172.21.15.37:0/1941076579 wait complete. 2024-09-21T06:33:43.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:33:43.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 -- 172.21.15.37:0/555314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e40ccd60 msgr2=0x7ff5e40cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 --2- 172.21.15.37:0/555314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e40ccd60 0x7ff5e40cddc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40723d0 tx=0x7ff5dc02aa30 comp rx=0 tx=0).stop 2024-09-21T06:33:43.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 -- 172.21.15.37:0/555314960 shutdown_connections 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 --2- 172.21.15.37:0/555314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e40ccd60 0x7ff5e40cddc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 --2- 172.21.15.37:0/555314960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e415ede0 0x7ff5e40cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 --2- 172.21.15.37:0/555314960 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415b820 0x7ff5e415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 -- 172.21.15.37:0/555314960 >> 172.21.15.37:0/555314960 conn(0x7ff5e406f940 msgr2=0x7ff5e40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 -- 172.21.15.37:0/555314960 shutdown_connections 2024-09-21T06:33:43.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.807+0000 7ff5ea800640 1 -- 172.21.15.37:0/555314960 wait complete. 2024-09-21T06:33:43.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.809+0000 7ff5ea800640 1 Processor -- start 2024-09-21T06:33:43.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.809+0000 7ff5ea800640 1 -- start start 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 0x7ff5e4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 0x7ff5e414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4070cf0 con 0x7ff5e415b820 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4072d10 con 0x7ff5e415ede0 2024-09-21T06:33:43.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5ea800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e4070e70 con 0x7ff5e40ccd60 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38606/0 (socket says 172.21.15.37:38606) 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 -- 172.21.15.37:0/3108810729 learned_addr learned my addr 172.21.15.37:0/3108810729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e9400640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 0x7ff5e4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e9e00640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 0x7ff5e414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 -- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 msgr2=0x7ff5e4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 0x7ff5e4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 -- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 msgr2=0x7ff5e414f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 0x7ff5e414f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e8a00640 1 -- 172.21.15.37:0/3108810729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e4146e70 con 0x7ff5e415b820 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e9400640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 0x7ff5e4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:43.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.811+0000 7ff5e9e00640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 0x7ff5e414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5e8a00640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00095a0 tx=0x7ff5d0009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0018020 con 0x7ff5e415b820 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5d0014910 con 0x7ff5e415b820 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e4156b00 con 0x7ff5e415b820 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0014d20 con 0x7ff5e415b820 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.812+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e41570c0 con 0x7ff5e415b820 2024-09-21T06:33:43.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.813+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e40cd350 con 0x7ff5e415b820 2024-09-21T06:33:43.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.815+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5d001b020 con 0x7ff5e415b820 2024-09-21T06:33:43.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.815+0000 7ff5e1600640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 0x7ff5c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:43.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.816+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7ff5d009af50 con 0x7ff5e415b820 2024-09-21T06:33:43.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.816+0000 7ff5e9400640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 0x7ff5c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:43.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.817+0000 7ff5e9400640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 0x7ff5c407b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009b00 tx=0x7ff5d8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:43.820+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5e40cd350 con 0x7ff5e415b820 2024-09-21T06:33:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.022+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff5e415e8a0 con 0x7ff5e415b820 2024-09-21T06:33:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.022+0000 7ff5e1600640 1 -- 172.21.15.37:0/3108810729 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) ==== 84+0+65 (secure 0 0 0) 0x7ff5d0022020 con 0x7ff5e415b820 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:33:44.024 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 msgr2=0x7ff5c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 0x7ff5c407b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009b00 tx=0x7ff5d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 msgr2=0x7ff5e4146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00095a0 tx=0x7ff5d0009a60 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 shutdown_connections 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5c4078b90 0x7ff5c407b0a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5e415b820 0x7ff5e4146930 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ede0 0x7ff5e414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 --2- 172.21.15.37:0/3108810729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40ccd60 0x7ff5e4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 >> 172.21.15.37:0/3108810729 conn(0x7ff5e406f940 msgr2=0x7ff5e415d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:44.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 shutdown_connections 2024-09-21T06:33:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.025+0000 7ff5ea800640 1 -- 172.21.15.37:0/3108810729 wait complete. 2024-09-21T06:33:44.048 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:33:44.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:33:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 -- 172.21.15.37:0/626692078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cf370 msgr2=0x7f0e680717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 --2- 172.21.15.37:0/626692078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cf370 0x7f0e680717b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0e680d1830 tx=0x7f0e5402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 -- 172.21.15.37:0/626692078 shutdown_connections 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 --2- 172.21.15.37:0/626692078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cf370 0x7f0e680717b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 --2- 172.21.15.37:0/626692078 >> v2:172.21.15.150:3300/0 conn(0x7f0e6815b920 0x7f0e680ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 -- 172.21.15.37:0/626692078 >> 172.21.15.37:0/626692078 conn(0x7f0e680d1210 msgr2=0x7f0e680cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 -- 172.21.15.37:0/626692078 shutdown_connections 2024-09-21T06:33:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.280+0000 7f0e6f000640 1 -- 172.21.15.37:0/626692078 wait complete. 2024-09-21T06:33:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.282+0000 7f0e6f000640 1 Processor -- start 2024-09-21T06:33:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.283+0000 7f0e6f000640 1 -- start start 2024-09-21T06:33:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 0x7f0e68157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 0x7f0e68156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b3ee0 con 0x7f0e680cd8e0 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b3d60 con 0x7f0e6815b920 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6f000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680b4060 con 0x7f0e680cf370 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39242/0 (socket says 172.21.15.37:39242) 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.284+0000 7f0e6dc00640 1 -- 172.21.15.37:0/4253376902 learned_addr learned my addr 172.21.15.37:0/4253376902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6d200640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 0x7f0e68157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6e600640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 0x7f0e68156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 -- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 msgr2=0x7f0e68157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 0x7f0e68157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 -- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 msgr2=0x7f0e68156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 0x7f0e68156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 -- 172.21.15.37:0/4253376902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6814a0a0 con 0x7f0e680cd8e0 2024-09-21T06:33:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6e600640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 0x7f0e68156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6d200640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 0x7f0e68157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6dc00640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d740 tx=0x7f0e5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c017070 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e5c009cc0 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0106c0 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.285+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6814a380 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.286+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6814a940 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.287+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e680b4580 con 0x7f0e680cd8e0 2024-09-21T06:33:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.288+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e5c01a070 con 0x7f0e680cd8e0 2024-09-21T06:33:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.289+0000 7f0e65600640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 0x7f0e4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.289+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f0e5c09ad50 con 0x7f0e680cd8e0 2024-09-21T06:33:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.290+0000 7f0e6d200640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 0x7f0e4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.290+0000 7f0e6d200640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 0x7f0e4807b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e54033000 tx=0x7f0e54004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.293+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e5c0632b0 con 0x7f0e680cd8e0 2024-09-21T06:33:44.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:44.490+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0e68069d30 con 0x7f0e680cd8e0 2024-09-21T06:33:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.019+0000 7f0e65600640 1 -- 172.21.15.37:0/4253376902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) ==== 138+0+0 (secure 0 0 0) 0x7f0e5c062bb0 con 0x7f0e680cd8e0 2024-09-21T06:33:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:33:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 msgr2=0x7f0e4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 0x7f0e4807b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e54033000 tx=0x7f0e54004730 comp rx=0 tx=0).stop 2024-09-21T06:33:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 msgr2=0x7f0e6814b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d740 tx=0x7f0e5c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 shutdown_connections 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e48078b90 0x7f0e4807b0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6815b920 0x7f0e68156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e680cf370 0x7f0e68157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 --2- 172.21.15.37:0/4253376902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e680cd8e0 0x7f0e6814b140 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 >> 172.21.15.37:0/4253376902 conn(0x7f0e680d1210 msgr2=0x7f0e68155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 shutdown_connections 2024-09-21T06:33:45.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.022+0000 7f0e6f000640 1 -- 172.21.15.37:0/4253376902 wait complete. 2024-09-21T06:33:45.058 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:33:45.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.292+0000 7f8e81a00640 1 -- 172.21.15.37:0/2572450419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c0ccf20 msgr2=0x7f8e7c0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.292+0000 7f8e81a00640 1 --2- 172.21.15.37:0/2572450419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c0ccf20 0x7f8e7c0b4990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005fd0 tx=0x7f8e6402a540 comp rx=0 tx=0).stop 2024-09-21T06:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 -- 172.21.15.37:0/2572450419 shutdown_connections 2024-09-21T06:33:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 --2- 172.21.15.37:0/2572450419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b4ed0 0x7f8e7c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 --2- 172.21.15.37:0/2572450419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c0ccf20 0x7f8e7c0b4990 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 -- 172.21.15.37:0/2572450419 >> 172.21.15.37:0/2572450419 conn(0x7f8e7c06f940 msgr2=0x7f8e7c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 -- 172.21.15.37:0/2572450419 shutdown_connections 2024-09-21T06:33:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.293+0000 7f8e81a00640 1 -- 172.21.15.37:0/2572450419 wait complete. 2024-09-21T06:33:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.295+0000 7f8e81a00640 1 Processor -- start 2024-09-21T06:33:45.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.295+0000 7f8e81a00640 1 -- start start 2024-09-21T06:33:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.296+0000 7f8e81a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0b4ed0 0x7f8e7c1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.296+0000 7f8e81a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0cddc0 0x7f8e7c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.296+0000 7f8e81a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c0ce2e0 con 0x7f8e7c153e90 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c06e5f0 con 0x7f8e7c0b4ed0 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c15d460 con 0x7f8e7c0cddc0 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39256/0 (socket says 172.21.15.37:39256) 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 -- 172.21.15.37:0/264548520 learned_addr learned my addr 172.21.15.37:0/264548520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e7be00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0b4ed0 0x7f8e7c1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e73e00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0cddc0 0x7f8e7c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 -- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0cddc0 msgr2=0x7f8e7c1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0cddc0 0x7f8e7c1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 -- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0b4ed0 msgr2=0x7f8e7c1f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0b4ed0 0x7f8e7c1f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.297+0000 7f8e81000640 1 -- 172.21.15.37:0/264548520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c145460 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e81000640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9c0 tx=0x7f8e7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74004d20 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e7400cb20 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74005020 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c155920 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.298+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c155ee0 con 0x7f8e7c153e90 2024-09-21T06:33:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.299+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c15d8e0 con 0x7f8e7c153e90 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.301+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e740051c0 con 0x7f8e7c153e90 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.302+0000 7f8e78c00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.303+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5619+0+0 (secure 0 0 0) 0x7f8e7409b930 con 0x7f8e7c153e90 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.303+0000 7f8e7be00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.304+0000 7f8e7be00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c001720 tx=0x7f8e6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.306+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e74016030 con 0x7f8e7c153e90 2024-09-21T06:33:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:45.506+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8e7c0cf310 con 0x7f8e7c153e90 2024-09-21T06:33:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.028+0000 7f8e78c00640 1 -- 172.21.15.37:0/264548520 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7f8e740637e0 con 0x7f8e7c153e90 2024-09-21T06:33:46.030 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:33:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 msgr2=0x7f8e5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c001720 tx=0x7f8e6c009040 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 msgr2=0x7f8e7c0afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9c0 tx=0x7f8e7400ee80 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 shutdown_connections 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e7c153e90 0x7f8e7c0afc60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0cddc0 0x7f8e7c1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 --2- 172.21.15.37:0/264548520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0b4ed0 0x7f8e7c1f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 >> 172.21.15.37:0/264548520 conn(0x7f8e7c06f940 msgr2=0x7f8e7c0ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 shutdown_connections 2024-09-21T06:33:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.031+0000 7f8e81a00640 1 -- 172.21.15.37:0/264548520 wait complete. 2024-09-21T06:33:46.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 -- 172.21.15.37:0/2263954326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 msgr2=0x7f41c8071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 --2- 172.21.15.37:0/2263954326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c8071570 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005fd0 tx=0x7f41c002aa10 comp rx=0 tx=0).stop 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 -- 172.21.15.37:0/2263954326 shutdown_connections 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 --2- 172.21.15.37:0/2263954326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c8071570 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 -- 172.21.15.37:0/2263954326 >> 172.21.15.37:0/2263954326 conn(0x7f41c80d1020 msgr2=0x7f41c80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 -- 172.21.15.37:0/2263954326 shutdown_connections 2024-09-21T06:33:46.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.306+0000 7f41cec00640 1 -- 172.21.15.37:0/2263954326 wait complete. 2024-09-21T06:33:46.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.309+0000 7f41cec00640 1 Processor -- start 2024-09-21T06:33:46.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.309+0000 7f41cec00640 1 -- start start 2024-09-21T06:33:46.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 0x7f41c80ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c8152450 0x7f41c81f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b42e0 con 0x7f41c8071160 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b4160 con 0x7f41c8152450 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cec00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f41c80b4460 con 0x7f41c8072320 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39274/0 (socket says 172.21.15.37:39274) 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.310+0000 7f41cd800640 1 -- 172.21.15.37:0/1380471288 learned_addr learned my addr 172.21.15.37:0/1380471288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41ce200640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c8152450 0x7f41c81f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cce00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 0x7f41c80ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 -- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 msgr2=0x7f41c80ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 0x7f41c80ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 -- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c8152450 msgr2=0x7f41c81f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c8152450 0x7f41c81f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 -- 172.21.15.37:0/1380471288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c80ad560 con 0x7f41c8071160 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cce00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 0x7f41c80ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41cd800640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c400 tx=0x7f41bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc017070 con 0x7f41c8071160 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41bc0053c0 con 0x7f41c8071160 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.311+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc003850 con 0x7f41c8071160 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.312+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c80ad840 con 0x7f41c8071160 2024-09-21T06:33:46.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.312+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c80ade00 con 0x7f41c8071160 2024-09-21T06:33:46.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.313+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8071570 con 0x7f41c8071160 2024-09-21T06:33:46.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.314+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41bc01a070 con 0x7f41c8071160 2024-09-21T06:33:46.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.315+0000 7f41c5600640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 0x7f41a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.315+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6001+0+0 (secure 0 0 0) 0x7f41bc09a1d0 con 0x7f41c8071160 2024-09-21T06:33:46.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.316+0000 7f41cce00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 0x7f41a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.316+0000 7f41cce00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 0x7f41a807b0d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41b4007fd0 tx=0x7f41b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:46.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.320+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41c8071ab0 con 0x7f41c8071160 2024-09-21T06:33:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:46.561+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f41c81512d0 con 0x7f41c8071160 2024-09-21T06:33:47.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.031+0000 7f41c5600640 1 -- 172.21.15.37:0/1380471288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v85) ==== 174+0+0 (secure 0 0 0) 0x7f41bc061e80 con 0x7f41c8071160 2024-09-21T06:33:47.032 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 13 and data pool 14 2024-09-21T06:33:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 msgr2=0x7f41a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 0x7f41a807b0d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41b4007fd0 tx=0x7f41b4009040 comp rx=0 tx=0).stop 2024-09-21T06:33:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 msgr2=0x7f41c81553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c400 tx=0x7f41bc00c8c0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 shutdown_connections 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f41a8078bc0 0x7f41a807b0d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c8152450 0x7f41c81f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c8072320 0x7f41c80ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 --2- 172.21.15.37:0/1380471288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8071160 0x7f41c81553c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 >> 172.21.15.37:0/1380471288 conn(0x7f41c80d1020 msgr2=0x7f41c80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 shutdown_connections 2024-09-21T06:33:47.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.034+0000 7f41cec00640 1 -- 172.21.15.37:0/1380471288 wait complete. 2024-09-21T06:33:47.087 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 -- 172.21.15.37:0/1412935767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f8004800 msgr2=0x7f76f8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 --2- 172.21.15.37:0/1412935767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f8004800 0x7f76f8004c10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f76fc001690 tx=0x7f76f8005830 comp rx=0 tx=0).stop 2024-09-21T06:33:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 -- 172.21.15.37:0/1412935767 shutdown_connections 2024-09-21T06:33:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 --2- 172.21.15.37:0/1412935767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f8004800 0x7f76f8004c10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 -- 172.21.15.37:0/1412935767 >> 172.21.15.37:0/1412935767 conn(0x7f77140d4360 msgr2=0x7f77140d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 -- 172.21.15.37:0/1412935767 shutdown_connections 2024-09-21T06:33:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.315+0000 7f7718c00640 1 -- 172.21.15.37:0/1412935767 wait complete. 2024-09-21T06:33:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.317+0000 7f7718c00640 1 Processor -- start 2024-09-21T06:33:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.317+0000 7f7718c00640 1 -- start start 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f8004800 0x7f7714157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77141f54a0 0x7f7714155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77140cf950 con 0x7f77141f5090 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77140cf7d0 con 0x7f76f8004800 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7718c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f77140cfad0 con 0x7f77141f54a0 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f770aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f770aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39294/0 (socket says 172.21.15.37:39294) 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f770aa00640 1 -- 172.21.15.37:0/4147412555 learned_addr learned my addr 172.21.15.37:0/4147412555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7713400640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f8004800 0x7f7714157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f7713e00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77141f54a0 0x7f7714155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f770aa00640 1 -- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77141f54a0 msgr2=0x7f7714155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.319+0000 7f770aa00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77141f54a0 0x7f7714155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770aa00640 1 -- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f8004800 msgr2=0x7f7714157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770aa00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f8004800 0x7f7714157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770aa00640 1 -- 172.21.15.37:0/4147412555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7714156b40 con 0x7f77141f5090 2024-09-21T06:33:47.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770aa00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f76fc02a8d0 tx=0x7f76fc0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc003a00 con 0x7f77141f5090 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76fc039070 con 0x7f77141f5090 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc033420 con 0x7f77141f5090 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7714156e20 con 0x7f77141f5090 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.320+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141573e0 con 0x7f77141f5090 2024-09-21T06:33:47.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.321+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77140cfe80 con 0x7f77141f5090 2024-09-21T06:33:47.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.323+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76fc044020 con 0x7f77141f5090 2024-09-21T06:33:47.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.324+0000 7f770be00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.324+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f76fc0ba6e0 con 0x7f77141f5090 2024-09-21T06:33:47.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.324+0000 7f7713400640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.325+0000 7f7713400640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 0x7f76f007b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f771406b880 tx=0x7f77040067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.328+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76fc082330 con 0x7f77141f5090 2024-09-21T06:33:47.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.528+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77141568d0 con 0x7f77141f5090 2024-09-21T06:33:47.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.529+0000 7f770be00640 1 -- 172.21.15.37:0/4147412555 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17018 (secure 0 0 0) 0x7f76fc086d60 con 0x7f77141f5090 2024-09-21T06:33:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:47.531 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:46.562699+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:33:44.503000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T06:33:45.519092+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 msgr2=0x7f76f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 0x7f76f007b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f771406b880 tx=0x7f77040067a0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 msgr2=0x7f7714145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f76fc02a8d0 tx=0x7f76fc0063a0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 shutdown_connections 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77141f5090 0x7f7714145f30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77141f54a0 0x7f7714155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 --2- 172.21.15.37:0/4147412555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f8004800 0x7f7714157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 >> 172.21.15.37:0/4147412555 conn(0x7f77140d4360 msgr2=0x7f77140b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 shutdown_connections 2024-09-21T06:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.532+0000 7f7718c00640 1 -- 172.21.15.37:0/4147412555 wait complete. 2024-09-21T06:33:47.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 -- 172.21.15.37:0/1074576388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f687c0043a0 msgr2=0x7f687c0048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 --2- 172.21.15.37:0/1074576388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f687c0043a0 0x7f687c0048c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6880009b50 tx=0x7f687c005830 comp rx=0 tx=0).stop 2024-09-21T06:33:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f6897e00640 1 -- 172.21.15.37:0/1074576388 reap_dead start 2024-09-21T06:33:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 -- 172.21.15.37:0/1074576388 shutdown_connections 2024-09-21T06:33:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 -- 172.21.15.37:0/1074576388 >> 172.21.15.37:0/1074576388 conn(0x7f68980d4360 msgr2=0x7f68980d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 -- 172.21.15.37:0/1074576388 shutdown_connections 2024-09-21T06:33:47.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.781+0000 7f689cc00640 1 -- 172.21.15.37:0/1074576388 wait complete. 2024-09-21T06:33:47.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.783+0000 7f689cc00640 1 Processor -- start 2024-09-21T06:33:47.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.783+0000 7f689cc00640 1 -- start start 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f687c0043a0 0x7f68981616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68980ca540 0x7f68981f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f689815e970 con 0x7f68980cd5e0 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f689815e7f0 con 0x7f68980ca540 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f689cc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f68980abd00 con 0x7f687c0043a0 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f6897e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f6897e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39318/0 (socket says 172.21.15.37:39318) 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f6897e00640 1 -- 172.21.15.37:0/262658118 learned_addr learned my addr 172.21.15.37:0/262658118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f6897400640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f687c0043a0 0x7f68981616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.785+0000 7f688ea00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68980ca540 0x7f68981f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 -- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f687c0043a0 msgr2=0x7f68981616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f687c0043a0 0x7f68981616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 -- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68980ca540 msgr2=0x7f68981f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68980ca540 0x7f68981f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 -- 172.21.15.37:0/262658118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980cdf30 con 0x7f68980cd5e0 2024-09-21T06:33:47.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f6897e00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68980d0c30 tx=0x7f6890006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011900 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6890018070 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011d80 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68981e90a0 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.786+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981e82e0 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.787+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68981f92c0 con 0x7f68980cd5e0 2024-09-21T06:33:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.789+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f689001d460 con 0x7f68980cd5e0 2024-09-21T06:33:47.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.790+0000 7f688fe00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 0x7f687407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.790+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f689009b750 con 0x7f68980cd5e0 2024-09-21T06:33:47.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.790+0000 7f6897400640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 0x7f687407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.791+0000 7f6897400640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 0x7f687407b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6888006fd0 tx=0x7f6888006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.794+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68980cdf30 con 0x7f68980cd5e0 2024-09-21T06:33:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.992+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68980cf4b0 con 0x7f68980cd5e0 2024-09-21T06:33:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.994+0000 7f688fe00640 1 -- 172.21.15.37:0/262658118 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17018 (secure 0 0 0) 0x7f68900633a0 con 0x7f68980cd5e0 2024-09-21T06:33:47.995 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:47.995 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:46.562699+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:33:44.503000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T06:33:45.519092+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:47.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 msgr2=0x7f687407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 0x7f687407b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6888006fd0 tx=0x7f6888006b10 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 msgr2=0x7f68980cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68980d0c30 tx=0x7f6890006430 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 shutdown_connections 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6874078b90 0x7f687407b0a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68980cd5e0 0x7f68980cd9f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68980ca540 0x7f68981f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 --2- 172.21.15.37:0/262658118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f687c0043a0 0x7f68981616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 >> 172.21.15.37:0/262658118 conn(0x7f68980d4360 msgr2=0x7f689806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 shutdown_connections 2024-09-21T06:33:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:47.996+0000 7f689cc00640 1 -- 172.21.15.37:0/262658118 wait complete. 2024-09-21T06:33:48.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:33:48.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 -- 172.21.15.37:0/4018706289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 msgr2=0x7f8b4c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/4018706289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b4c005220 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8b60001f00 tx=0x7f8b4c005830 comp rx=0 tx=0).stop 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6d200640 1 -- 172.21.15.37:0/4018706289 reap_dead start 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 -- 172.21.15.37:0/4018706289 shutdown_connections 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 -- 172.21.15.37:0/4018706289 >> 172.21.15.37:0/4018706289 conn(0x7f8b680cd570 msgr2=0x7f8b680cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 -- 172.21.15.37:0/4018706289 shutdown_connections 2024-09-21T06:33:48.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.253+0000 7f8b6dc00640 1 -- 172.21.15.37:0/4018706289 wait complete. 2024-09-21T06:33:48.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.255+0000 7f8b6dc00640 1 Processor -- start 2024-09-21T06:33:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.256+0000 7f8b6dc00640 1 -- start start 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 0x7f8b6814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b681f8e60 0x7f8b68157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6806e6e0 con 0x7f8b4c004d90 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6806e560 con 0x7f8b68155d80 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6dc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6815b7e0 con 0x7f8b681f8e60 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39344/0 (socket says 172.21.15.37:39344) 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b67e00640 1 -- 172.21.15.37:0/2614273930 learned_addr learned my addr 172.21.15.37:0/2614273930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b6d200640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b681f8e60 0x7f8b68157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.257+0000 7f8b5fe00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 0x7f8b6814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 -- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b681f8e60 msgr2=0x7f8b68157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b681f8e60 0x7f8b68157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 -- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 msgr2=0x7f8b6814fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 0x7f8b6814fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 -- 172.21.15.37:0/2614273930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681f86a0 con 0x7f8b4c004d90 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b5fe00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 0x7f8b6814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b67e00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b10 tx=0x7f8b58009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5801e070 con 0x7f8b4c004d90 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b5800fe40 con 0x7f8b4c004d90 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b580042b0 con 0x7f8b4c004d90 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b68155ae0 con 0x7f8b4c004d90 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.258+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b680b6390 con 0x7f8b4c004d90 2024-09-21T06:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.260+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6815bd70 con 0x7f8b4c004d90 2024-09-21T06:33:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.261+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b58021070 con 0x7f8b4c004d90 2024-09-21T06:33:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.262+0000 7f8b64c00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 0x7f8b4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.262+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f8b5809ba50 con 0x7f8b4c004d90 2024-09-21T06:33:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.262+0000 7f8b5fe00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 0x7f8b4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:48.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.263+0000 7f8b5fe00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 0x7f8b4407b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b6815b3b0 tx=0x7f8b50006720 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:48.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.266+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b580636d0 con 0x7f8b4c004d90 2024-09-21T06:33:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:48.499+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8b6815b3e0 con 0x7f8b4c004d90 2024-09-21T06:33:49.015 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:33:49.013+0000 7fa6d7a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:33:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.059+0000 7f8b64c00640 1 -- 172.21.15.37:0/2614273930 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) ==== 118+0+0 (secure 0 0 0) 0x7f8b58068100 con 0x7f8b4c004d90 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 msgr2=0x7f8b4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 0x7f8b4407b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b6815b3b0 tx=0x7f8b50006720 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 msgr2=0x7f8b680b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b10 tx=0x7f8b58009e90 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 shutdown_connections 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b44078b90 0x7f8b4407b0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b681f8e60 0x7f8b68157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b68155d80 0x7f8b6814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 --2- 172.21.15.37:0/2614273930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b4c004d90 0x7f8b680b5a60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 >> 172.21.15.37:0/2614273930 conn(0x7f8b680cd570 msgr2=0x7f8b680cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 shutdown_connections 2024-09-21T06:33:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.062+0000 7f8b6dc00640 1 -- 172.21.15.37:0/2614273930 wait complete. 2024-09-21T06:33:49.087 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:33:49.087 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:33:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.318+0000 7f227b200640 1 -- 172.21.15.37:0/1120970820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274071c10 msgr2=0x7f22740cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.318+0000 7f227b200640 1 --2- 172.21.15.37:0/1120970820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274071c10 0x7f22740cc6c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2268009830 tx=0x7f226802a520 comp rx=0 tx=0).stop 2024-09-21T06:33:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 -- 172.21.15.37:0/1120970820 shutdown_connections 2024-09-21T06:33:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 --2- 172.21.15.37:0/1120970820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22740ccc00 0x7f22741604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 --2- 172.21.15.37:0/1120970820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274071c10 0x7f22740cc6c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 -- 172.21.15.37:0/1120970820 >> 172.21.15.37:0/1120970820 conn(0x7f22740ceaf0 msgr2=0x7f22740cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:49.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 -- 172.21.15.37:0/1120970820 shutdown_connections 2024-09-21T06:33:49.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.319+0000 7f227b200640 1 -- 172.21.15.37:0/1120970820 wait complete. 2024-09-21T06:33:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.321+0000 7f227b200640 1 Processor -- start 2024-09-21T06:33:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.321+0000 7f227b200640 1 -- start start 2024-09-21T06:33:49.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2274071c10 0x7f227415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22741609e0 0x7f22740b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22741536d0 con 0x7f22740ccc00 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2274153550 con 0x7f22741609e0 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.322+0000 7f227b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2274153850 con 0x7f2274071c10 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39360/0 (socket says 172.21.15.37:39360) 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 -- 172.21.15.37:0/984163357 learned_addr learned my addr 172.21.15.37:0/984163357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279e00640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2274071c10 0x7f227415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f227a800640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22741609e0 0x7f22740b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 -- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2274071c10 msgr2=0x7f227415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2274071c10 0x7f227415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 -- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22741609e0 msgr2=0x7f22740b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22741609e0 0x7f22740b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.323+0000 7f2279400640 1 -- 172.21.15.37:0/984163357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22741f8e50 con 0x7f22740ccc00 2024-09-21T06:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f2279400640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f226400e9f0 tx=0x7f226400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226400cd20 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2264004020 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264004470 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22741f9130 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.324+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741f96f0 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.325+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22740cc6c0 con 0x7f22740ccc00 2024-09-21T06:33:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.326+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22640035f0 con 0x7f22740ccc00 2024-09-21T06:33:49.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.327+0000 7f226e000640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 0x7f225807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.327+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f226409b820 con 0x7f22740ccc00 2024-09-21T06:33:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.328+0000 7f2279e00640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 0x7f225807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.328+0000 7f2279e00640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 0x7f225807b0d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2268034000 tx=0x7f2268004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:49.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.332+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22640639b0 con 0x7f22740ccc00 2024-09-21T06:33:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:49.567+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f22740cf850 con 0x7f22740ccc00 2024-09-21T06:33:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.067+0000 7f226e000640 1 -- 172.21.15.37:0/984163357 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) ==== 122+0+0 (secure 0 0 0) 0x7f22640632b0 con 0x7f22740ccc00 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.069+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 msgr2=0x7f225807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.069+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 0x7f225807b0d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2268034000 tx=0x7f2268004730 comp rx=0 tx=0).stop 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.069+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 msgr2=0x7f22740b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.069+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f226400e9f0 tx=0x7f226400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.069+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 shutdown_connections 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2258078bc0 0x7f225807b0d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22740ccc00 0x7f22740b76d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22741609e0 0x7f22740b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 --2- 172.21.15.37:0/984163357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2274071c10 0x7f227415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 >> 172.21.15.37:0/984163357 conn(0x7f22740ceaf0 msgr2=0x7f22740d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 shutdown_connections 2024-09-21T06:33:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.070+0000 7f227b200640 1 -- 172.21.15.37:0/984163357 wait complete. 2024-09-21T06:33:50.093 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:33:50.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:50.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 -- 172.21.15.37:0/1650648010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef8071870 msgr2=0x7f9ef8145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 --2- 172.21.15.37:0/1650648010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef8071870 0x7f9ef8145040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005fd0 tx=0x7f9ee002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 -- 172.21.15.37:0/1650648010 shutdown_connections 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 --2- 172.21.15.37:0/1650648010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef8071870 0x7f9ef8145040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 --2- 172.21.15.37:0/1650648010 >> v2:172.21.15.150:3300/0 conn(0x7f9ef8160990 0x7f9ef80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 -- 172.21.15.37:0/1650648010 >> 172.21.15.37:0/1650648010 conn(0x7f9ef80d1020 msgr2=0x7f9ef80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 -- 172.21.15.37:0/1650648010 shutdown_connections 2024-09-21T06:33:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.319+0000 7f9efda00640 1 -- 172.21.15.37:0/1650648010 wait complete. 2024-09-21T06:33:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.321+0000 7f9efda00640 1 Processor -- start 2024-09-21T06:33:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.322+0000 7f9efda00640 1 -- start start 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef80d0b00 0x7f9ef8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ef815a080 0x7f9ef8139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cdf00 con 0x7f9ef8160990 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cdd80 con 0x7f9ef815a080 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efda00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef80ce080 con 0x7f9ef80d0b00 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9eefe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9eefe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39374/0 (socket says 172.21.15.37:39374) 2024-09-21T06:33:50.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9eefe00640 1 -- 172.21.15.37:0/1296588871 learned_addr learned my addr 172.21.15.37:0/1296588871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9efd000640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ef815a080 0x7f9ef8139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.323+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef80d0b00 0x7f9ef8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 -- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef80d0b00 msgr2=0x7f9ef8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef80d0b00 0x7f9ef8159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 -- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ef815a080 msgr2=0x7f9ef8139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ef815a080 0x7f9ef8139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 -- 172.21.15.37:0/1296588871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef8159890 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9eefe00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0000c00 tx=0x7f9ee00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003e030 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ee0003ca0 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.324+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0035040 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.325+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef8141e60 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.325+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef813f0c0 con 0x7f9ef8160990 2024-09-21T06:33:50.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.326+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef81443b0 con 0x7f9ef8160990 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.327+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ee0041030 con 0x7f9ef8160990 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.328+0000 7f9ef4c00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 0x7f9ed407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.328+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9ee00bb360 con 0x7f9ef8160990 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.329+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 0x7f9ed407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.329+0000 7f9ef7e00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 0x7f9ed407b0d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8000f60 tx=0x7f9ee8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.333+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ee0083040 con 0x7f9ef8160990 2024-09-21T06:33:50.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:33:50.434 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:33:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.569+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ef80ce5a0 con 0x7f9ef8160990 2024-09-21T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.570+0000 7f9ef4c00640 1 -- 172.21.15.37:0/1296588871 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5228 (secure 0 0 0) 0x7f9ee0087a70 con 0x7f9ef8160990 2024-09-21T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"btime":"2024-09-21T06:33:50:060429+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:33:49.568429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392]},"id":6}]} 2024-09-21T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 89 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 msgr2=0x7f9ed407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 0x7f9ed407b0d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8000f60 tx=0x7f9ee8009040 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 msgr2=0x7f9ef8139a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0000c00 tx=0x7f9ee00038a0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 shutdown_connections 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9ed4078bc0 0x7f9ed407b0d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ef8160990 0x7f9ef8139a50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ef815a080 0x7f9ef8139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 --2- 172.21.15.37:0/1296588871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ef80d0b00 0x7f9ef8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 >> 172.21.15.37:0/1296588871 conn(0x7f9ef80d1020 msgr2=0x7f9ef815be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:50.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 shutdown_connections 2024-09-21T06:33:50.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.572+0000 7f9efda00640 1 -- 172.21.15.37:0/1296588871 wait complete. 2024-09-21T06:33:50.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fc6600640 1 -- 172.21.15.37:0/1997221705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 msgr2=0x7f0fa4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1997221705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fa4004800 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009b50 tx=0x7f0fa4005830 comp rx=0 tx=0).stop 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fbfe00640 1 -- 172.21.15.37:0/1997221705 reap_dead start 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fc6600640 1 -- 172.21.15.37:0/1997221705 shutdown_connections 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fc6600640 1 -- 172.21.15.37:0/1997221705 >> 172.21.15.37:0/1997221705 conn(0x7f0fc00cca40 msgr2=0x7f0fc00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.823+0000 7f0fc6600640 1 -- 172.21.15.37:0/1997221705 shutdown_connections 2024-09-21T06:33:50.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.824+0000 7f0fc6600640 1 -- 172.21.15.37:0/1997221705 wait complete. 2024-09-21T06:33:50.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.826+0000 7f0fc6600640 1 Processor -- start 2024-09-21T06:33:50.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.826+0000 7f0fc6600640 1 -- start start 2024-09-21T06:33:50.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc0154670 0x7f0fc00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc01576a0 0x7f0fc0157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc006df70 con 0x7f0fa4003fd0 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc006ddf0 con 0x7f0fc0154670 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.827+0000 7f0fc6600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc006e0f0 con 0x7f0fc01576a0 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39398/0 (socket says 172.21.15.37:39398) 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 -- 172.21.15.37:0/1218393697 learned_addr learned my addr 172.21.15.37:0/1218393697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbea00640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc0154670 0x7f0fc00b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbfe00640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc01576a0 0x7f0fc0157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 -- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc01576a0 msgr2=0x7f0fc0157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc01576a0 0x7f0fc0157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 -- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc0154670 msgr2=0x7f0fc00b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc0154670 0x7f0fc00b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.828+0000 7f0fbf400640 1 -- 172.21.15.37:0/1218393697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc00a0f40 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fbf400640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0006600 tx=0x7f0fb00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0015070 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fb0011920 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc0156bc0 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0011d70 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.829+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc0154a80 con 0x7f0fa4003fd0 2024-09-21T06:33:50.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.830+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc0069420 con 0x7f0fa4003fd0 2024-09-21T06:33:50.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.831+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0fb0018070 con 0x7f0fa4003fd0 2024-09-21T06:33:50.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.832+0000 7f0fb7400640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 0x7f0fa007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:50.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.832+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0fb009bc80 con 0x7f0fa4003fd0 2024-09-21T06:33:50.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.833+0000 7f0fbea00640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 0x7f0fa007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:50.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.833+0000 7f0fbea00640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 0x7f0fa007b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0fac035000 tx=0x7f0fac0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:50.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:50.837+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0fc0069420 con 0x7f0fa4003fd0 2024-09-21T06:33:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.033+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fc00cbd10 con 0x7f0fa4003fd0 2024-09-21T06:33:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.034+0000 7f0fb7400640 1 -- 172.21.15.37:0/1218393697 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17005 (secure 0 0 0) 0x7f0fc00cbd10 con 0x7f0fa4003fd0 2024-09-21T06:33:51.035 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:51.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:48.044956+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:33:44.503000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T06:33:45.519092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 msgr2=0x7f0fa007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 0x7f0fa007b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0fac035000 tx=0x7f0fac0046c0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 msgr2=0x7f0fc01f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0006600 tx=0x7f0fb00039c0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 shutdown_connections 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0fa0078bc0 0x7f0fa007b0d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc01576a0 0x7f0fc0157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc0154670 0x7f0fc00b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 --2- 172.21.15.37:0/1218393697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fa4003fd0 0x7f0fc01f35c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 >> 172.21.15.37:0/1218393697 conn(0x7f0fc00cca40 msgr2=0x7f0fc0070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 shutdown_connections 2024-09-21T06:33:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.036+0000 7f0fc6600640 1 -- 172.21.15.37:0/1218393697 wait complete. 2024-09-21T06:33:51.059 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:33:51.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:33:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 -- 172.21.15.37:0/1006023719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28ec004d90 msgr2=0x7f28ec005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 --2- 172.21.15.37:0/1006023719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28ec004d90 0x7f28ec005220 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f290000c610 tx=0x7f28ec005830 comp rx=0 tx=0).stop 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f2907e00640 1 -- 172.21.15.37:0/1006023719 reap_dead start 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 -- 172.21.15.37:0/1006023719 shutdown_connections 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 -- 172.21.15.37:0/1006023719 >> 172.21.15.37:0/1006023719 conn(0x7f290806f940 msgr2=0x7f29080b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 -- 172.21.15.37:0/1006023719 shutdown_connections 2024-09-21T06:33:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.287+0000 7f290cc00640 1 -- 172.21.15.37:0/1006023719 wait complete. 2024-09-21T06:33:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.289+0000 7f290cc00640 1 Processor -- start 2024-09-21T06:33:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.290+0000 7f290cc00640 1 -- start start 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28ec004d90 0x7f29080aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29081f9100 0x7f2908073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29080b52d0 con 0x7f2908074210 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29080b5150 con 0x7f29081f9100 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f290cc00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f290806d330 con 0x7f28ec004d90 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f2907e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f2907e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39414/0 (socket says 172.21.15.37:39414) 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.291+0000 7f2907e00640 1 -- 172.21.15.37:0/4288011035 learned_addr learned my addr 172.21.15.37:0/4288011035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907400640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28ec004d90 0x7f29080aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f28ffe00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29081f9100 0x7f2908073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 -- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28ec004d90 msgr2=0x7f29080aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28ec004d90 0x7f29080aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 -- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29081f9100 msgr2=0x7f2908073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29081f9100 0x7f2908073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 -- 172.21.15.37:0/4288011035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2908072ca0 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f2907e00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f290000c730 tx=0x7f2900004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.292+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290002b900 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.293+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f290002bb90 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.293+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000060a0 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.293+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2908072f10 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.293+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29080734d0 con 0x7f2908074210 2024-09-21T06:33:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.294+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290806d8c0 con 0x7f2908074210 2024-09-21T06:33:51.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.295+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2900006240 con 0x7f2908074210 2024-09-21T06:33:51.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.296+0000 7f28ff400640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 0x7f28e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.296+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f29000bae40 con 0x7f2908074210 2024-09-21T06:33:51.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.297+0000 7f2907400640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 0x7f28e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.297+0000 7f2907400640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 0x7f28e407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000c00 tx=0x7f28f8006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:51.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.301+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2900082b20 con 0x7f2908074210 2024-09-21T06:33:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.495+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2908145e70 con 0x7f28e4078b90 2024-09-21T06:33:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.500+0000 7f28ff400640 1 -- 172.21.15.37:0/4288011035 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7f2908145e70 con 0x7f28e4078b90 2024-09-21T06:33:51.505 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:33:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 msgr2=0x7f28e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 0x7f28e407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000c00 tx=0x7f28f8006b10 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 msgr2=0x7f2908074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f290000c730 tx=0x7f2900004bb0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 shutdown_connections 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f28e4078b90 0x7f28e407b0a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2908074210 0x7f2908074620 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29081f9100 0x7f2908073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 --2- 172.21.15.37:0/4288011035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28ec004d90 0x7f29080aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 >> 172.21.15.37:0/4288011035 conn(0x7f290806f940 msgr2=0x7f29080cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 shutdown_connections 2024-09-21T06:33:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.504+0000 7f290cc00640 1 -- 172.21.15.37:0/4288011035 wait complete. 2024-09-21T06:33:51.530 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-21T06:33:51.185402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259368,"kb_used_data":9844,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715352,"statfs":{"total":773094113280,"available":772828520448,"internally_reserved":0,"allocated":10080256,"data_stored":5243936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001473"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058080+0000","last_change":"2024-09-21T06:33:47.049389+0000","last_active":"2024-09-21T06:33:47.058080+0000","last_peered":"2024-09-21T06:33:47.058080+0000","last_clean":"2024-09-21T06:33:47.058080+0000","last_became_active":"2024-09-21T06:33:47.048667+0000","last_became_peered":"2024-09-21T06:33:47.048667+0000","last_unstale":"2024-09-21T06:33:47.058080+0000","last_undegraded":"2024-09-21T06:33:47.058080+0000","last_fullsized":"2024-09-21T06:33:47.058080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:09.587111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077491+0000","last_change":"2024-09-21T06:33:47.072197+0000","last_active":"2024-09-21T06:33:47.077491+0000","last_peered":"2024-09-21T06:33:47.077491+0000","last_clean":"2024-09-21T06:33:47.077491+0000","last_became_active":"2024-09-21T06:33:47.071516+0000","last_became_peered":"2024-09-21T06:33:47.071516+0000","last_unstale":"2024-09-21T06:33:47.077491+0000","last_undegraded":"2024-09-21T06:33:47.077491+0000","last_fullsized":"2024-09-21T06:33:47.077491+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083356+0000","last_change":"2024-09-21T06:33:47.080958+0000","last_active":"2024-09-21T06:33:47.083356+0000","last_peered":"2024-09-21T06:33:47.083356+0000","last_clean":"2024-09-21T06:33:47.083356+0000","last_became_active":"2024-09-21T06:33:47.080628+0000","last_became_peered":"2024-09-21T06:33:47.080628+0000","last_unstale":"2024-09-21T06:33:47.083356+0000","last_undegraded":"2024-09-21T06:33:47.083356+0000","last_fullsized":"2024-09-21T06:33:47.083356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:26.186706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083199+0000","last_change":"2024-09-21T06:33:47.079500+0000","last_active":"2024-09-21T06:33:47.083199+0000","last_peered":"2024-09-21T06:33:47.083199+0000","last_clean":"2024-09-21T06:33:47.083199+0000","last_became_active":"2024-09-21T06:33:47.078806+0000","last_became_peered":"2024-09-21T06:33:47.078806+0000","last_unstale":"2024-09-21T06:33:47.083199+0000","last_undegraded":"2024-09-21T06:33:47.083199+0000","last_fullsized":"2024-09-21T06:33:47.083199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:22.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075024+0000","last_change":"2024-09-21T06:33:47.067426+0000","last_active":"2024-09-21T06:33:47.075024+0000","last_peered":"2024-09-21T06:33:47.075024+0000","last_clean":"2024-09-21T06:33:47.075024+0000","last_became_active":"2024-09-21T06:33:47.066970+0000","last_became_peered":"2024-09-21T06:33:47.066970+0000","last_unstale":"2024-09-21T06:33:47.075024+0000","last_undegraded":"2024-09-21T06:33:47.075024+0000","last_fullsized":"2024-09-21T06:33:47.075024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:34.760478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069828+0000","last_change":"2024-09-21T06:33:47.062230+0000","last_active":"2024-09-21T06:33:47.069828+0000","last_peered":"2024-09-21T06:33:47.069828+0000","last_clean":"2024-09-21T06:33:47.069828+0000","last_became_active":"2024-09-21T06:33:47.060278+0000","last_became_peered":"2024-09-21T06:33:47.060278+0000","last_unstale":"2024-09-21T06:33:47.069828+0000","last_undegraded":"2024-09-21T06:33:47.069828+0000","last_fullsized":"2024-09-21T06:33:47.069828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:05.599195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079824+0000","last_change":"2024-09-21T06:33:47.076019+0000","last_active":"2024-09-21T06:33:47.079824+0000","last_peered":"2024-09-21T06:33:47.079824+0000","last_clean":"2024-09-21T06:33:47.079824+0000","last_became_active":"2024-09-21T06:33:47.075274+0000","last_became_peered":"2024-09-21T06:33:47.075274+0000","last_unstale":"2024-09-21T06:33:47.079824+0000","last_undegraded":"2024-09-21T06:33:47.079824+0000","last_fullsized":"2024-09-21T06:33:47.079824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.454721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060849+0000","last_change":"2024-09-21T06:33:47.048923+0000","last_active":"2024-09-21T06:33:47.060849+0000","last_peered":"2024-09-21T06:33:47.060849+0000","last_clean":"2024-09-21T06:33:47.060849+0000","last_became_active":"2024-09-21T06:33:47.047971+0000","last_became_peered":"2024-09-21T06:33:47.047971+0000","last_unstale":"2024-09-21T06:33:47.060849+0000","last_undegraded":"2024-09-21T06:33:47.060849+0000","last_fullsized":"2024-09-21T06:33:47.060849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:50.754265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.071666+0000","last_change":"2024-09-21T06:33:47.061055+0000","last_active":"2024-09-21T06:33:47.071666+0000","last_peered":"2024-09-21T06:33:47.071666+0000","last_clean":"2024-09-21T06:33:47.071666+0000","last_became_active":"2024-09-21T06:33:47.060703+0000","last_became_peered":"2024-09-21T06:33:47.060703+0000","last_unstale":"2024-09-21T06:33:47.071666+0000","last_undegraded":"2024-09-21T06:33:47.071666+0000","last_fullsized":"2024-09-21T06:33:47.071666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.964035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083288+0000","last_change":"2024-09-21T06:33:47.080473+0000","last_active":"2024-09-21T06:33:47.083288+0000","last_peered":"2024-09-21T06:33:47.083288+0000","last_clean":"2024-09-21T06:33:47.083288+0000","last_became_active":"2024-09-21T06:33:47.080163+0000","last_became_peered":"2024-09-21T06:33:47.080163+0000","last_unstale":"2024-09-21T06:33:47.083288+0000","last_undegraded":"2024-09-21T06:33:47.083288+0000","last_fullsized":"2024-09-21T06:33:47.083288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:07.512354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080048+0000","last_change":"2024-09-21T06:33:47.076242+0000","last_active":"2024-09-21T06:33:47.080048+0000","last_peered":"2024-09-21T06:33:47.080048+0000","last_clean":"2024-09-21T06:33:47.080048+0000","last_became_active":"2024-09-21T06:33:47.075589+0000","last_became_peered":"2024-09-21T06:33:47.075589+0000","last_unstale":"2024-09-21T06:33:47.080048+0000","last_undegraded":"2024-09-21T06:33:47.080048+0000","last_fullsized":"2024-09-21T06:33:47.080048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:06.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.081178+0000","last_change":"2024-09-21T06:33:47.078321+0000","last_active":"2024-09-21T06:33:47.081178+0000","last_peered":"2024-09-21T06:33:47.081178+0000","last_clean":"2024-09-21T06:33:47.081178+0000","last_became_active":"2024-09-21T06:33:47.073777+0000","last_became_peered":"2024-09-21T06:33:47.073777+0000","last_unstale":"2024-09-21T06:33:47.081178+0000","last_undegraded":"2024-09-21T06:33:47.081178+0000","last_fullsized":"2024-09-21T06:33:47.081178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.039561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375004+0000","last_change":"2024-09-21T06:33:47.054106+0000","last_active":"2024-09-21T06:33:48.375004+0000","last_peered":"2024-09-21T06:33:48.375004+0000","last_clean":"2024-09-21T06:33:48.375004+0000","last_became_active":"2024-09-21T06:33:47.052851+0000","last_became_peered":"2024-09-21T06:33:47.052851+0000","last_unstale":"2024-09-21T06:33:48.375004+0000","last_undegraded":"2024-09-21T06:33:48.375004+0000","last_fullsized":"2024-09-21T06:33:48.375004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:00.605543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077519+0000","last_change":"2024-09-21T06:33:47.073657+0000","last_active":"2024-09-21T06:33:47.077519+0000","last_peered":"2024-09-21T06:33:47.077519+0000","last_clean":"2024-09-21T06:33:47.077519+0000","last_became_active":"2024-09-21T06:33:47.070661+0000","last_became_peered":"2024-09-21T06:33:47.070661+0000","last_unstale":"2024-09-21T06:33:47.077519+0000","last_undegraded":"2024-09-21T06:33:47.077519+0000","last_fullsized":"2024-09-21T06:33:47.077519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:12.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083400+0000","last_change":"2024-09-21T06:33:47.081483+0000","last_active":"2024-09-21T06:33:47.083400+0000","last_peered":"2024-09-21T06:33:47.083400+0000","last_clean":"2024-09-21T06:33:47.083400+0000","last_became_active":"2024-09-21T06:33:47.079784+0000","last_became_peered":"2024-09-21T06:33:47.079784+0000","last_unstale":"2024-09-21T06:33:47.083400+0000","last_undegraded":"2024-09-21T06:33:47.083400+0000","last_fullsized":"2024-09-21T06:33:47.083400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:59.939368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079186+0000","last_change":"2024-09-21T06:33:47.069798+0000","last_active":"2024-09-21T06:33:47.079186+0000","last_peered":"2024-09-21T06:33:47.079186+0000","last_clean":"2024-09-21T06:33:47.079186+0000","last_became_active":"2024-09-21T06:33:47.068713+0000","last_became_peered":"2024-09-21T06:33:47.068713+0000","last_unstale":"2024-09-21T06:33:47.079186+0000","last_undegraded":"2024-09-21T06:33:47.079186+0000","last_fullsized":"2024-09-21T06:33:47.079186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:52.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058548+0000","last_change":"2024-09-21T06:33:47.049006+0000","last_active":"2024-09-21T06:33:47.058548+0000","last_peered":"2024-09-21T06:33:47.058548+0000","last_clean":"2024-09-21T06:33:47.058548+0000","last_became_active":"2024-09-21T06:33:47.048672+0000","last_became_peered":"2024-09-21T06:33:47.048672+0000","last_unstale":"2024-09-21T06:33:47.058548+0000","last_undegraded":"2024-09-21T06:33:47.058548+0000","last_fullsized":"2024-09-21T06:33:47.058548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:59.279272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082983+0000","last_change":"2024-09-21T06:33:47.078853+0000","last_active":"2024-09-21T06:33:47.082983+0000","last_peered":"2024-09-21T06:33:47.082983+0000","last_clean":"2024-09-21T06:33:47.082983+0000","last_became_active":"2024-09-21T06:33:47.078548+0000","last_became_peered":"2024-09-21T06:33:47.078548+0000","last_unstale":"2024-09-21T06:33:47.082983+0000","last_undegraded":"2024-09-21T06:33:47.082983+0000","last_fullsized":"2024-09-21T06:33:47.082983+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:34.981609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082347+0000","last_change":"2024-09-21T06:33:47.077280+0000","last_active":"2024-09-21T06:33:47.082347+0000","last_peered":"2024-09-21T06:33:47.082347+0000","last_clean":"2024-09-21T06:33:47.082347+0000","last_became_active":"2024-09-21T06:33:47.076940+0000","last_became_peered":"2024-09-21T06:33:47.076940+0000","last_unstale":"2024-09-21T06:33:47.082347+0000","last_undegraded":"2024-09-21T06:33:47.082347+0000","last_fullsized":"2024-09-21T06:33:47.082347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:02.117809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060127+0000","last_change":"2024-09-21T06:33:47.051768+0000","last_active":"2024-09-21T06:33:47.060127+0000","last_peered":"2024-09-21T06:33:47.060127+0000","last_clean":"2024-09-21T06:33:47.060127+0000","last_became_active":"2024-09-21T06:33:47.051472+0000","last_became_peered":"2024-09-21T06:33:47.051472+0000","last_unstale":"2024-09-21T06:33:47.060127+0000","last_undegraded":"2024-09-21T06:33:47.060127+0000","last_fullsized":"2024-09-21T06:33:47.060127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:41.422642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074075+0000","last_change":"2024-09-21T06:33:47.062997+0000","last_active":"2024-09-21T06:33:47.074075+0000","last_peered":"2024-09-21T06:33:47.074075+0000","last_clean":"2024-09-21T06:33:47.074075+0000","last_became_active":"2024-09-21T06:33:47.062608+0000","last_became_peered":"2024-09-21T06:33:47.062608+0000","last_unstale":"2024-09-21T06:33:47.074075+0000","last_undegraded":"2024-09-21T06:33:47.074075+0000","last_fullsized":"2024-09-21T06:33:47.074075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:29.364069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.061435+0000","last_change":"2024-09-21T06:33:47.053231+0000","last_active":"2024-09-21T06:33:47.061435+0000","last_peered":"2024-09-21T06:33:47.061435+0000","last_clean":"2024-09-21T06:33:47.061435+0000","last_became_active":"2024-09-21T06:33:47.052877+0000","last_became_peered":"2024-09-21T06:33:47.052877+0000","last_unstale":"2024-09-21T06:33:47.061435+0000","last_undegraded":"2024-09-21T06:33:47.061435+0000","last_fullsized":"2024-09-21T06:33:47.061435+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.185385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077450+0000","last_change":"2024-09-21T06:33:47.069847+0000","last_active":"2024-09-21T06:33:47.077450+0000","last_peered":"2024-09-21T06:33:47.077450+0000","last_clean":"2024-09-21T06:33:47.077450+0000","last_became_active":"2024-09-21T06:33:47.064139+0000","last_became_peered":"2024-09-21T06:33:47.064139+0000","last_unstale":"2024-09-21T06:33:47.077450+0000","last_undegraded":"2024-09-21T06:33:47.077450+0000","last_fullsized":"2024-09-21T06:33:47.077450+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:16.965639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077350+0000","last_change":"2024-09-21T06:33:47.075184+0000","last_active":"2024-09-21T06:33:47.077350+0000","last_peered":"2024-09-21T06:33:47.077350+0000","last_clean":"2024-09-21T06:33:47.077350+0000","last_became_active":"2024-09-21T06:33:47.074823+0000","last_became_peered":"2024-09-21T06:33:47.074823+0000","last_unstale":"2024-09-21T06:33:47.077350+0000","last_undegraded":"2024-09-21T06:33:47.077350+0000","last_fullsized":"2024-09-21T06:33:47.077350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:02.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075134+0000","last_change":"2024-09-21T06:33:47.068701+0000","last_active":"2024-09-21T06:33:47.075134+0000","last_peered":"2024-09-21T06:33:47.075134+0000","last_clean":"2024-09-21T06:33:47.075134+0000","last_became_active":"2024-09-21T06:33:47.067891+0000","last_became_peered":"2024-09-21T06:33:47.067891+0000","last_unstale":"2024-09-21T06:33:47.075134+0000","last_undegraded":"2024-09-21T06:33:47.075134+0000","last_fullsized":"2024-09-21T06:33:47.075134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.185385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077437+0000","last_change":"2024-09-21T06:33:47.071849+0000","last_active":"2024-09-21T06:33:47.077437+0000","last_peered":"2024-09-21T06:33:47.077437+0000","last_clean":"2024-09-21T06:33:47.077437+0000","last_became_active":"2024-09-21T06:33:47.071015+0000","last_became_peered":"2024-09-21T06:33:47.071015+0000","last_unstale":"2024-09-21T06:33:47.077437+0000","last_undegraded":"2024-09-21T06:33:47.077437+0000","last_fullsized":"2024-09-21T06:33:47.077437+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080050+0000","last_change":"2024-09-21T06:33:47.078274+0000","last_active":"2024-09-21T06:33:47.080050+0000","last_peered":"2024-09-21T06:33:47.080050+0000","last_clean":"2024-09-21T06:33:47.080050+0000","last_became_active":"2024-09-21T06:33:47.077743+0000","last_became_peered":"2024-09-21T06:33:47.077743+0000","last_unstale":"2024-09-21T06:33:47.080050+0000","last_undegraded":"2024-09-21T06:33:47.080050+0000","last_fullsized":"2024-09-21T06:33:47.080050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:34.823696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375316+0000","last_change":"2024-09-21T06:33:47.047428+0000","last_active":"2024-09-21T06:33:48.375316+0000","last_peered":"2024-09-21T06:33:48.375316+0000","last_clean":"2024-09-21T06:33:48.375316+0000","last_became_active":"2024-09-21T06:33:47.045850+0000","last_became_peered":"2024-09-21T06:33:47.045850+0000","last_unstale":"2024-09-21T06:33:48.375316+0000","last_undegraded":"2024-09-21T06:33:48.375316+0000","last_fullsized":"2024-09-21T06:33:48.375316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:20.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075051+0000","last_change":"2024-09-21T06:33:47.068008+0000","last_active":"2024-09-21T06:33:47.075051+0000","last_peered":"2024-09-21T06:33:47.075051+0000","last_clean":"2024-09-21T06:33:47.075051+0000","last_became_active":"2024-09-21T06:33:47.067659+0000","last_became_peered":"2024-09-21T06:33:47.067659+0000","last_unstale":"2024-09-21T06:33:47.075051+0000","last_undegraded":"2024-09-21T06:33:47.075051+0000","last_fullsized":"2024-09-21T06:33:47.075051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:40.174249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060753+0000","last_change":"2024-09-21T06:33:47.058440+0000","last_active":"2024-09-21T06:33:47.060753+0000","last_peered":"2024-09-21T06:33:47.060753+0000","last_clean":"2024-09-21T06:33:47.060753+0000","last_became_active":"2024-09-21T06:33:47.057544+0000","last_became_peered":"2024-09-21T06:33:47.057544+0000","last_unstale":"2024-09-21T06:33:47.060753+0000","last_undegraded":"2024-09-21T06:33:47.060753+0000","last_fullsized":"2024-09-21T06:33:47.060753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:42.956401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060678+0000","last_change":"2024-09-21T06:33:47.055177+0000","last_active":"2024-09-21T06:33:47.060678+0000","last_peered":"2024-09-21T06:33:47.060678+0000","last_clean":"2024-09-21T06:33:47.060678+0000","last_became_active":"2024-09-21T06:33:47.054684+0000","last_became_peered":"2024-09-21T06:33:47.054684+0000","last_unstale":"2024-09-21T06:33:47.060678+0000","last_undegraded":"2024-09-21T06:33:47.060678+0000","last_fullsized":"2024-09-21T06:33:47.060678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:16.965639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.076960+0000","last_change":"2024-09-21T06:33:47.069319+0000","last_active":"2024-09-21T06:33:47.076960+0000","last_peered":"2024-09-21T06:33:47.076960+0000","last_clean":"2024-09-21T06:33:47.076960+0000","last_became_active":"2024-09-21T06:33:47.068322+0000","last_became_peered":"2024-09-21T06:33:47.068322+0000","last_unstale":"2024-09-21T06:33:47.076960+0000","last_undegraded":"2024-09-21T06:33:47.076960+0000","last_fullsized":"2024-09-21T06:33:47.076960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:13.321633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058172+0000","last_change":"2024-09-21T06:33:47.048633+0000","last_active":"2024-09-21T06:33:47.058172+0000","last_peered":"2024-09-21T06:33:47.058172+0000","last_clean":"2024-09-21T06:33:47.058172+0000","last_became_active":"2024-09-21T06:33:47.048307+0000","last_became_peered":"2024-09-21T06:33:47.048307+0000","last_unstale":"2024-09-21T06:33:47.058172+0000","last_undegraded":"2024-09-21T06:33:47.058172+0000","last_fullsized":"2024-09-21T06:33:47.058172+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.821183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083012+0000","last_change":"2024-09-21T06:33:46.061041+0000","last_active":"2024-09-21T06:33:47.083012+0000","last_peered":"2024-09-21T06:33:47.083012+0000","last_clean":"2024-09-21T06:33:47.083012+0000","last_became_active":"2024-09-21T06:33:46.060737+0000","last_became_peered":"2024-09-21T06:33:46.060737+0000","last_unstale":"2024-09-21T06:33:47.083012+0000","last_undegraded":"2024-09-21T06:33:47.083012+0000","last_fullsized":"2024-09-21T06:33:47.083012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.564485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059959+0000","last_change":"2024-09-21T06:33:47.050929+0000","last_active":"2024-09-21T06:33:47.059959+0000","last_peered":"2024-09-21T06:33:47.059959+0000","last_clean":"2024-09-21T06:33:47.059959+0000","last_became_active":"2024-09-21T06:33:47.050633+0000","last_became_peered":"2024-09-21T06:33:47.050633+0000","last_unstale":"2024-09-21T06:33:47.059959+0000","last_undegraded":"2024-09-21T06:33:47.059959+0000","last_fullsized":"2024-09-21T06:33:47.059959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.139550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.073983+0000","last_change":"2024-09-21T06:33:46.052928+0000","last_active":"2024-09-21T06:33:47.073983+0000","last_peered":"2024-09-21T06:33:47.073983+0000","last_clean":"2024-09-21T06:33:47.073983+0000","last_became_active":"2024-09-21T06:33:46.052587+0000","last_became_peered":"2024-09-21T06:33:46.052587+0000","last_unstale":"2024-09-21T06:33:47.073983+0000","last_undegraded":"2024-09-21T06:33:47.073983+0000","last_fullsized":"2024-09-21T06:33:47.073983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:21.388222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.050871+0000","last_change":"2024-09-21T06:33:47.047368+0000","last_active":"2024-09-21T06:33:47.050871+0000","last_peered":"2024-09-21T06:33:47.050871+0000","last_clean":"2024-09-21T06:33:47.050871+0000","last_became_active":"2024-09-21T06:33:47.044714+0000","last_became_peered":"2024-09-21T06:33:47.044714+0000","last_unstale":"2024-09-21T06:33:47.050871+0000","last_undegraded":"2024-09-21T06:33:47.050871+0000","last_fullsized":"2024-09-21T06:33:47.050871+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:58.606357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074023+0000","last_change":"2024-09-21T06:33:46.059829+0000","last_active":"2024-09-21T06:33:47.074023+0000","last_peered":"2024-09-21T06:33:47.074023+0000","last_clean":"2024-09-21T06:33:47.074023+0000","last_became_active":"2024-09-21T06:33:46.059421+0000","last_became_peered":"2024-09-21T06:33:46.059421+0000","last_unstale":"2024-09-21T06:33:47.074023+0000","last_undegraded":"2024-09-21T06:33:47.074023+0000","last_fullsized":"2024-09-21T06:33:47.074023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:53.798551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083414+0000","last_change":"2024-09-21T06:33:47.082237+0000","last_active":"2024-09-21T06:33:47.083414+0000","last_peered":"2024-09-21T06:33:47.083414+0000","last_clean":"2024-09-21T06:33:47.083414+0000","last_became_active":"2024-09-21T06:33:47.080611+0000","last_became_peered":"2024-09-21T06:33:47.080611+0000","last_unstale":"2024-09-21T06:33:47.083414+0000","last_undegraded":"2024-09-21T06:33:47.083414+0000","last_fullsized":"2024-09-21T06:33:47.083414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:56.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069162+0000","last_change":"2024-09-21T06:33:46.041123+0000","last_active":"2024-09-21T06:33:47.069162+0000","last_peered":"2024-09-21T06:33:47.069162+0000","last_clean":"2024-09-21T06:33:47.069162+0000","last_became_active":"2024-09-21T06:33:46.040811+0000","last_became_peered":"2024-09-21T06:33:46.040811+0000","last_unstale":"2024-09-21T06:33:47.069162+0000","last_undegraded":"2024-09-21T06:33:47.069162+0000","last_fullsized":"2024-09-21T06:33:47.069162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:55.210198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374859+0000","last_change":"2024-09-21T06:33:47.060297+0000","last_active":"2024-09-21T06:33:48.374859+0000","last_peered":"2024-09-21T06:33:48.374859+0000","last_clean":"2024-09-21T06:33:48.374859+0000","last_became_active":"2024-09-21T06:33:47.059069+0000","last_became_peered":"2024-09-21T06:33:47.059069+0000","last_unstale":"2024-09-21T06:33:48.374859+0000","last_undegraded":"2024-09-21T06:33:48.374859+0000","last_fullsized":"2024-09-21T06:33:48.374859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:50.000043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080663+0000","last_change":"2024-09-21T06:33:46.055305+0000","last_active":"2024-09-21T06:33:47.080663+0000","last_peered":"2024-09-21T06:33:47.080663+0000","last_clean":"2024-09-21T06:33:47.080663+0000","last_became_active":"2024-09-21T06:33:46.054987+0000","last_became_peered":"2024-09-21T06:33:46.054987+0000","last_unstale":"2024-09-21T06:33:47.080663+0000","last_undegraded":"2024-09-21T06:33:47.080663+0000","last_fullsized":"2024-09-21T06:33:47.080663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:08.584194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077237+0000","last_change":"2024-09-21T06:33:47.074979+0000","last_active":"2024-09-21T06:33:47.077237+0000","last_peered":"2024-09-21T06:33:47.077237+0000","last_clean":"2024-09-21T06:33:47.077237+0000","last_became_active":"2024-09-21T06:33:47.074640+0000","last_became_peered":"2024-09-21T06:33:47.074640+0000","last_unstale":"2024-09-21T06:33:47.077237+0000","last_undegraded":"2024-09-21T06:33:47.077237+0000","last_fullsized":"2024-09-21T06:33:47.077237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:02.267714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374665+0000","last_change":"2024-09-21T06:33:46.053873+0000","last_active":"2024-09-21T06:33:48.374665+0000","last_peered":"2024-09-21T06:33:48.374665+0000","last_clean":"2024-09-21T06:33:48.374665+0000","last_became_active":"2024-09-21T06:33:46.053464+0000","last_became_peered":"2024-09-21T06:33:46.053464+0000","last_unstale":"2024-09-21T06:33:48.374665+0000","last_undegraded":"2024-09-21T06:33:48.374665+0000","last_fullsized":"2024-09-21T06:33:48.374665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:23.020990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079085+0000","last_change":"2024-09-21T06:33:47.074330+0000","last_active":"2024-09-21T06:33:47.079085+0000","last_peered":"2024-09-21T06:33:47.079085+0000","last_clean":"2024-09-21T06:33:47.079085+0000","last_became_active":"2024-09-21T06:33:47.073692+0000","last_became_peered":"2024-09-21T06:33:47.073692+0000","last_unstale":"2024-09-21T06:33:47.079085+0000","last_undegraded":"2024-09-21T06:33:47.079085+0000","last_fullsized":"2024-09-21T06:33:47.079085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:23.606097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.071226+0000","last_change":"2024-09-21T06:33:46.063703+0000","last_active":"2024-09-21T06:33:47.071226+0000","last_peered":"2024-09-21T06:33:47.071226+0000","last_clean":"2024-09-21T06:33:47.071226+0000","last_became_active":"2024-09-21T06:33:46.063351+0000","last_became_peered":"2024-09-21T06:33:46.063351+0000","last_unstale":"2024-09-21T06:33:47.071226+0000","last_undegraded":"2024-09-21T06:33:47.071226+0000","last_fullsized":"2024-09-21T06:33:47.071226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:20.775850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.070549+0000","last_change":"2024-09-21T06:33:47.060777+0000","last_active":"2024-09-21T06:33:47.070549+0000","last_peered":"2024-09-21T06:33:47.070549+0000","last_clean":"2024-09-21T06:33:47.070549+0000","last_became_active":"2024-09-21T06:33:47.060472+0000","last_became_peered":"2024-09-21T06:33:47.060472+0000","last_unstale":"2024-09-21T06:33:47.070549+0000","last_undegraded":"2024-09-21T06:33:47.070549+0000","last_fullsized":"2024-09-21T06:33:47.070549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046409+0000","last_change":"2024-09-21T06:33:46.055199+0000","last_active":"2024-09-21T06:33:47.046409+0000","last_peered":"2024-09-21T06:33:47.046409+0000","last_clean":"2024-09-21T06:33:47.046409+0000","last_became_active":"2024-09-21T06:33:46.054875+0000","last_became_peered":"2024-09-21T06:33:46.054875+0000","last_unstale":"2024-09-21T06:33:47.046409+0000","last_undegraded":"2024-09-21T06:33:47.046409+0000","last_fullsized":"2024-09-21T06:33:47.046409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:30.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.066702+0000","last_change":"2024-09-21T06:33:47.061559+0000","last_active":"2024-09-21T06:33:47.066702+0000","last_peered":"2024-09-21T06:33:47.066702+0000","last_clean":"2024-09-21T06:33:47.066702+0000","last_became_active":"2024-09-21T06:33:47.061260+0000","last_became_peered":"2024-09-21T06:33:47.061260+0000","last_unstale":"2024-09-21T06:33:47.066702+0000","last_undegraded":"2024-09-21T06:33:47.066702+0000","last_fullsized":"2024-09-21T06:33:47.066702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:01.659246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.056535+0000","last_change":"2024-09-21T06:33:46.057265+0000","last_active":"2024-09-21T06:33:47.056535+0000","last_peered":"2024-09-21T06:33:47.056535+0000","last_clean":"2024-09-21T06:33:47.056535+0000","last_became_active":"2024-09-21T06:33:46.056922+0000","last_became_peered":"2024-09-21T06:33:46.056922+0000","last_unstale":"2024-09-21T06:33:47.056535+0000","last_undegraded":"2024-09-21T06:33:47.056535+0000","last_fullsized":"2024-09-21T06:33:47.056535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:06.564753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.057685+0000","last_change":"2024-09-21T06:33:47.051711+0000","last_active":"2024-09-21T06:33:47.057685+0000","last_peered":"2024-09-21T06:33:47.057685+0000","last_clean":"2024-09-21T06:33:47.057685+0000","last_became_active":"2024-09-21T06:33:47.051341+0000","last_became_peered":"2024-09-21T06:33:47.051341+0000","last_unstale":"2024-09-21T06:33:47.057685+0000","last_undegraded":"2024-09-21T06:33:47.057685+0000","last_fullsized":"2024-09-21T06:33:47.057685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.067699+0000","last_change":"2024-09-21T06:33:46.064163+0000","last_active":"2024-09-21T06:33:47.067699+0000","last_peered":"2024-09-21T06:33:47.067699+0000","last_clean":"2024-09-21T06:33:47.067699+0000","last_became_active":"2024-09-21T06:33:46.063730+0000","last_became_peered":"2024-09-21T06:33:46.063730+0000","last_unstale":"2024-09-21T06:33:47.067699+0000","last_undegraded":"2024-09-21T06:33:47.067699+0000","last_fullsized":"2024-09-21T06:33:47.067699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:18.525686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072364+0000","last_change":"2024-09-21T06:33:47.067614+0000","last_active":"2024-09-21T06:33:47.072364+0000","last_peered":"2024-09-21T06:33:47.072364+0000","last_clean":"2024-09-21T06:33:47.072364+0000","last_became_active":"2024-09-21T06:33:47.067316+0000","last_became_peered":"2024-09-21T06:33:47.067316+0000","last_unstale":"2024-09-21T06:33:47.072364+0000","last_undegraded":"2024-09-21T06:33:47.072364+0000","last_fullsized":"2024-09-21T06:33:47.072364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:02.117809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.076061+0000","last_change":"2024-09-21T06:33:46.048771+0000","last_active":"2024-09-21T06:33:47.076061+0000","last_peered":"2024-09-21T06:33:47.076061+0000","last_clean":"2024-09-21T06:33:47.076061+0000","last_became_active":"2024-09-21T06:33:46.048425+0000","last_became_peered":"2024-09-21T06:33:46.048425+0000","last_unstale":"2024-09-21T06:33:47.076061+0000","last_undegraded":"2024-09-21T06:33:47.076061+0000","last_fullsized":"2024-09-21T06:33:47.076061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:46.135412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.065054+0000","last_change":"2024-09-21T06:33:47.056092+0000","last_active":"2024-09-21T06:33:47.065054+0000","last_peered":"2024-09-21T06:33:47.065054+0000","last_clean":"2024-09-21T06:33:47.065054+0000","last_became_active":"2024-09-21T06:33:47.055781+0000","last_became_peered":"2024-09-21T06:33:47.055781+0000","last_unstale":"2024-09-21T06:33:47.065054+0000","last_undegraded":"2024-09-21T06:33:47.065054+0000","last_fullsized":"2024-09-21T06:33:47.065054+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:05.180101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.085518+0000","last_change":"2024-09-21T06:33:46.057772+0000","last_active":"2024-09-21T06:33:47.085518+0000","last_peered":"2024-09-21T06:33:47.085518+0000","last_clean":"2024-09-21T06:33:47.085518+0000","last_became_active":"2024-09-21T06:33:46.057449+0000","last_became_peered":"2024-09-21T06:33:46.057449+0000","last_unstale":"2024-09-21T06:33:47.085518+0000","last_undegraded":"2024-09-21T06:33:47.085518+0000","last_fullsized":"2024-09-21T06:33:47.085518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:45.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059144+0000","last_change":"2024-09-21T06:33:47.055473+0000","last_active":"2024-09-21T06:33:47.059144+0000","last_peered":"2024-09-21T06:33:47.059144+0000","last_clean":"2024-09-21T06:33:47.059144+0000","last_became_active":"2024-09-21T06:33:47.055116+0000","last_became_peered":"2024-09-21T06:33:47.055116+0000","last_unstale":"2024-09-21T06:33:47.059144+0000","last_undegraded":"2024-09-21T06:33:47.059144+0000","last_fullsized":"2024-09-21T06:33:47.059144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074965+0000","last_change":"2024-09-21T06:33:46.052421+0000","last_active":"2024-09-21T06:33:47.074965+0000","last_peered":"2024-09-21T06:33:47.074965+0000","last_clean":"2024-09-21T06:33:47.074965+0000","last_became_active":"2024-09-21T06:33:46.052012+0000","last_became_peered":"2024-09-21T06:33:46.052012+0000","last_unstale":"2024-09-21T06:33:47.074965+0000","last_undegraded":"2024-09-21T06:33:47.074965+0000","last_fullsized":"2024-09-21T06:33:47.074965+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:37.302876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059601+0000","last_change":"2024-09-21T06:33:47.052386+0000","last_active":"2024-09-21T06:33:47.059601+0000","last_peered":"2024-09-21T06:33:47.059601+0000","last_clean":"2024-09-21T06:33:47.059601+0000","last_became_active":"2024-09-21T06:33:47.052063+0000","last_became_peered":"2024-09-21T06:33:47.052063+0000","last_unstale":"2024-09-21T06:33:47.059601+0000","last_undegraded":"2024-09-21T06:33:47.059601+0000","last_fullsized":"2024-09-21T06:33:47.059601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:23.247913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.051589+0000","last_change":"2024-09-21T06:33:46.049200+0000","last_active":"2024-09-21T06:33:47.051589+0000","last_peered":"2024-09-21T06:33:47.051589+0000","last_clean":"2024-09-21T06:33:47.051589+0000","last_became_active":"2024-09-21T06:33:46.048872+0000","last_became_peered":"2024-09-21T06:33:46.048872+0000","last_unstale":"2024-09-21T06:33:47.051589+0000","last_undegraded":"2024-09-21T06:33:47.051589+0000","last_fullsized":"2024-09-21T06:33:47.051589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:39.171331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.078616+0000","last_change":"2024-09-21T06:33:47.076673+0000","last_active":"2024-09-21T06:33:47.078616+0000","last_peered":"2024-09-21T06:33:47.078616+0000","last_clean":"2024-09-21T06:33:47.078616+0000","last_became_active":"2024-09-21T06:33:47.076364+0000","last_became_peered":"2024-09-21T06:33:47.076364+0000","last_unstale":"2024-09-21T06:33:47.078616+0000","last_undegraded":"2024-09-21T06:33:47.078616+0000","last_fullsized":"2024-09-21T06:33:47.078616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:45.500543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.053291+0000","last_change":"2024-09-21T06:33:46.048872+0000","last_active":"2024-09-21T06:33:47.053291+0000","last_peered":"2024-09-21T06:33:47.053291+0000","last_clean":"2024-09-21T06:33:47.053291+0000","last_became_active":"2024-09-21T06:33:46.047724+0000","last_became_peered":"2024-09-21T06:33:46.047724+0000","last_unstale":"2024-09-21T06:33:47.053291+0000","last_undegraded":"2024-09-21T06:33:47.053291+0000","last_fullsized":"2024-09-21T06:33:47.053291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:45.182467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375230+0000","last_change":"2024-09-21T06:33:47.055971+0000","last_active":"2024-09-21T06:33:48.375230+0000","last_peered":"2024-09-21T06:33:48.375230+0000","last_clean":"2024-09-21T06:33:48.375230+0000","last_became_active":"2024-09-21T06:33:47.054548+0000","last_became_peered":"2024-09-21T06:33:47.054548+0000","last_unstale":"2024-09-21T06:33:48.375230+0000","last_undegraded":"2024-09-21T06:33:48.375230+0000","last_fullsized":"2024-09-21T06:33:48.375230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:44.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075652+0000","last_change":"2024-09-21T06:33:46.057016+0000","last_active":"2024-09-21T06:33:47.075652+0000","last_peered":"2024-09-21T06:33:47.075652+0000","last_clean":"2024-09-21T06:33:47.075652+0000","last_became_active":"2024-09-21T06:33:46.056558+0000","last_became_peered":"2024-09-21T06:33:46.056558+0000","last_unstale":"2024-09-21T06:33:47.075652+0000","last_undegraded":"2024-09-21T06:33:47.075652+0000","last_fullsized":"2024-09-21T06:33:47.075652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.193585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375300+0000","last_change":"2024-09-21T06:33:47.054185+0000","last_active":"2024-09-21T06:33:48.375300+0000","last_peered":"2024-09-21T06:33:48.375300+0000","last_clean":"2024-09-21T06:33:48.375300+0000","last_became_active":"2024-09-21T06:33:47.053781+0000","last_became_peered":"2024-09-21T06:33:47.053781+0000","last_unstale":"2024-09-21T06:33:48.375300+0000","last_undegraded":"2024-09-21T06:33:48.375300+0000","last_fullsized":"2024-09-21T06:33:48.375300+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.545044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080692+0000","last_change":"2024-09-21T06:33:46.065588+0000","last_active":"2024-09-21T06:33:47.080692+0000","last_peered":"2024-09-21T06:33:47.080692+0000","last_clean":"2024-09-21T06:33:47.080692+0000","last_became_active":"2024-09-21T06:33:46.065238+0000","last_became_peered":"2024-09-21T06:33:46.065238+0000","last_unstale":"2024-09-21T06:33:47.080692+0000","last_undegraded":"2024-09-21T06:33:47.080692+0000","last_fullsized":"2024-09-21T06:33:47.080692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:45.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059984+0000","last_change":"2024-09-21T06:33:47.055606+0000","last_active":"2024-09-21T06:33:47.059984+0000","last_peered":"2024-09-21T06:33:47.059984+0000","last_clean":"2024-09-21T06:33:47.059984+0000","last_became_active":"2024-09-21T06:33:47.054352+0000","last_became_peered":"2024-09-21T06:33:47.054352+0000","last_unstale":"2024-09-21T06:33:47.059984+0000","last_undegraded":"2024-09-21T06:33:47.059984+0000","last_fullsized":"2024-09-21T06:33:47.059984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:22.767018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.066955+0000","last_change":"2024-09-21T06:33:46.064273+0000","last_active":"2024-09-21T06:33:47.066955+0000","last_peered":"2024-09-21T06:33:47.066955+0000","last_clean":"2024-09-21T06:33:47.066955+0000","last_became_active":"2024-09-21T06:33:46.063891+0000","last_became_peered":"2024-09-21T06:33:46.063891+0000","last_unstale":"2024-09-21T06:33:47.066955+0000","last_undegraded":"2024-09-21T06:33:47.066955+0000","last_fullsized":"2024-09-21T06:33:47.066955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:33.056388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075223+0000","last_change":"2024-09-21T06:33:47.067310+0000","last_active":"2024-09-21T06:33:47.075223+0000","last_peered":"2024-09-21T06:33:47.075223+0000","last_clean":"2024-09-21T06:33:47.075223+0000","last_became_active":"2024-09-21T06:33:47.066992+0000","last_became_peered":"2024-09-21T06:33:47.066992+0000","last_unstale":"2024-09-21T06:33:47.075223+0000","last_undegraded":"2024-09-21T06:33:47.075223+0000","last_fullsized":"2024-09-21T06:33:47.075223+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:46.313895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075304+0000","last_change":"2024-09-21T06:33:46.058058+0000","last_active":"2024-09-21T06:33:47.075304+0000","last_peered":"2024-09-21T06:33:47.075304+0000","last_clean":"2024-09-21T06:33:47.075304+0000","last_became_active":"2024-09-21T06:33:46.057732+0000","last_became_peered":"2024-09-21T06:33:46.057732+0000","last_unstale":"2024-09-21T06:33:47.075304+0000","last_undegraded":"2024-09-21T06:33:47.075304+0000","last_fullsized":"2024-09-21T06:33:47.075304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:36.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060876+0000","last_change":"2024-09-21T06:33:47.052967+0000","last_active":"2024-09-21T06:33:47.060876+0000","last_peered":"2024-09-21T06:33:47.060876+0000","last_clean":"2024-09-21T06:33:47.060876+0000","last_became_active":"2024-09-21T06:33:47.049730+0000","last_became_peered":"2024-09-21T06:33:47.049730+0000","last_unstale":"2024-09-21T06:33:47.060876+0000","last_undegraded":"2024-09-21T06:33:47.060876+0000","last_fullsized":"2024-09-21T06:33:47.060876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:15.919789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074748+0000","last_change":"2024-09-21T06:33:46.056268+0000","last_active":"2024-09-21T06:33:47.074748+0000","last_peered":"2024-09-21T06:33:47.074748+0000","last_clean":"2024-09-21T06:33:47.074748+0000","last_became_active":"2024-09-21T06:33:46.055972+0000","last_became_peered":"2024-09-21T06:33:46.055972+0000","last_unstale":"2024-09-21T06:33:47.074748+0000","last_undegraded":"2024-09-21T06:33:47.074748+0000","last_fullsized":"2024-09-21T06:33:47.074748+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:22.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069213+0000","last_change":"2024-09-21T06:33:47.064498+0000","last_active":"2024-09-21T06:33:47.069213+0000","last_peered":"2024-09-21T06:33:47.069213+0000","last_clean":"2024-09-21T06:33:47.069213+0000","last_became_active":"2024-09-21T06:33:47.064177+0000","last_became_peered":"2024-09-21T06:33:47.064177+0000","last_unstale":"2024-09-21T06:33:47.069213+0000","last_undegraded":"2024-09-21T06:33:47.069213+0000","last_fullsized":"2024-09-21T06:33:47.069213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:07.197873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.039436+0000","last_change":"2024-09-21T06:33:46.048786+0000","last_active":"2024-09-21T06:33:47.039436+0000","last_peered":"2024-09-21T06:33:47.039436+0000","last_clean":"2024-09-21T06:33:47.039436+0000","last_became_active":"2024-09-21T06:33:46.048476+0000","last_became_peered":"2024-09-21T06:33:46.048476+0000","last_unstale":"2024-09-21T06:33:47.039436+0000","last_undegraded":"2024-09-21T06:33:47.039436+0000","last_fullsized":"2024-09-21T06:33:47.039436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:40.162518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082343+0000","last_change":"2024-09-21T06:33:47.077204+0000","last_active":"2024-09-21T06:33:47.082343+0000","last_peered":"2024-09-21T06:33:47.082343+0000","last_clean":"2024-09-21T06:33:47.082343+0000","last_became_active":"2024-09-21T06:33:47.076903+0000","last_became_peered":"2024-09-21T06:33:47.076903+0000","last_unstale":"2024-09-21T06:33:47.082343+0000","last_undegraded":"2024-09-21T06:33:47.082343+0000","last_fullsized":"2024-09-21T06:33:47.082343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.042506+0000","last_change":"2024-09-21T06:33:46.040890+0000","last_active":"2024-09-21T06:33:47.042506+0000","last_peered":"2024-09-21T06:33:47.042506+0000","last_clean":"2024-09-21T06:33:47.042506+0000","last_became_active":"2024-09-21T06:33:46.040561+0000","last_became_peered":"2024-09-21T06:33:46.040561+0000","last_unstale":"2024-09-21T06:33:47.042506+0000","last_undegraded":"2024-09-21T06:33:47.042506+0000","last_fullsized":"2024-09-21T06:33:47.042506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:25.724373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072834+0000","last_change":"2024-09-21T06:33:47.068301+0000","last_active":"2024-09-21T06:33:47.072834+0000","last_peered":"2024-09-21T06:33:47.072834+0000","last_clean":"2024-09-21T06:33:47.072834+0000","last_became_active":"2024-09-21T06:33:47.067884+0000","last_became_peered":"2024-09-21T06:33:47.067884+0000","last_unstale":"2024-09-21T06:33:47.072834+0000","last_undegraded":"2024-09-21T06:33:47.072834+0000","last_fullsized":"2024-09-21T06:33:47.072834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:29.206815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374713+0000","last_change":"2024-09-21T06:33:46.046363+0000","last_active":"2024-09-21T06:33:48.374713+0000","last_peered":"2024-09-21T06:33:48.374713+0000","last_clean":"2024-09-21T06:33:48.374713+0000","last_became_active":"2024-09-21T06:33:46.045396+0000","last_became_peered":"2024-09-21T06:33:46.045396+0000","last_unstale":"2024-09-21T06:33:48.374713+0000","last_undegraded":"2024-09-21T06:33:48.374713+0000","last_fullsized":"2024-09-21T06:33:48.374713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.700884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082026+0000","last_change":"2024-09-21T06:33:47.068955+0000","last_active":"2024-09-21T06:33:47.082026+0000","last_peered":"2024-09-21T06:33:47.082026+0000","last_clean":"2024-09-21T06:33:47.082026+0000","last_became_active":"2024-09-21T06:33:47.068597+0000","last_became_peered":"2024-09-21T06:33:47.068597+0000","last_unstale":"2024-09-21T06:33:47.082026+0000","last_undegraded":"2024-09-21T06:33:47.082026+0000","last_fullsized":"2024-09-21T06:33:47.082026+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.078044+0000","last_change":"2024-09-21T06:33:46.055047+0000","last_active":"2024-09-21T06:33:47.078044+0000","last_peered":"2024-09-21T06:33:47.078044+0000","last_clean":"2024-09-21T06:33:47.078044+0000","last_became_active":"2024-09-21T06:33:46.049265+0000","last_became_peered":"2024-09-21T06:33:46.049265+0000","last_unstale":"2024-09-21T06:33:47.078044+0000","last_undegraded":"2024-09-21T06:33:47.078044+0000","last_fullsized":"2024-09-21T06:33:47.078044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:18.427398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375294+0000","last_change":"2024-09-21T06:33:47.054767+0000","last_active":"2024-09-21T06:33:48.375294+0000","last_peered":"2024-09-21T06:33:48.375294+0000","last_clean":"2024-09-21T06:33:48.375294+0000","last_became_active":"2024-09-21T06:33:47.053368+0000","last_became_peered":"2024-09-21T06:33:47.053368+0000","last_unstale":"2024-09-21T06:33:48.375294+0000","last_undegraded":"2024-09-21T06:33:48.375294+0000","last_fullsized":"2024-09-21T06:33:48.375294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.193300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.038445+0000","last_change":"2024-09-21T06:33:46.062160+0000","last_active":"2024-09-21T06:33:47.038445+0000","last_peered":"2024-09-21T06:33:47.038445+0000","last_clean":"2024-09-21T06:33:47.038445+0000","last_became_active":"2024-09-21T06:33:46.061856+0000","last_became_peered":"2024-09-21T06:33:46.061856+0000","last_unstale":"2024-09-21T06:33:47.038445+0000","last_undegraded":"2024-09-21T06:33:47.038445+0000","last_fullsized":"2024-09-21T06:33:47.038445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:04.971480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375350+0000","last_change":"2024-09-21T06:33:47.057823+0000","last_active":"2024-09-21T06:33:48.375350+0000","last_peered":"2024-09-21T06:33:48.375350+0000","last_clean":"2024-09-21T06:33:48.375350+0000","last_became_active":"2024-09-21T06:33:47.057456+0000","last_became_peered":"2024-09-21T06:33:47.057456+0000","last_unstale":"2024-09-21T06:33:48.375350+0000","last_undegraded":"2024-09-21T06:33:48.375350+0000","last_fullsized":"2024-09-21T06:33:48.375350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:47.539777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.047504+0000","last_change":"2024-09-21T06:33:46.054337+0000","last_active":"2024-09-21T06:33:47.047504+0000","last_peered":"2024-09-21T06:33:47.047504+0000","last_clean":"2024-09-21T06:33:47.047504+0000","last_became_active":"2024-09-21T06:33:46.053991+0000","last_became_peered":"2024-09-21T06:33:46.053991+0000","last_unstale":"2024-09-21T06:33:47.047504+0000","last_undegraded":"2024-09-21T06:33:47.047504+0000","last_fullsized":"2024-09-21T06:33:47.047504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.757561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072637+0000","last_change":"2024-09-21T06:33:47.063064+0000","last_active":"2024-09-21T06:33:47.072637+0000","last_peered":"2024-09-21T06:33:47.072637+0000","last_clean":"2024-09-21T06:33:47.072637+0000","last_became_active":"2024-09-21T06:33:47.062642+0000","last_became_peered":"2024-09-21T06:33:47.062642+0000","last_unstale":"2024-09-21T06:33:47.072637+0000","last_undegraded":"2024-09-21T06:33:47.072637+0000","last_fullsized":"2024-09-21T06:33:47.072637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.821183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.036824+0000","last_change":"2024-09-21T06:33:46.054125+0000","last_active":"2024-09-21T06:33:47.036824+0000","last_peered":"2024-09-21T06:33:47.036824+0000","last_clean":"2024-09-21T06:33:47.036824+0000","last_became_active":"2024-09-21T06:33:46.053836+0000","last_became_peered":"2024-09-21T06:33:46.053836+0000","last_unstale":"2024-09-21T06:33:47.036824+0000","last_undegraded":"2024-09-21T06:33:47.036824+0000","last_fullsized":"2024-09-21T06:33:47.036824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:26.637396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046083+0000","last_change":"2024-09-21T06:33:47.044583+0000","last_active":"2024-09-21T06:33:47.046083+0000","last_peered":"2024-09-21T06:33:47.046083+0000","last_clean":"2024-09-21T06:33:47.046083+0000","last_became_active":"2024-09-21T06:33:47.044317+0000","last_became_peered":"2024-09-21T06:33:47.044317+0000","last_unstale":"2024-09-21T06:33:47.046083+0000","last_undegraded":"2024-09-21T06:33:47.046083+0000","last_fullsized":"2024-09-21T06:33:47.046083+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:18.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.053607+0000","last_change":"2024-09-21T06:33:46.053489+0000","last_active":"2024-09-21T06:33:47.053607+0000","last_peered":"2024-09-21T06:33:47.053607+0000","last_clean":"2024-09-21T06:33:47.053607+0000","last_became_active":"2024-09-21T06:33:46.052998+0000","last_became_peered":"2024-09-21T06:33:46.052998+0000","last_unstale":"2024-09-21T06:33:47.053607+0000","last_undegraded":"2024-09-21T06:33:47.053607+0000","last_fullsized":"2024-09-21T06:33:47.053607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:15.962721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082407+0000","last_change":"2024-09-21T06:33:47.078829+0000","last_active":"2024-09-21T06:33:47.082407+0000","last_peered":"2024-09-21T06:33:47.082407+0000","last_clean":"2024-09-21T06:33:47.082407+0000","last_became_active":"2024-09-21T06:33:47.078448+0000","last_became_peered":"2024-09-21T06:33:47.078448+0000","last_unstale":"2024-09-21T06:33:47.082407+0000","last_undegraded":"2024-09-21T06:33:47.082407+0000","last_fullsized":"2024-09-21T06:33:47.082407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:29.113451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.047828+0000","last_change":"2024-09-21T06:33:46.061868+0000","last_active":"2024-09-21T06:33:47.047828+0000","last_peered":"2024-09-21T06:33:47.047828+0000","last_clean":"2024-09-21T06:33:47.047828+0000","last_became_active":"2024-09-21T06:33:46.061388+0000","last_became_peered":"2024-09-21T06:33:46.061388+0000","last_unstale":"2024-09-21T06:33:47.047828+0000","last_undegraded":"2024-09-21T06:33:47.047828+0000","last_fullsized":"2024-09-21T06:33:47.047828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:02.320057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.055932+0000","last_change":"2024-09-21T06:33:47.049373+0000","last_active":"2024-09-21T06:33:47.055932+0000","last_peered":"2024-09-21T06:33:47.055932+0000","last_clean":"2024-09-21T06:33:47.055932+0000","last_became_active":"2024-09-21T06:33:47.048946+0000","last_became_peered":"2024-09-21T06:33:47.048946+0000","last_unstale":"2024-09-21T06:33:47.055932+0000","last_undegraded":"2024-09-21T06:33:47.055932+0000","last_fullsized":"2024-09-21T06:33:47.055932+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:40.174249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.048169+0000","last_change":"2024-09-21T06:33:46.061024+0000","last_active":"2024-09-21T06:33:47.048169+0000","last_peered":"2024-09-21T06:33:47.048169+0000","last_clean":"2024-09-21T06:33:47.048169+0000","last_became_active":"2024-09-21T06:33:46.060765+0000","last_became_peered":"2024-09-21T06:33:46.060765+0000","last_unstale":"2024-09-21T06:33:47.048169+0000","last_undegraded":"2024-09-21T06:33:47.048169+0000","last_fullsized":"2024-09-21T06:33:47.048169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:22.062567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077155+0000","last_change":"2024-09-21T06:33:47.066496+0000","last_active":"2024-09-21T06:33:47.077155+0000","last_peered":"2024-09-21T06:33:47.077155+0000","last_clean":"2024-09-21T06:33:47.077155+0000","last_became_active":"2024-09-21T06:33:47.065685+0000","last_became_peered":"2024-09-21T06:33:47.065685+0000","last_unstale":"2024-09-21T06:33:47.077155+0000","last_undegraded":"2024-09-21T06:33:47.077155+0000","last_fullsized":"2024-09-21T06:33:47.077155+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:31.908193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046601+0000","last_change":"2024-09-21T06:33:46.049249+0000","last_active":"2024-09-21T06:33:47.046601+0000","last_peered":"2024-09-21T06:33:47.046601+0000","last_clean":"2024-09-21T06:33:47.046601+0000","last_became_active":"2024-09-21T06:33:46.048712+0000","last_became_peered":"2024-09-21T06:33:46.048712+0000","last_unstale":"2024-09-21T06:33:47.046601+0000","last_undegraded":"2024-09-21T06:33:47.046601+0000","last_fullsized":"2024-09-21T06:33:47.046601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:17.029566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075092+0000","last_change":"2024-09-21T06:33:47.065102+0000","last_active":"2024-09-21T06:33:47.075092+0000","last_peered":"2024-09-21T06:33:47.075092+0000","last_clean":"2024-09-21T06:33:47.075092+0000","last_became_active":"2024-09-21T06:33:47.064778+0000","last_became_peered":"2024-09-21T06:33:47.064778+0000","last_unstale":"2024-09-21T06:33:47.075092+0000","last_undegraded":"2024-09-21T06:33:47.075092+0000","last_fullsized":"2024-09-21T06:33:47.075092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:09.587111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.051129+0000","last_change":"2024-09-21T06:33:46.057731+0000","last_active":"2024-09-21T06:33:47.051129+0000","last_peered":"2024-09-21T06:33:47.051129+0000","last_clean":"2024-09-21T06:33:47.051129+0000","last_became_active":"2024-09-21T06:33:46.057316+0000","last_became_peered":"2024-09-21T06:33:46.057316+0000","last_unstale":"2024-09-21T06:33:47.051129+0000","last_undegraded":"2024-09-21T06:33:47.051129+0000","last_fullsized":"2024-09-21T06:33:47.051129+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.757561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046674+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:33:47.046674+0000","last_peered":"2024-09-21T06:33:47.046674+0000","last_clean":"2024-09-21T06:33:47.046674+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:33:47.046674+0000","last_undegraded":"2024-09-21T06:33:47.046674+0000","last_fullsized":"2024-09-21T06:33:47.046674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":506461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509515,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":508197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":506461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":508185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1110016,"data_stored":507065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509517,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1097515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":506531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":1103521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:33:51.534 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:33:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 -- 172.21.15.37:0/2550944143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a406e160 msgr2=0x7fd8a406e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 --2- 172.21.15.37:0/2550944143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a406e160 0x7fd8a406e570 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005fd0 tx=0x7fd88c02a540 comp rx=0 tx=0).stop 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 -- 172.21.15.37:0/2550944143 shutdown_connections 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 --2- 172.21.15.37:0/2550944143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a406e160 0x7fd8a406e570 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 --2- 172.21.15.37:0/2550944143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 -- 172.21.15.37:0/2550944143 >> 172.21.15.37:0/2550944143 conn(0x7fd8a406f940 msgr2=0x7fd8a40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 -- 172.21.15.37:0/2550944143 shutdown_connections 2024-09-21T06:33:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.795+0000 7fd8aa000640 1 -- 172.21.15.37:0/2550944143 wait complete. 2024-09-21T06:33:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.797+0000 7fd8aa000640 1 Processor -- start 2024-09-21T06:33:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.797+0000 7fd8aa000640 1 -- start start 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 0x7fd8a4133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a40b4c50 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a40b4ad0 con 0x7fd8a406e160 2024-09-21T06:33:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8aa000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a40b4dd0 con 0x7fd8a415dfe0 2024-09-21T06:33:51.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd89be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd89be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39420/0 (socket says 172.21.15.37:39420) 2024-09-21T06:33:51.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd89be00640 1 -- 172.21.15.37:0/3710410665 learned_addr learned my addr 172.21.15.37:0/3710410665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.799+0000 7fd8a9600640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a8c00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 0x7fd8a4133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 -- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 msgr2=0x7fd8a4156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 -- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 msgr2=0x7fd8a4133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 0x7fd8a4133780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 -- 172.21.15.37:0/3710410665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a41f6e50 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a9600640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:51.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a8c00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 0x7fd8a4133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd89be00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd88c034040 tx=0x7fd88c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c03e030 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd88c003e00 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a41f7130 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c035040 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.800+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a41f76f0 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.801+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a41f6bc0 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.803+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd88c041030 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.804+0000 7fd8a1600640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 0x7fd88007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:51.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.804+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd88c0bb750 con 0x7fd8a40cc8d0 2024-09-21T06:33:51.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.804+0000 7fd8a8c00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 0x7fd88007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:51.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.805+0000 7fd8a8c00640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 0x7fd88007b0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd894009aa0 tx=0x7fd894005db0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:51.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:51.808+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd88c083960 con 0x7fd8a40cc8d0 2024-09-21T06:33:52.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.004+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8a40b52f0 con 0x7fd880078bc0 2024-09-21T06:33:52.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.009+0000 7fd8a1600640 1 -- 172.21.15.37:0/3710410665 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7fd8a40b52f0 con 0x7fd880078bc0 2024-09-21T06:33:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:52.014 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 msgr2=0x7fd88007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 0x7fd88007b0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd894009aa0 tx=0x7fd894005db0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 msgr2=0x7fd8a4155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd88c034040 tx=0x7fd88c003910 comp rx=0 tx=0).stop 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 shutdown_connections 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd880078bc0 0x7fd88007b0d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a40cc8d0 0x7fd8a4155b10 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8a415dfe0 0x7fd8a4156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 --2- 172.21.15.37:0/3710410665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a406e160 0x7fd8a4133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 >> 172.21.15.37:0/3710410665 conn(0x7fd8a406f940 msgr2=0x7fd8a40cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 shutdown_connections 2024-09-21T06:33:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.013+0000 7fd8aa000640 1 -- 172.21.15.37:0/3710410665 wait complete. 2024-09-21T06:33:52.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-21T06:33:51.185402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259368,"kb_used_data":9844,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715352,"statfs":{"total":773094113280,"available":772828520448,"internally_reserved":0,"allocated":10080256,"data_stored":5243936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001473"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058080+0000","last_change":"2024-09-21T06:33:47.049389+0000","last_active":"2024-09-21T06:33:47.058080+0000","last_peered":"2024-09-21T06:33:47.058080+0000","last_clean":"2024-09-21T06:33:47.058080+0000","last_became_active":"2024-09-21T06:33:47.048667+0000","last_became_peered":"2024-09-21T06:33:47.048667+0000","last_unstale":"2024-09-21T06:33:47.058080+0000","last_undegraded":"2024-09-21T06:33:47.058080+0000","last_fullsized":"2024-09-21T06:33:47.058080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:09.587111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077491+0000","last_change":"2024-09-21T06:33:47.072197+0000","last_active":"2024-09-21T06:33:47.077491+0000","last_peered":"2024-09-21T06:33:47.077491+0000","last_clean":"2024-09-21T06:33:47.077491+0000","last_became_active":"2024-09-21T06:33:47.071516+0000","last_became_peered":"2024-09-21T06:33:47.071516+0000","last_unstale":"2024-09-21T06:33:47.077491+0000","last_undegraded":"2024-09-21T06:33:47.077491+0000","last_fullsized":"2024-09-21T06:33:47.077491+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083356+0000","last_change":"2024-09-21T06:33:47.080958+0000","last_active":"2024-09-21T06:33:47.083356+0000","last_peered":"2024-09-21T06:33:47.083356+0000","last_clean":"2024-09-21T06:33:47.083356+0000","last_became_active":"2024-09-21T06:33:47.080628+0000","last_became_peered":"2024-09-21T06:33:47.080628+0000","last_unstale":"2024-09-21T06:33:47.083356+0000","last_undegraded":"2024-09-21T06:33:47.083356+0000","last_fullsized":"2024-09-21T06:33:47.083356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:26.186706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083199+0000","last_change":"2024-09-21T06:33:47.079500+0000","last_active":"2024-09-21T06:33:47.083199+0000","last_peered":"2024-09-21T06:33:47.083199+0000","last_clean":"2024-09-21T06:33:47.083199+0000","last_became_active":"2024-09-21T06:33:47.078806+0000","last_became_peered":"2024-09-21T06:33:47.078806+0000","last_unstale":"2024-09-21T06:33:47.083199+0000","last_undegraded":"2024-09-21T06:33:47.083199+0000","last_fullsized":"2024-09-21T06:33:47.083199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:22.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075024+0000","last_change":"2024-09-21T06:33:47.067426+0000","last_active":"2024-09-21T06:33:47.075024+0000","last_peered":"2024-09-21T06:33:47.075024+0000","last_clean":"2024-09-21T06:33:47.075024+0000","last_became_active":"2024-09-21T06:33:47.066970+0000","last_became_peered":"2024-09-21T06:33:47.066970+0000","last_unstale":"2024-09-21T06:33:47.075024+0000","last_undegraded":"2024-09-21T06:33:47.075024+0000","last_fullsized":"2024-09-21T06:33:47.075024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:34.760478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069828+0000","last_change":"2024-09-21T06:33:47.062230+0000","last_active":"2024-09-21T06:33:47.069828+0000","last_peered":"2024-09-21T06:33:47.069828+0000","last_clean":"2024-09-21T06:33:47.069828+0000","last_became_active":"2024-09-21T06:33:47.060278+0000","last_became_peered":"2024-09-21T06:33:47.060278+0000","last_unstale":"2024-09-21T06:33:47.069828+0000","last_undegraded":"2024-09-21T06:33:47.069828+0000","last_fullsized":"2024-09-21T06:33:47.069828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:05.599195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079824+0000","last_change":"2024-09-21T06:33:47.076019+0000","last_active":"2024-09-21T06:33:47.079824+0000","last_peered":"2024-09-21T06:33:47.079824+0000","last_clean":"2024-09-21T06:33:47.079824+0000","last_became_active":"2024-09-21T06:33:47.075274+0000","last_became_peered":"2024-09-21T06:33:47.075274+0000","last_unstale":"2024-09-21T06:33:47.079824+0000","last_undegraded":"2024-09-21T06:33:47.079824+0000","last_fullsized":"2024-09-21T06:33:47.079824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.454721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060849+0000","last_change":"2024-09-21T06:33:47.048923+0000","last_active":"2024-09-21T06:33:47.060849+0000","last_peered":"2024-09-21T06:33:47.060849+0000","last_clean":"2024-09-21T06:33:47.060849+0000","last_became_active":"2024-09-21T06:33:47.047971+0000","last_became_peered":"2024-09-21T06:33:47.047971+0000","last_unstale":"2024-09-21T06:33:47.060849+0000","last_undegraded":"2024-09-21T06:33:47.060849+0000","last_fullsized":"2024-09-21T06:33:47.060849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:50.754265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.071666+0000","last_change":"2024-09-21T06:33:47.061055+0000","last_active":"2024-09-21T06:33:47.071666+0000","last_peered":"2024-09-21T06:33:47.071666+0000","last_clean":"2024-09-21T06:33:47.071666+0000","last_became_active":"2024-09-21T06:33:47.060703+0000","last_became_peered":"2024-09-21T06:33:47.060703+0000","last_unstale":"2024-09-21T06:33:47.071666+0000","last_undegraded":"2024-09-21T06:33:47.071666+0000","last_fullsized":"2024-09-21T06:33:47.071666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.964035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083288+0000","last_change":"2024-09-21T06:33:47.080473+0000","last_active":"2024-09-21T06:33:47.083288+0000","last_peered":"2024-09-21T06:33:47.083288+0000","last_clean":"2024-09-21T06:33:47.083288+0000","last_became_active":"2024-09-21T06:33:47.080163+0000","last_became_peered":"2024-09-21T06:33:47.080163+0000","last_unstale":"2024-09-21T06:33:47.083288+0000","last_undegraded":"2024-09-21T06:33:47.083288+0000","last_fullsized":"2024-09-21T06:33:47.083288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:07.512354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080048+0000","last_change":"2024-09-21T06:33:47.076242+0000","last_active":"2024-09-21T06:33:47.080048+0000","last_peered":"2024-09-21T06:33:47.080048+0000","last_clean":"2024-09-21T06:33:47.080048+0000","last_became_active":"2024-09-21T06:33:47.075589+0000","last_became_peered":"2024-09-21T06:33:47.075589+0000","last_unstale":"2024-09-21T06:33:47.080048+0000","last_undegraded":"2024-09-21T06:33:47.080048+0000","last_fullsized":"2024-09-21T06:33:47.080048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:06.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.081178+0000","last_change":"2024-09-21T06:33:47.078321+0000","last_active":"2024-09-21T06:33:47.081178+0000","last_peered":"2024-09-21T06:33:47.081178+0000","last_clean":"2024-09-21T06:33:47.081178+0000","last_became_active":"2024-09-21T06:33:47.073777+0000","last_became_peered":"2024-09-21T06:33:47.073777+0000","last_unstale":"2024-09-21T06:33:47.081178+0000","last_undegraded":"2024-09-21T06:33:47.081178+0000","last_fullsized":"2024-09-21T06:33:47.081178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.039561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375004+0000","last_change":"2024-09-21T06:33:47.054106+0000","last_active":"2024-09-21T06:33:48.375004+0000","last_peered":"2024-09-21T06:33:48.375004+0000","last_clean":"2024-09-21T06:33:48.375004+0000","last_became_active":"2024-09-21T06:33:47.052851+0000","last_became_peered":"2024-09-21T06:33:47.052851+0000","last_unstale":"2024-09-21T06:33:48.375004+0000","last_undegraded":"2024-09-21T06:33:48.375004+0000","last_fullsized":"2024-09-21T06:33:48.375004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:00.605543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077519+0000","last_change":"2024-09-21T06:33:47.073657+0000","last_active":"2024-09-21T06:33:47.077519+0000","last_peered":"2024-09-21T06:33:47.077519+0000","last_clean":"2024-09-21T06:33:47.077519+0000","last_became_active":"2024-09-21T06:33:47.070661+0000","last_became_peered":"2024-09-21T06:33:47.070661+0000","last_unstale":"2024-09-21T06:33:47.077519+0000","last_undegraded":"2024-09-21T06:33:47.077519+0000","last_fullsized":"2024-09-21T06:33:47.077519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:12.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083400+0000","last_change":"2024-09-21T06:33:47.081483+0000","last_active":"2024-09-21T06:33:47.083400+0000","last_peered":"2024-09-21T06:33:47.083400+0000","last_clean":"2024-09-21T06:33:47.083400+0000","last_became_active":"2024-09-21T06:33:47.079784+0000","last_became_peered":"2024-09-21T06:33:47.079784+0000","last_unstale":"2024-09-21T06:33:47.083400+0000","last_undegraded":"2024-09-21T06:33:47.083400+0000","last_fullsized":"2024-09-21T06:33:47.083400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:59.939368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079186+0000","last_change":"2024-09-21T06:33:47.069798+0000","last_active":"2024-09-21T06:33:47.079186+0000","last_peered":"2024-09-21T06:33:47.079186+0000","last_clean":"2024-09-21T06:33:47.079186+0000","last_became_active":"2024-09-21T06:33:47.068713+0000","last_became_peered":"2024-09-21T06:33:47.068713+0000","last_unstale":"2024-09-21T06:33:47.079186+0000","last_undegraded":"2024-09-21T06:33:47.079186+0000","last_fullsized":"2024-09-21T06:33:47.079186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:52.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058548+0000","last_change":"2024-09-21T06:33:47.049006+0000","last_active":"2024-09-21T06:33:47.058548+0000","last_peered":"2024-09-21T06:33:47.058548+0000","last_clean":"2024-09-21T06:33:47.058548+0000","last_became_active":"2024-09-21T06:33:47.048672+0000","last_became_peered":"2024-09-21T06:33:47.048672+0000","last_unstale":"2024-09-21T06:33:47.058548+0000","last_undegraded":"2024-09-21T06:33:47.058548+0000","last_fullsized":"2024-09-21T06:33:47.058548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:59.279272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082983+0000","last_change":"2024-09-21T06:33:47.078853+0000","last_active":"2024-09-21T06:33:47.082983+0000","last_peered":"2024-09-21T06:33:47.082983+0000","last_clean":"2024-09-21T06:33:47.082983+0000","last_became_active":"2024-09-21T06:33:47.078548+0000","last_became_peered":"2024-09-21T06:33:47.078548+0000","last_unstale":"2024-09-21T06:33:47.082983+0000","last_undegraded":"2024-09-21T06:33:47.082983+0000","last_fullsized":"2024-09-21T06:33:47.082983+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:34.981609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082347+0000","last_change":"2024-09-21T06:33:47.077280+0000","last_active":"2024-09-21T06:33:47.082347+0000","last_peered":"2024-09-21T06:33:47.082347+0000","last_clean":"2024-09-21T06:33:47.082347+0000","last_became_active":"2024-09-21T06:33:47.076940+0000","last_became_peered":"2024-09-21T06:33:47.076940+0000","last_unstale":"2024-09-21T06:33:47.082347+0000","last_undegraded":"2024-09-21T06:33:47.082347+0000","last_fullsized":"2024-09-21T06:33:47.082347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:02.117809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060127+0000","last_change":"2024-09-21T06:33:47.051768+0000","last_active":"2024-09-21T06:33:47.060127+0000","last_peered":"2024-09-21T06:33:47.060127+0000","last_clean":"2024-09-21T06:33:47.060127+0000","last_became_active":"2024-09-21T06:33:47.051472+0000","last_became_peered":"2024-09-21T06:33:47.051472+0000","last_unstale":"2024-09-21T06:33:47.060127+0000","last_undegraded":"2024-09-21T06:33:47.060127+0000","last_fullsized":"2024-09-21T06:33:47.060127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:41.422642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074075+0000","last_change":"2024-09-21T06:33:47.062997+0000","last_active":"2024-09-21T06:33:47.074075+0000","last_peered":"2024-09-21T06:33:47.074075+0000","last_clean":"2024-09-21T06:33:47.074075+0000","last_became_active":"2024-09-21T06:33:47.062608+0000","last_became_peered":"2024-09-21T06:33:47.062608+0000","last_unstale":"2024-09-21T06:33:47.074075+0000","last_undegraded":"2024-09-21T06:33:47.074075+0000","last_fullsized":"2024-09-21T06:33:47.074075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:29.364069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.061435+0000","last_change":"2024-09-21T06:33:47.053231+0000","last_active":"2024-09-21T06:33:47.061435+0000","last_peered":"2024-09-21T06:33:47.061435+0000","last_clean":"2024-09-21T06:33:47.061435+0000","last_became_active":"2024-09-21T06:33:47.052877+0000","last_became_peered":"2024-09-21T06:33:47.052877+0000","last_unstale":"2024-09-21T06:33:47.061435+0000","last_undegraded":"2024-09-21T06:33:47.061435+0000","last_fullsized":"2024-09-21T06:33:47.061435+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.185385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077450+0000","last_change":"2024-09-21T06:33:47.069847+0000","last_active":"2024-09-21T06:33:47.077450+0000","last_peered":"2024-09-21T06:33:47.077450+0000","last_clean":"2024-09-21T06:33:47.077450+0000","last_became_active":"2024-09-21T06:33:47.064139+0000","last_became_peered":"2024-09-21T06:33:47.064139+0000","last_unstale":"2024-09-21T06:33:47.077450+0000","last_undegraded":"2024-09-21T06:33:47.077450+0000","last_fullsized":"2024-09-21T06:33:47.077450+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:16.965639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077350+0000","last_change":"2024-09-21T06:33:47.075184+0000","last_active":"2024-09-21T06:33:47.077350+0000","last_peered":"2024-09-21T06:33:47.077350+0000","last_clean":"2024-09-21T06:33:47.077350+0000","last_became_active":"2024-09-21T06:33:47.074823+0000","last_became_peered":"2024-09-21T06:33:47.074823+0000","last_unstale":"2024-09-21T06:33:47.077350+0000","last_undegraded":"2024-09-21T06:33:47.077350+0000","last_fullsized":"2024-09-21T06:33:47.077350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:02.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075134+0000","last_change":"2024-09-21T06:33:47.068701+0000","last_active":"2024-09-21T06:33:47.075134+0000","last_peered":"2024-09-21T06:33:47.075134+0000","last_clean":"2024-09-21T06:33:47.075134+0000","last_became_active":"2024-09-21T06:33:47.067891+0000","last_became_peered":"2024-09-21T06:33:47.067891+0000","last_unstale":"2024-09-21T06:33:47.075134+0000","last_undegraded":"2024-09-21T06:33:47.075134+0000","last_fullsized":"2024-09-21T06:33:47.075134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.185385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077437+0000","last_change":"2024-09-21T06:33:47.071849+0000","last_active":"2024-09-21T06:33:47.077437+0000","last_peered":"2024-09-21T06:33:47.077437+0000","last_clean":"2024-09-21T06:33:47.077437+0000","last_became_active":"2024-09-21T06:33:47.071015+0000","last_became_peered":"2024-09-21T06:33:47.071015+0000","last_unstale":"2024-09-21T06:33:47.077437+0000","last_undegraded":"2024-09-21T06:33:47.077437+0000","last_fullsized":"2024-09-21T06:33:47.077437+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080050+0000","last_change":"2024-09-21T06:33:47.078274+0000","last_active":"2024-09-21T06:33:47.080050+0000","last_peered":"2024-09-21T06:33:47.080050+0000","last_clean":"2024-09-21T06:33:47.080050+0000","last_became_active":"2024-09-21T06:33:47.077743+0000","last_became_peered":"2024-09-21T06:33:47.077743+0000","last_unstale":"2024-09-21T06:33:47.080050+0000","last_undegraded":"2024-09-21T06:33:47.080050+0000","last_fullsized":"2024-09-21T06:33:47.080050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:34.823696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375316+0000","last_change":"2024-09-21T06:33:47.047428+0000","last_active":"2024-09-21T06:33:48.375316+0000","last_peered":"2024-09-21T06:33:48.375316+0000","last_clean":"2024-09-21T06:33:48.375316+0000","last_became_active":"2024-09-21T06:33:47.045850+0000","last_became_peered":"2024-09-21T06:33:47.045850+0000","last_unstale":"2024-09-21T06:33:48.375316+0000","last_undegraded":"2024-09-21T06:33:48.375316+0000","last_fullsized":"2024-09-21T06:33:48.375316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:20.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075051+0000","last_change":"2024-09-21T06:33:47.068008+0000","last_active":"2024-09-21T06:33:47.075051+0000","last_peered":"2024-09-21T06:33:47.075051+0000","last_clean":"2024-09-21T06:33:47.075051+0000","last_became_active":"2024-09-21T06:33:47.067659+0000","last_became_peered":"2024-09-21T06:33:47.067659+0000","last_unstale":"2024-09-21T06:33:47.075051+0000","last_undegraded":"2024-09-21T06:33:47.075051+0000","last_fullsized":"2024-09-21T06:33:47.075051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:40.174249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060753+0000","last_change":"2024-09-21T06:33:47.058440+0000","last_active":"2024-09-21T06:33:47.060753+0000","last_peered":"2024-09-21T06:33:47.060753+0000","last_clean":"2024-09-21T06:33:47.060753+0000","last_became_active":"2024-09-21T06:33:47.057544+0000","last_became_peered":"2024-09-21T06:33:47.057544+0000","last_unstale":"2024-09-21T06:33:47.060753+0000","last_undegraded":"2024-09-21T06:33:47.060753+0000","last_fullsized":"2024-09-21T06:33:47.060753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:42.956401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060678+0000","last_change":"2024-09-21T06:33:47.055177+0000","last_active":"2024-09-21T06:33:47.060678+0000","last_peered":"2024-09-21T06:33:47.060678+0000","last_clean":"2024-09-21T06:33:47.060678+0000","last_became_active":"2024-09-21T06:33:47.054684+0000","last_became_peered":"2024-09-21T06:33:47.054684+0000","last_unstale":"2024-09-21T06:33:47.060678+0000","last_undegraded":"2024-09-21T06:33:47.060678+0000","last_fullsized":"2024-09-21T06:33:47.060678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:16.965639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.076960+0000","last_change":"2024-09-21T06:33:47.069319+0000","last_active":"2024-09-21T06:33:47.076960+0000","last_peered":"2024-09-21T06:33:47.076960+0000","last_clean":"2024-09-21T06:33:47.076960+0000","last_became_active":"2024-09-21T06:33:47.068322+0000","last_became_peered":"2024-09-21T06:33:47.068322+0000","last_unstale":"2024-09-21T06:33:47.076960+0000","last_undegraded":"2024-09-21T06:33:47.076960+0000","last_fullsized":"2024-09-21T06:33:47.076960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:13.321633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.058172+0000","last_change":"2024-09-21T06:33:47.048633+0000","last_active":"2024-09-21T06:33:47.058172+0000","last_peered":"2024-09-21T06:33:47.058172+0000","last_clean":"2024-09-21T06:33:47.058172+0000","last_became_active":"2024-09-21T06:33:47.048307+0000","last_became_peered":"2024-09-21T06:33:47.048307+0000","last_unstale":"2024-09-21T06:33:47.058172+0000","last_undegraded":"2024-09-21T06:33:47.058172+0000","last_fullsized":"2024-09-21T06:33:47.058172+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.821183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083012+0000","last_change":"2024-09-21T06:33:46.061041+0000","last_active":"2024-09-21T06:33:47.083012+0000","last_peered":"2024-09-21T06:33:47.083012+0000","last_clean":"2024-09-21T06:33:47.083012+0000","last_became_active":"2024-09-21T06:33:46.060737+0000","last_became_peered":"2024-09-21T06:33:46.060737+0000","last_unstale":"2024-09-21T06:33:47.083012+0000","last_undegraded":"2024-09-21T06:33:47.083012+0000","last_fullsized":"2024-09-21T06:33:47.083012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.564485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059959+0000","last_change":"2024-09-21T06:33:47.050929+0000","last_active":"2024-09-21T06:33:47.059959+0000","last_peered":"2024-09-21T06:33:47.059959+0000","last_clean":"2024-09-21T06:33:47.059959+0000","last_became_active":"2024-09-21T06:33:47.050633+0000","last_became_peered":"2024-09-21T06:33:47.050633+0000","last_unstale":"2024-09-21T06:33:47.059959+0000","last_undegraded":"2024-09-21T06:33:47.059959+0000","last_fullsized":"2024-09-21T06:33:47.059959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.139550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.073983+0000","last_change":"2024-09-21T06:33:46.052928+0000","last_active":"2024-09-21T06:33:47.073983+0000","last_peered":"2024-09-21T06:33:47.073983+0000","last_clean":"2024-09-21T06:33:47.073983+0000","last_became_active":"2024-09-21T06:33:46.052587+0000","last_became_peered":"2024-09-21T06:33:46.052587+0000","last_unstale":"2024-09-21T06:33:47.073983+0000","last_undegraded":"2024-09-21T06:33:47.073983+0000","last_fullsized":"2024-09-21T06:33:47.073983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:21.388222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.050871+0000","last_change":"2024-09-21T06:33:47.047368+0000","last_active":"2024-09-21T06:33:47.050871+0000","last_peered":"2024-09-21T06:33:47.050871+0000","last_clean":"2024-09-21T06:33:47.050871+0000","last_became_active":"2024-09-21T06:33:47.044714+0000","last_became_peered":"2024-09-21T06:33:47.044714+0000","last_unstale":"2024-09-21T06:33:47.050871+0000","last_undegraded":"2024-09-21T06:33:47.050871+0000","last_fullsized":"2024-09-21T06:33:47.050871+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:58.606357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074023+0000","last_change":"2024-09-21T06:33:46.059829+0000","last_active":"2024-09-21T06:33:47.074023+0000","last_peered":"2024-09-21T06:33:47.074023+0000","last_clean":"2024-09-21T06:33:47.074023+0000","last_became_active":"2024-09-21T06:33:46.059421+0000","last_became_peered":"2024-09-21T06:33:46.059421+0000","last_unstale":"2024-09-21T06:33:47.074023+0000","last_undegraded":"2024-09-21T06:33:47.074023+0000","last_fullsized":"2024-09-21T06:33:47.074023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:53.798551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.083414+0000","last_change":"2024-09-21T06:33:47.082237+0000","last_active":"2024-09-21T06:33:47.083414+0000","last_peered":"2024-09-21T06:33:47.083414+0000","last_clean":"2024-09-21T06:33:47.083414+0000","last_became_active":"2024-09-21T06:33:47.080611+0000","last_became_peered":"2024-09-21T06:33:47.080611+0000","last_unstale":"2024-09-21T06:33:47.083414+0000","last_undegraded":"2024-09-21T06:33:47.083414+0000","last_fullsized":"2024-09-21T06:33:47.083414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:56.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069162+0000","last_change":"2024-09-21T06:33:46.041123+0000","last_active":"2024-09-21T06:33:47.069162+0000","last_peered":"2024-09-21T06:33:47.069162+0000","last_clean":"2024-09-21T06:33:47.069162+0000","last_became_active":"2024-09-21T06:33:46.040811+0000","last_became_peered":"2024-09-21T06:33:46.040811+0000","last_unstale":"2024-09-21T06:33:47.069162+0000","last_undegraded":"2024-09-21T06:33:47.069162+0000","last_fullsized":"2024-09-21T06:33:47.069162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:55.210198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374859+0000","last_change":"2024-09-21T06:33:47.060297+0000","last_active":"2024-09-21T06:33:48.374859+0000","last_peered":"2024-09-21T06:33:48.374859+0000","last_clean":"2024-09-21T06:33:48.374859+0000","last_became_active":"2024-09-21T06:33:47.059069+0000","last_became_peered":"2024-09-21T06:33:47.059069+0000","last_unstale":"2024-09-21T06:33:48.374859+0000","last_undegraded":"2024-09-21T06:33:48.374859+0000","last_fullsized":"2024-09-21T06:33:48.374859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:50.000043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080663+0000","last_change":"2024-09-21T06:33:46.055305+0000","last_active":"2024-09-21T06:33:47.080663+0000","last_peered":"2024-09-21T06:33:47.080663+0000","last_clean":"2024-09-21T06:33:47.080663+0000","last_became_active":"2024-09-21T06:33:46.054987+0000","last_became_peered":"2024-09-21T06:33:46.054987+0000","last_unstale":"2024-09-21T06:33:47.080663+0000","last_undegraded":"2024-09-21T06:33:47.080663+0000","last_fullsized":"2024-09-21T06:33:47.080663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:08.584194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077237+0000","last_change":"2024-09-21T06:33:47.074979+0000","last_active":"2024-09-21T06:33:47.077237+0000","last_peered":"2024-09-21T06:33:47.077237+0000","last_clean":"2024-09-21T06:33:47.077237+0000","last_became_active":"2024-09-21T06:33:47.074640+0000","last_became_peered":"2024-09-21T06:33:47.074640+0000","last_unstale":"2024-09-21T06:33:47.077237+0000","last_undegraded":"2024-09-21T06:33:47.077237+0000","last_fullsized":"2024-09-21T06:33:47.077237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:02.267714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374665+0000","last_change":"2024-09-21T06:33:46.053873+0000","last_active":"2024-09-21T06:33:48.374665+0000","last_peered":"2024-09-21T06:33:48.374665+0000","last_clean":"2024-09-21T06:33:48.374665+0000","last_became_active":"2024-09-21T06:33:46.053464+0000","last_became_peered":"2024-09-21T06:33:46.053464+0000","last_unstale":"2024-09-21T06:33:48.374665+0000","last_undegraded":"2024-09-21T06:33:48.374665+0000","last_fullsized":"2024-09-21T06:33:48.374665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:23.020990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.079085+0000","last_change":"2024-09-21T06:33:47.074330+0000","last_active":"2024-09-21T06:33:47.079085+0000","last_peered":"2024-09-21T06:33:47.079085+0000","last_clean":"2024-09-21T06:33:47.079085+0000","last_became_active":"2024-09-21T06:33:47.073692+0000","last_became_peered":"2024-09-21T06:33:47.073692+0000","last_unstale":"2024-09-21T06:33:47.079085+0000","last_undegraded":"2024-09-21T06:33:47.079085+0000","last_fullsized":"2024-09-21T06:33:47.079085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:23.606097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.071226+0000","last_change":"2024-09-21T06:33:46.063703+0000","last_active":"2024-09-21T06:33:47.071226+0000","last_peered":"2024-09-21T06:33:47.071226+0000","last_clean":"2024-09-21T06:33:47.071226+0000","last_became_active":"2024-09-21T06:33:46.063351+0000","last_became_peered":"2024-09-21T06:33:46.063351+0000","last_unstale":"2024-09-21T06:33:47.071226+0000","last_undegraded":"2024-09-21T06:33:47.071226+0000","last_fullsized":"2024-09-21T06:33:47.071226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:20.775850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.070549+0000","last_change":"2024-09-21T06:33:47.060777+0000","last_active":"2024-09-21T06:33:47.070549+0000","last_peered":"2024-09-21T06:33:47.070549+0000","last_clean":"2024-09-21T06:33:47.070549+0000","last_became_active":"2024-09-21T06:33:47.060472+0000","last_became_peered":"2024-09-21T06:33:47.060472+0000","last_unstale":"2024-09-21T06:33:47.070549+0000","last_undegraded":"2024-09-21T06:33:47.070549+0000","last_fullsized":"2024-09-21T06:33:47.070549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046409+0000","last_change":"2024-09-21T06:33:46.055199+0000","last_active":"2024-09-21T06:33:47.046409+0000","last_peered":"2024-09-21T06:33:47.046409+0000","last_clean":"2024-09-21T06:33:47.046409+0000","last_became_active":"2024-09-21T06:33:46.054875+0000","last_became_peered":"2024-09-21T06:33:46.054875+0000","last_unstale":"2024-09-21T06:33:47.046409+0000","last_undegraded":"2024-09-21T06:33:47.046409+0000","last_fullsized":"2024-09-21T06:33:47.046409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:30.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.066702+0000","last_change":"2024-09-21T06:33:47.061559+0000","last_active":"2024-09-21T06:33:47.066702+0000","last_peered":"2024-09-21T06:33:47.066702+0000","last_clean":"2024-09-21T06:33:47.066702+0000","last_became_active":"2024-09-21T06:33:47.061260+0000","last_became_peered":"2024-09-21T06:33:47.061260+0000","last_unstale":"2024-09-21T06:33:47.066702+0000","last_undegraded":"2024-09-21T06:33:47.066702+0000","last_fullsized":"2024-09-21T06:33:47.066702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:01.659246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.056535+0000","last_change":"2024-09-21T06:33:46.057265+0000","last_active":"2024-09-21T06:33:47.056535+0000","last_peered":"2024-09-21T06:33:47.056535+0000","last_clean":"2024-09-21T06:33:47.056535+0000","last_became_active":"2024-09-21T06:33:46.056922+0000","last_became_peered":"2024-09-21T06:33:46.056922+0000","last_unstale":"2024-09-21T06:33:47.056535+0000","last_undegraded":"2024-09-21T06:33:47.056535+0000","last_fullsized":"2024-09-21T06:33:47.056535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:06.564753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.057685+0000","last_change":"2024-09-21T06:33:47.051711+0000","last_active":"2024-09-21T06:33:47.057685+0000","last_peered":"2024-09-21T06:33:47.057685+0000","last_clean":"2024-09-21T06:33:47.057685+0000","last_became_active":"2024-09-21T06:33:47.051341+0000","last_became_peered":"2024-09-21T06:33:47.051341+0000","last_unstale":"2024-09-21T06:33:47.057685+0000","last_undegraded":"2024-09-21T06:33:47.057685+0000","last_fullsized":"2024-09-21T06:33:47.057685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.067699+0000","last_change":"2024-09-21T06:33:46.064163+0000","last_active":"2024-09-21T06:33:47.067699+0000","last_peered":"2024-09-21T06:33:47.067699+0000","last_clean":"2024-09-21T06:33:47.067699+0000","last_became_active":"2024-09-21T06:33:46.063730+0000","last_became_peered":"2024-09-21T06:33:46.063730+0000","last_unstale":"2024-09-21T06:33:47.067699+0000","last_undegraded":"2024-09-21T06:33:47.067699+0000","last_fullsized":"2024-09-21T06:33:47.067699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:18.525686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072364+0000","last_change":"2024-09-21T06:33:47.067614+0000","last_active":"2024-09-21T06:33:47.072364+0000","last_peered":"2024-09-21T06:33:47.072364+0000","last_clean":"2024-09-21T06:33:47.072364+0000","last_became_active":"2024-09-21T06:33:47.067316+0000","last_became_peered":"2024-09-21T06:33:47.067316+0000","last_unstale":"2024-09-21T06:33:47.072364+0000","last_undegraded":"2024-09-21T06:33:47.072364+0000","last_fullsized":"2024-09-21T06:33:47.072364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:02.117809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.076061+0000","last_change":"2024-09-21T06:33:46.048771+0000","last_active":"2024-09-21T06:33:47.076061+0000","last_peered":"2024-09-21T06:33:47.076061+0000","last_clean":"2024-09-21T06:33:47.076061+0000","last_became_active":"2024-09-21T06:33:46.048425+0000","last_became_peered":"2024-09-21T06:33:46.048425+0000","last_unstale":"2024-09-21T06:33:47.076061+0000","last_undegraded":"2024-09-21T06:33:47.076061+0000","last_fullsized":"2024-09-21T06:33:47.076061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:46.135412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.065054+0000","last_change":"2024-09-21T06:33:47.056092+0000","last_active":"2024-09-21T06:33:47.065054+0000","last_peered":"2024-09-21T06:33:47.065054+0000","last_clean":"2024-09-21T06:33:47.065054+0000","last_became_active":"2024-09-21T06:33:47.055781+0000","last_became_peered":"2024-09-21T06:33:47.055781+0000","last_unstale":"2024-09-21T06:33:47.065054+0000","last_undegraded":"2024-09-21T06:33:47.065054+0000","last_fullsized":"2024-09-21T06:33:47.065054+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:05.180101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.085518+0000","last_change":"2024-09-21T06:33:46.057772+0000","last_active":"2024-09-21T06:33:47.085518+0000","last_peered":"2024-09-21T06:33:47.085518+0000","last_clean":"2024-09-21T06:33:47.085518+0000","last_became_active":"2024-09-21T06:33:46.057449+0000","last_became_peered":"2024-09-21T06:33:46.057449+0000","last_unstale":"2024-09-21T06:33:47.085518+0000","last_undegraded":"2024-09-21T06:33:47.085518+0000","last_fullsized":"2024-09-21T06:33:47.085518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:45.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059144+0000","last_change":"2024-09-21T06:33:47.055473+0000","last_active":"2024-09-21T06:33:47.059144+0000","last_peered":"2024-09-21T06:33:47.059144+0000","last_clean":"2024-09-21T06:33:47.059144+0000","last_became_active":"2024-09-21T06:33:47.055116+0000","last_became_peered":"2024-09-21T06:33:47.055116+0000","last_unstale":"2024-09-21T06:33:47.059144+0000","last_undegraded":"2024-09-21T06:33:47.059144+0000","last_fullsized":"2024-09-21T06:33:47.059144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074965+0000","last_change":"2024-09-21T06:33:46.052421+0000","last_active":"2024-09-21T06:33:47.074965+0000","last_peered":"2024-09-21T06:33:47.074965+0000","last_clean":"2024-09-21T06:33:47.074965+0000","last_became_active":"2024-09-21T06:33:46.052012+0000","last_became_peered":"2024-09-21T06:33:46.052012+0000","last_unstale":"2024-09-21T06:33:47.074965+0000","last_undegraded":"2024-09-21T06:33:47.074965+0000","last_fullsized":"2024-09-21T06:33:47.074965+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:37.302876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059601+0000","last_change":"2024-09-21T06:33:47.052386+0000","last_active":"2024-09-21T06:33:47.059601+0000","last_peered":"2024-09-21T06:33:47.059601+0000","last_clean":"2024-09-21T06:33:47.059601+0000","last_became_active":"2024-09-21T06:33:47.052063+0000","last_became_peered":"2024-09-21T06:33:47.052063+0000","last_unstale":"2024-09-21T06:33:47.059601+0000","last_undegraded":"2024-09-21T06:33:47.059601+0000","last_fullsized":"2024-09-21T06:33:47.059601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:23.247913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.051589+0000","last_change":"2024-09-21T06:33:46.049200+0000","last_active":"2024-09-21T06:33:47.051589+0000","last_peered":"2024-09-21T06:33:47.051589+0000","last_clean":"2024-09-21T06:33:47.051589+0000","last_became_active":"2024-09-21T06:33:46.048872+0000","last_became_peered":"2024-09-21T06:33:46.048872+0000","last_unstale":"2024-09-21T06:33:47.051589+0000","last_undegraded":"2024-09-21T06:33:47.051589+0000","last_fullsized":"2024-09-21T06:33:47.051589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:39.171331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.078616+0000","last_change":"2024-09-21T06:33:47.076673+0000","last_active":"2024-09-21T06:33:47.078616+0000","last_peered":"2024-09-21T06:33:47.078616+0000","last_clean":"2024-09-21T06:33:47.078616+0000","last_became_active":"2024-09-21T06:33:47.076364+0000","last_became_peered":"2024-09-21T06:33:47.076364+0000","last_unstale":"2024-09-21T06:33:47.078616+0000","last_undegraded":"2024-09-21T06:33:47.078616+0000","last_fullsized":"2024-09-21T06:33:47.078616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:45.500543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.053291+0000","last_change":"2024-09-21T06:33:46.048872+0000","last_active":"2024-09-21T06:33:47.053291+0000","last_peered":"2024-09-21T06:33:47.053291+0000","last_clean":"2024-09-21T06:33:47.053291+0000","last_became_active":"2024-09-21T06:33:46.047724+0000","last_became_peered":"2024-09-21T06:33:46.047724+0000","last_unstale":"2024-09-21T06:33:47.053291+0000","last_undegraded":"2024-09-21T06:33:47.053291+0000","last_fullsized":"2024-09-21T06:33:47.053291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:45.182467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375230+0000","last_change":"2024-09-21T06:33:47.055971+0000","last_active":"2024-09-21T06:33:48.375230+0000","last_peered":"2024-09-21T06:33:48.375230+0000","last_clean":"2024-09-21T06:33:48.375230+0000","last_became_active":"2024-09-21T06:33:47.054548+0000","last_became_peered":"2024-09-21T06:33:47.054548+0000","last_unstale":"2024-09-21T06:33:48.375230+0000","last_undegraded":"2024-09-21T06:33:48.375230+0000","last_fullsized":"2024-09-21T06:33:48.375230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:44.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075652+0000","last_change":"2024-09-21T06:33:46.057016+0000","last_active":"2024-09-21T06:33:47.075652+0000","last_peered":"2024-09-21T06:33:47.075652+0000","last_clean":"2024-09-21T06:33:47.075652+0000","last_became_active":"2024-09-21T06:33:46.056558+0000","last_became_peered":"2024-09-21T06:33:46.056558+0000","last_unstale":"2024-09-21T06:33:47.075652+0000","last_undegraded":"2024-09-21T06:33:47.075652+0000","last_fullsized":"2024-09-21T06:33:47.075652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.193585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375300+0000","last_change":"2024-09-21T06:33:47.054185+0000","last_active":"2024-09-21T06:33:48.375300+0000","last_peered":"2024-09-21T06:33:48.375300+0000","last_clean":"2024-09-21T06:33:48.375300+0000","last_became_active":"2024-09-21T06:33:47.053781+0000","last_became_peered":"2024-09-21T06:33:47.053781+0000","last_unstale":"2024-09-21T06:33:48.375300+0000","last_undegraded":"2024-09-21T06:33:48.375300+0000","last_fullsized":"2024-09-21T06:33:48.375300+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.545044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.080692+0000","last_change":"2024-09-21T06:33:46.065588+0000","last_active":"2024-09-21T06:33:47.080692+0000","last_peered":"2024-09-21T06:33:47.080692+0000","last_clean":"2024-09-21T06:33:47.080692+0000","last_became_active":"2024-09-21T06:33:46.065238+0000","last_became_peered":"2024-09-21T06:33:46.065238+0000","last_unstale":"2024-09-21T06:33:47.080692+0000","last_undegraded":"2024-09-21T06:33:47.080692+0000","last_fullsized":"2024-09-21T06:33:47.080692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:45.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.059984+0000","last_change":"2024-09-21T06:33:47.055606+0000","last_active":"2024-09-21T06:33:47.059984+0000","last_peered":"2024-09-21T06:33:47.059984+0000","last_clean":"2024-09-21T06:33:47.059984+0000","last_became_active":"2024-09-21T06:33:47.054352+0000","last_became_peered":"2024-09-21T06:33:47.054352+0000","last_unstale":"2024-09-21T06:33:47.059984+0000","last_undegraded":"2024-09-21T06:33:47.059984+0000","last_fullsized":"2024-09-21T06:33:47.059984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:22.767018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.066955+0000","last_change":"2024-09-21T06:33:46.064273+0000","last_active":"2024-09-21T06:33:47.066955+0000","last_peered":"2024-09-21T06:33:47.066955+0000","last_clean":"2024-09-21T06:33:47.066955+0000","last_became_active":"2024-09-21T06:33:46.063891+0000","last_became_peered":"2024-09-21T06:33:46.063891+0000","last_unstale":"2024-09-21T06:33:47.066955+0000","last_undegraded":"2024-09-21T06:33:47.066955+0000","last_fullsized":"2024-09-21T06:33:47.066955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:33.056388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075223+0000","last_change":"2024-09-21T06:33:47.067310+0000","last_active":"2024-09-21T06:33:47.075223+0000","last_peered":"2024-09-21T06:33:47.075223+0000","last_clean":"2024-09-21T06:33:47.075223+0000","last_became_active":"2024-09-21T06:33:47.066992+0000","last_became_peered":"2024-09-21T06:33:47.066992+0000","last_unstale":"2024-09-21T06:33:47.075223+0000","last_undegraded":"2024-09-21T06:33:47.075223+0000","last_fullsized":"2024-09-21T06:33:47.075223+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:46.313895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075304+0000","last_change":"2024-09-21T06:33:46.058058+0000","last_active":"2024-09-21T06:33:47.075304+0000","last_peered":"2024-09-21T06:33:47.075304+0000","last_clean":"2024-09-21T06:33:47.075304+0000","last_became_active":"2024-09-21T06:33:46.057732+0000","last_became_peered":"2024-09-21T06:33:46.057732+0000","last_unstale":"2024-09-21T06:33:47.075304+0000","last_undegraded":"2024-09-21T06:33:47.075304+0000","last_fullsized":"2024-09-21T06:33:47.075304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:36.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.060876+0000","last_change":"2024-09-21T06:33:47.052967+0000","last_active":"2024-09-21T06:33:47.060876+0000","last_peered":"2024-09-21T06:33:47.060876+0000","last_clean":"2024-09-21T06:33:47.060876+0000","last_became_active":"2024-09-21T06:33:47.049730+0000","last_became_peered":"2024-09-21T06:33:47.049730+0000","last_unstale":"2024-09-21T06:33:47.060876+0000","last_undegraded":"2024-09-21T06:33:47.060876+0000","last_fullsized":"2024-09-21T06:33:47.060876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:15.919789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.074748+0000","last_change":"2024-09-21T06:33:46.056268+0000","last_active":"2024-09-21T06:33:47.074748+0000","last_peered":"2024-09-21T06:33:47.074748+0000","last_clean":"2024-09-21T06:33:47.074748+0000","last_became_active":"2024-09-21T06:33:46.055972+0000","last_became_peered":"2024-09-21T06:33:46.055972+0000","last_unstale":"2024-09-21T06:33:47.074748+0000","last_undegraded":"2024-09-21T06:33:47.074748+0000","last_fullsized":"2024-09-21T06:33:47.074748+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:22.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.069213+0000","last_change":"2024-09-21T06:33:47.064498+0000","last_active":"2024-09-21T06:33:47.069213+0000","last_peered":"2024-09-21T06:33:47.069213+0000","last_clean":"2024-09-21T06:33:47.069213+0000","last_became_active":"2024-09-21T06:33:47.064177+0000","last_became_peered":"2024-09-21T06:33:47.064177+0000","last_unstale":"2024-09-21T06:33:47.069213+0000","last_undegraded":"2024-09-21T06:33:47.069213+0000","last_fullsized":"2024-09-21T06:33:47.069213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:07.197873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.039436+0000","last_change":"2024-09-21T06:33:46.048786+0000","last_active":"2024-09-21T06:33:47.039436+0000","last_peered":"2024-09-21T06:33:47.039436+0000","last_clean":"2024-09-21T06:33:47.039436+0000","last_became_active":"2024-09-21T06:33:46.048476+0000","last_became_peered":"2024-09-21T06:33:46.048476+0000","last_unstale":"2024-09-21T06:33:47.039436+0000","last_undegraded":"2024-09-21T06:33:47.039436+0000","last_fullsized":"2024-09-21T06:33:47.039436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:40.162518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082343+0000","last_change":"2024-09-21T06:33:47.077204+0000","last_active":"2024-09-21T06:33:47.082343+0000","last_peered":"2024-09-21T06:33:47.082343+0000","last_clean":"2024-09-21T06:33:47.082343+0000","last_became_active":"2024-09-21T06:33:47.076903+0000","last_became_peered":"2024-09-21T06:33:47.076903+0000","last_unstale":"2024-09-21T06:33:47.082343+0000","last_undegraded":"2024-09-21T06:33:47.082343+0000","last_fullsized":"2024-09-21T06:33:47.082343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:03.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.042506+0000","last_change":"2024-09-21T06:33:46.040890+0000","last_active":"2024-09-21T06:33:47.042506+0000","last_peered":"2024-09-21T06:33:47.042506+0000","last_clean":"2024-09-21T06:33:47.042506+0000","last_became_active":"2024-09-21T06:33:46.040561+0000","last_became_peered":"2024-09-21T06:33:46.040561+0000","last_unstale":"2024-09-21T06:33:47.042506+0000","last_undegraded":"2024-09-21T06:33:47.042506+0000","last_fullsized":"2024-09-21T06:33:47.042506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:25.724373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072834+0000","last_change":"2024-09-21T06:33:47.068301+0000","last_active":"2024-09-21T06:33:47.072834+0000","last_peered":"2024-09-21T06:33:47.072834+0000","last_clean":"2024-09-21T06:33:47.072834+0000","last_became_active":"2024-09-21T06:33:47.067884+0000","last_became_peered":"2024-09-21T06:33:47.067884+0000","last_unstale":"2024-09-21T06:33:47.072834+0000","last_undegraded":"2024-09-21T06:33:47.072834+0000","last_fullsized":"2024-09-21T06:33:47.072834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:29.206815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.374713+0000","last_change":"2024-09-21T06:33:46.046363+0000","last_active":"2024-09-21T06:33:48.374713+0000","last_peered":"2024-09-21T06:33:48.374713+0000","last_clean":"2024-09-21T06:33:48.374713+0000","last_became_active":"2024-09-21T06:33:46.045396+0000","last_became_peered":"2024-09-21T06:33:46.045396+0000","last_unstale":"2024-09-21T06:33:48.374713+0000","last_undegraded":"2024-09-21T06:33:48.374713+0000","last_fullsized":"2024-09-21T06:33:48.374713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.700884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082026+0000","last_change":"2024-09-21T06:33:47.068955+0000","last_active":"2024-09-21T06:33:47.082026+0000","last_peered":"2024-09-21T06:33:47.082026+0000","last_clean":"2024-09-21T06:33:47.082026+0000","last_became_active":"2024-09-21T06:33:47.068597+0000","last_became_peered":"2024-09-21T06:33:47.068597+0000","last_unstale":"2024-09-21T06:33:47.082026+0000","last_undegraded":"2024-09-21T06:33:47.082026+0000","last_fullsized":"2024-09-21T06:33:47.082026+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:48.882133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.078044+0000","last_change":"2024-09-21T06:33:46.055047+0000","last_active":"2024-09-21T06:33:47.078044+0000","last_peered":"2024-09-21T06:33:47.078044+0000","last_clean":"2024-09-21T06:33:47.078044+0000","last_became_active":"2024-09-21T06:33:46.049265+0000","last_became_peered":"2024-09-21T06:33:46.049265+0000","last_unstale":"2024-09-21T06:33:47.078044+0000","last_undegraded":"2024-09-21T06:33:47.078044+0000","last_fullsized":"2024-09-21T06:33:47.078044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:18.427398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375294+0000","last_change":"2024-09-21T06:33:47.054767+0000","last_active":"2024-09-21T06:33:48.375294+0000","last_peered":"2024-09-21T06:33:48.375294+0000","last_clean":"2024-09-21T06:33:48.375294+0000","last_became_active":"2024-09-21T06:33:47.053368+0000","last_became_peered":"2024-09-21T06:33:47.053368+0000","last_unstale":"2024-09-21T06:33:48.375294+0000","last_undegraded":"2024-09-21T06:33:48.375294+0000","last_fullsized":"2024-09-21T06:33:48.375294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.193300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.038445+0000","last_change":"2024-09-21T06:33:46.062160+0000","last_active":"2024-09-21T06:33:47.038445+0000","last_peered":"2024-09-21T06:33:47.038445+0000","last_clean":"2024-09-21T06:33:47.038445+0000","last_became_active":"2024-09-21T06:33:46.061856+0000","last_became_peered":"2024-09-21T06:33:46.061856+0000","last_unstale":"2024-09-21T06:33:47.038445+0000","last_undegraded":"2024-09-21T06:33:47.038445+0000","last_fullsized":"2024-09-21T06:33:47.038445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:04.971480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-21T06:33:48.375350+0000","last_change":"2024-09-21T06:33:47.057823+0000","last_active":"2024-09-21T06:33:48.375350+0000","last_peered":"2024-09-21T06:33:48.375350+0000","last_clean":"2024-09-21T06:33:48.375350+0000","last_became_active":"2024-09-21T06:33:47.057456+0000","last_became_peered":"2024-09-21T06:33:47.057456+0000","last_unstale":"2024-09-21T06:33:48.375350+0000","last_undegraded":"2024-09-21T06:33:48.375350+0000","last_fullsized":"2024-09-21T06:33:48.375350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:47.539777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.047504+0000","last_change":"2024-09-21T06:33:46.054337+0000","last_active":"2024-09-21T06:33:47.047504+0000","last_peered":"2024-09-21T06:33:47.047504+0000","last_clean":"2024-09-21T06:33:47.047504+0000","last_became_active":"2024-09-21T06:33:46.053991+0000","last_became_peered":"2024-09-21T06:33:46.053991+0000","last_unstale":"2024-09-21T06:33:47.047504+0000","last_undegraded":"2024-09-21T06:33:47.047504+0000","last_fullsized":"2024-09-21T06:33:47.047504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.757561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.072637+0000","last_change":"2024-09-21T06:33:47.063064+0000","last_active":"2024-09-21T06:33:47.072637+0000","last_peered":"2024-09-21T06:33:47.072637+0000","last_clean":"2024-09-21T06:33:47.072637+0000","last_became_active":"2024-09-21T06:33:47.062642+0000","last_became_peered":"2024-09-21T06:33:47.062642+0000","last_unstale":"2024-09-21T06:33:47.072637+0000","last_undegraded":"2024-09-21T06:33:47.072637+0000","last_fullsized":"2024-09-21T06:33:47.072637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.821183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.036824+0000","last_change":"2024-09-21T06:33:46.054125+0000","last_active":"2024-09-21T06:33:47.036824+0000","last_peered":"2024-09-21T06:33:47.036824+0000","last_clean":"2024-09-21T06:33:47.036824+0000","last_became_active":"2024-09-21T06:33:46.053836+0000","last_became_peered":"2024-09-21T06:33:46.053836+0000","last_unstale":"2024-09-21T06:33:47.036824+0000","last_undegraded":"2024-09-21T06:33:47.036824+0000","last_fullsized":"2024-09-21T06:33:47.036824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:26.637396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046083+0000","last_change":"2024-09-21T06:33:47.044583+0000","last_active":"2024-09-21T06:33:47.046083+0000","last_peered":"2024-09-21T06:33:47.046083+0000","last_clean":"2024-09-21T06:33:47.046083+0000","last_became_active":"2024-09-21T06:33:47.044317+0000","last_became_peered":"2024-09-21T06:33:47.044317+0000","last_unstale":"2024-09-21T06:33:47.046083+0000","last_undegraded":"2024-09-21T06:33:47.046083+0000","last_fullsized":"2024-09-21T06:33:47.046083+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:18.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.053607+0000","last_change":"2024-09-21T06:33:46.053489+0000","last_active":"2024-09-21T06:33:47.053607+0000","last_peered":"2024-09-21T06:33:47.053607+0000","last_clean":"2024-09-21T06:33:47.053607+0000","last_became_active":"2024-09-21T06:33:46.052998+0000","last_became_peered":"2024-09-21T06:33:46.052998+0000","last_unstale":"2024-09-21T06:33:47.053607+0000","last_undegraded":"2024-09-21T06:33:47.053607+0000","last_fullsized":"2024-09-21T06:33:47.053607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:15.962721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.082407+0000","last_change":"2024-09-21T06:33:47.078829+0000","last_active":"2024-09-21T06:33:47.082407+0000","last_peered":"2024-09-21T06:33:47.082407+0000","last_clean":"2024-09-21T06:33:47.082407+0000","last_became_active":"2024-09-21T06:33:47.078448+0000","last_became_peered":"2024-09-21T06:33:47.078448+0000","last_unstale":"2024-09-21T06:33:47.082407+0000","last_undegraded":"2024-09-21T06:33:47.082407+0000","last_fullsized":"2024-09-21T06:33:47.082407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:29.113451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.047828+0000","last_change":"2024-09-21T06:33:46.061868+0000","last_active":"2024-09-21T06:33:47.047828+0000","last_peered":"2024-09-21T06:33:47.047828+0000","last_clean":"2024-09-21T06:33:47.047828+0000","last_became_active":"2024-09-21T06:33:46.061388+0000","last_became_peered":"2024-09-21T06:33:46.061388+0000","last_unstale":"2024-09-21T06:33:47.047828+0000","last_undegraded":"2024-09-21T06:33:47.047828+0000","last_fullsized":"2024-09-21T06:33:47.047828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:02.320057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.055932+0000","last_change":"2024-09-21T06:33:47.049373+0000","last_active":"2024-09-21T06:33:47.055932+0000","last_peered":"2024-09-21T06:33:47.055932+0000","last_clean":"2024-09-21T06:33:47.055932+0000","last_became_active":"2024-09-21T06:33:47.048946+0000","last_became_peered":"2024-09-21T06:33:47.048946+0000","last_unstale":"2024-09-21T06:33:47.055932+0000","last_undegraded":"2024-09-21T06:33:47.055932+0000","last_fullsized":"2024-09-21T06:33:47.055932+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:40.174249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.048169+0000","last_change":"2024-09-21T06:33:46.061024+0000","last_active":"2024-09-21T06:33:47.048169+0000","last_peered":"2024-09-21T06:33:47.048169+0000","last_clean":"2024-09-21T06:33:47.048169+0000","last_became_active":"2024-09-21T06:33:46.060765+0000","last_became_peered":"2024-09-21T06:33:46.060765+0000","last_unstale":"2024-09-21T06:33:47.048169+0000","last_undegraded":"2024-09-21T06:33:47.048169+0000","last_fullsized":"2024-09-21T06:33:47.048169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:22.062567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.077155+0000","last_change":"2024-09-21T06:33:47.066496+0000","last_active":"2024-09-21T06:33:47.077155+0000","last_peered":"2024-09-21T06:33:47.077155+0000","last_clean":"2024-09-21T06:33:47.077155+0000","last_became_active":"2024-09-21T06:33:47.065685+0000","last_became_peered":"2024-09-21T06:33:47.065685+0000","last_unstale":"2024-09-21T06:33:47.077155+0000","last_undegraded":"2024-09-21T06:33:47.077155+0000","last_fullsized":"2024-09-21T06:33:47.077155+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:31.908193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046601+0000","last_change":"2024-09-21T06:33:46.049249+0000","last_active":"2024-09-21T06:33:47.046601+0000","last_peered":"2024-09-21T06:33:47.046601+0000","last_clean":"2024-09-21T06:33:47.046601+0000","last_became_active":"2024-09-21T06:33:46.048712+0000","last_became_peered":"2024-09-21T06:33:46.048712+0000","last_unstale":"2024-09-21T06:33:47.046601+0000","last_undegraded":"2024-09-21T06:33:47.046601+0000","last_fullsized":"2024-09-21T06:33:47.046601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:17.029566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.075092+0000","last_change":"2024-09-21T06:33:47.065102+0000","last_active":"2024-09-21T06:33:47.075092+0000","last_peered":"2024-09-21T06:33:47.075092+0000","last_clean":"2024-09-21T06:33:47.075092+0000","last_became_active":"2024-09-21T06:33:47.064778+0000","last_became_peered":"2024-09-21T06:33:47.064778+0000","last_unstale":"2024-09-21T06:33:47.075092+0000","last_undegraded":"2024-09-21T06:33:47.075092+0000","last_fullsized":"2024-09-21T06:33:47.075092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:46.020219+0000","last_clean_scrub_stamp":"2024-09-21T06:33:46.020219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:09.587111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.051129+0000","last_change":"2024-09-21T06:33:46.057731+0000","last_active":"2024-09-21T06:33:47.051129+0000","last_peered":"2024-09-21T06:33:47.051129+0000","last_clean":"2024-09-21T06:33:47.051129+0000","last_became_active":"2024-09-21T06:33:46.057316+0000","last_became_peered":"2024-09-21T06:33:46.057316+0000","last_unstale":"2024-09-21T06:33:47.051129+0000","last_undegraded":"2024-09-21T06:33:47.051129+0000","last_fullsized":"2024-09-21T06:33:47.051129+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:33:45.017302+0000","last_clean_scrub_stamp":"2024-09-21T06:33:45.017302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.757561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T06:33:47.046674+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:33:47.046674+0000","last_peered":"2024-09-21T06:33:47.046674+0000","last_clean":"2024-09-21T06:33:47.046674+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:33:47.046674+0000","last_undegraded":"2024-09-21T06:33:47.046674+0000","last_fullsized":"2024-09-21T06:33:47.046674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":506461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509515,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":508197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":506461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":508185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1110016,"data_stored":507065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509517,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1097515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":506531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":1103521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:33:52.039 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:33:52.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.308+0000 7fc1a9600640 1 -- 172.21.15.37:0/3289100020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 msgr2=0x7fc188004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.308+0000 7fc1a9600640 1 --2- 172.21.15.37:0/3289100020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc188004c10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40cd8c0 tx=0x7fc188005830 comp rx=0 tx=0).stop 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.309+0000 7fc1a9600640 1 -- 172.21.15.37:0/3289100020 shutdown_connections 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.309+0000 7fc1a9600640 1 --2- 172.21.15.37:0/3289100020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc188004c10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.309+0000 7fc1a9600640 1 -- 172.21.15.37:0/3289100020 >> 172.21.15.37:0/3289100020 conn(0x7fc1a40cca40 msgr2=0x7fc1a40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.309+0000 7fc1a9600640 1 -- 172.21.15.37:0/3289100020 shutdown_connections 2024-09-21T06:33:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.309+0000 7fc1a9600640 1 -- 172.21.15.37:0/3289100020 wait complete. 2024-09-21T06:33:52.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.311+0000 7fc1a9600640 1 Processor -- start 2024-09-21T06:33:52.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.311+0000 7fc1a9600640 1 -- start start 2024-09-21T06:33:52.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1a40c9e20 0x7fc1a4159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a40b5a20 0x7fc1a4159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40cc100 con 0x7fc188004800 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40cbf80 con 0x7fc1a40b5a20 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a9600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40cc280 con 0x7fc1a40c9e20 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39442/0 (socket says 172.21.15.37:39442) 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 -- 172.21.15.37:0/507239870 learned_addr learned my addr 172.21.15.37:0/507239870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a3400640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a40b5a20 0x7fc1a4159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc19be00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1a40c9e20 0x7fc1a4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 -- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1a40c9e20 msgr2=0x7fc1a4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1a40c9e20 0x7fc1a4159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 -- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a40b5a20 msgr2=0x7fc1a4159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a40b5a20 0x7fc1a4159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.313+0000 7fc1a2a00640 1 -- 172.21.15.37:0/507239870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a40a1020 con 0x7fc188004800 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc1a2a00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e570 tx=0x7fc19400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194015020 con 0x7fc188004800 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19400c9e0 con 0x7fc188004800 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19400cdf0 con 0x7fc188004800 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a41f53a0 con 0x7fc188004800 2024-09-21T06:33:52.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.314+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a41f5640 con 0x7fc188004800 2024-09-21T06:33:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.315+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a406e000 con 0x7fc188004800 2024-09-21T06:33:52.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.317+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc194018020 con 0x7fc188004800 2024-09-21T06:33:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.318+0000 7fc19aa00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.318+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc19409b3e0 con 0x7fc188004800 2024-09-21T06:33:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.318+0000 7fc19be00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.319+0000 7fc19be00640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 0x7fc18007b0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc18c032000 tx=0x7fc18c001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:52.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.322+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc19401f020 con 0x7fc188004800 2024-09-21T06:33:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.591+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc1a415b480 con 0x7fc188004800 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.595+0000 7fc19aa00640 1 -- 172.21.15.37:0/507239870 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fc194063690 con 0x7fc188004800 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:33:52.596 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:33:52.597 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:33:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 msgr2=0x7fc18007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 0x7fc18007b0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc18c032000 tx=0x7fc18c001190 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 msgr2=0x7fc1a41f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e570 tx=0x7fc19400ea30 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 shutdown_connections 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a40b5a20 0x7fc1a4159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1a40c9e20 0x7fc1a4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 --2- 172.21.15.37:0/507239870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc188004800 0x7fc1a41f4ce0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 >> 172.21.15.37:0/507239870 conn(0x7fc1a40cca40 msgr2=0x7fc1a415de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.597+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 shutdown_connections 2024-09-21T06:33:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.598+0000 7fc1a9600640 1 -- 172.21.15.37:0/507239870 wait complete. 2024-09-21T06:33:52.623 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:52.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 -- 172.21.15.37:0/2292287962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440d2520 msgr2=0x7f254415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 --2- 172.21.15.37:0/2292287962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440d2520 0x7f254415e5e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f253c005fd0 tx=0x7f253c02aa10 comp rx=0 tx=0).stop 2024-09-21T06:33:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 -- 172.21.15.37:0/2292287962 shutdown_connections 2024-09-21T06:33:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 --2- 172.21.15.37:0/2292287962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440d2520 0x7f254415e5e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 -- 172.21.15.37:0/2292287962 >> 172.21.15.37:0/2292287962 conn(0x7f25440d4360 msgr2=0x7f25440d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 -- 172.21.15.37:0/2292287962 shutdown_connections 2024-09-21T06:33:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.849+0000 7f254ae00640 1 -- 172.21.15.37:0/2292287962 wait complete. 2024-09-21T06:33:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.851+0000 7f254ae00640 1 Processor -- start 2024-09-21T06:33:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.851+0000 7f254ae00640 1 -- start start 2024-09-21T06:33:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 0x7f25441568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2544160c70 0x7f2544161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25440b8fd0 con 0x7f25440ceff0 2024-09-21T06:33:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25440b8e50 con 0x7f25440d2520 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254ae00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f25440b9150 con 0x7f2544160c70 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f2549a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f2549a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39470/0 (socket says 172.21.15.37:39470) 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f2549a00640 1 -- 172.21.15.37:0/1769505301 learned_addr learned my addr 172.21.15.37:0/1769505301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f2549000640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 0x7f25441568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.853+0000 7f254a400640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2544160c70 0x7f2544161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 -- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2544160c70 msgr2=0x7f2544161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2544160c70 0x7f2544161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 -- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 msgr2=0x7f25441568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 0x7f25441568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 -- 172.21.15.37:0/1769505301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2544147470 con 0x7f25440ceff0 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549000640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 0x7f25441568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2549a00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f253800c400 tx=0x7f253800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538017070 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25380053c0 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25441615c0 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538003850 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.854+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25440cc830 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.855+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254415e5e0 con 0x7f25440ceff0 2024-09-21T06:33:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.857+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f253801a070 con 0x7f25440ceff0 2024-09-21T06:33:52.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.857+0000 7f2541600640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 0x7f252407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.858+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f253809a3b0 con 0x7f25440ceff0 2024-09-21T06:33:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.858+0000 7f2549000640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 0x7f252407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.859+0000 7f2549000640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 0x7f252407b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f25441575f0 tx=0x7f2530009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:52.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:52.862+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25441615c0 con 0x7f25440ceff0 2024-09-21T06:33:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.099+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25440b9f00 con 0x7f25440ceff0 2024-09-21T06:33:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.100+0000 7f2541600640 1 -- 172.21.15.37:0/1769505301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5228 (secure 0 0 0) 0x7f25380625c0 con 0x7f25440ceff0 2024-09-21T06:33:53.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:53.101 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"btime":"2024-09-21T06:33:50:060429+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:33:49.568429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392]},"id":6}]} 2024-09-21T06:33:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 89 2024-09-21T06:33:53.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 msgr2=0x7f252407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 0x7f252407b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f25441575f0 tx=0x7f2530009040 comp rx=0 tx=0).stop 2024-09-21T06:33:53.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 msgr2=0x7f2544153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f253800c400 tx=0x7f253800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 shutdown_connections 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2524078bc0 0x7f252407b0d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2544160c70 0x7f2544161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25440d2520 0x7f25441568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 --2- 172.21.15.37:0/1769505301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25440ceff0 0x7f2544153a90 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 >> 172.21.15.37:0/1769505301 conn(0x7f25440d4360 msgr2=0x7f254406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 shutdown_connections 2024-09-21T06:33:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.102+0000 7f254ae00640 1 -- 172.21.15.37:0/1769505301 wait complete. 2024-09-21T06:33:53.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:33:46.562653+0000', 'modified': '2024-09-21T06:33:49.568429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7392}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7392': {'gid': 7392, 'name': 'd', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6836/2979710501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 2979710501}, {'type': 'v1', 'addr': '172.21.15.150:6836', 'nonce': 2979710501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7392, 'qdb_cluster': [7392]} 2024-09-21T06:33:53.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:33:53.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:33:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 -- 172.21.15.37:0/2241183820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c1605f0 msgr2=0x7fc64c0702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 --2- 172.21.15.37:0/2241183820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c1605f0 0x7fc64c0702a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc634009830 tx=0x7fc63402a530 comp rx=0 tx=0).stop 2024-09-21T06:33:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 -- 172.21.15.37:0/2241183820 shutdown_connections 2024-09-21T06:33:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 --2- 172.21.15.37:0/2241183820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c1605f0 0x7fc64c0702a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 -- 172.21.15.37:0/2241183820 >> 172.21.15.37:0/2241183820 conn(0x7fc64c0d05c0 msgr2=0x7fc64c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 -- 172.21.15.37:0/2241183820 shutdown_connections 2024-09-21T06:33:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.757+0000 7fc650a00640 1 -- 172.21.15.37:0/2241183820 wait complete. 2024-09-21T06:33:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.759+0000 7fc650a00640 1 Processor -- start 2024-09-21T06:33:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.759+0000 7fc650a00640 1 -- start start 2024-09-21T06:33:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.760+0000 7fc650a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c0cc0e0 0x7fc64c13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc650a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc650a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64c141670 0x7fc64c141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc650a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c0b3e60 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc650a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c0b3ce0 con 0x7fc64c0cc0e0 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc650a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc64c0b3fe0 con 0x7fc64c141670 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39476/0 (socket says 172.21.15.37:39476) 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 -- 172.21.15.37:0/2044844062 learned_addr learned my addr 172.21.15.37:0/2044844062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc64a800640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c0cc0e0 0x7fc64c13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc64b200640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64c141670 0x7fc64c141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 -- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64c141670 msgr2=0x7fc64c141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64c141670 0x7fc64c141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 -- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c0cc0e0 msgr2=0x7fc64c13c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c0cc0e0 0x7fc64c13c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.761+0000 7fc641e00640 1 -- 172.21.15.37:0/2044844062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64c13fef0 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc641e00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc634009950 tx=0x7fc634033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc643400640 1 -- 172.21.15.37:0/2044844062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634031d60 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc643400640 1 -- 172.21.15.37:0/2044844062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc634033e10 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64c1401d0 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc643400640 1 -- 172.21.15.37:0/2044844062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634033310 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.762+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64c13cd40 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.764+0000 7fc643400640 1 -- 172.21.15.37:0/2044844062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6340334b0 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.765+0000 7fc643400640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 0x7fc62807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.765+0000 7fc643400640 1 -- 172.21.15.37:0/2044844062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc6340bc330 con 0x7fc64c0cf6d0 2024-09-21T06:33:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.766+0000 7fc64a800640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 0x7fc62807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.766+0000 7fc6583d6740 1 Processor -- start 2024-09-21T06:33:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.766+0000 7fc6583d6740 1 -- start start 2024-09-21T06:33:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.766+0000 7fc64a800640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 0x7fc62807b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000f60 tx=0x7fc63c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x562308430b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x5623084310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56230838a430 con 0x56230803f590 2024-09-21T06:33:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56230837a5d0 con 0x562308060270 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562308369a00 con 0x5623080373b0 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc64b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc64b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39480/0 (socket says 172.21.15.37:39480) 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc64b200640 1 -- 172.21.15.37:0/1066594176 learned_addr learned my addr 172.21.15.37:0/1066594176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc64a800640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x562308430b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.768+0000 7fc641e00640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x5623084310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 -- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 msgr2=0x5623084310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x5623084310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 -- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 msgr2=0x562308430b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x562308430b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 -- 172.21.15.37:0/1066594176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562308332520 con 0x56230803f590 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc641e00640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x5623084310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc64b200640 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc644001fe0 tx=0x7fc64400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b8216fa8596a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc63b000640 1 -- 172.21.15.37:0/1066594176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64400ee50 con 0x56230803f590 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc63b000640 1 -- 172.21.15.37:0/1066594176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc644004a40 con 0x56230803f590 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.769+0000 7fc63b000640 1 -- 172.21.15.37:0/1066594176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644004c90 con 0x56230803f590 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 -- 172.21.15.37:0/1066594176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 msgr2=0x562308401150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc644001fe0 tx=0x7fc64400e530 comp rx=0 tx=0).stop 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 -- 172.21.15.37:0/1066594176 shutdown_connections 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x562308401150 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x5623084310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1066594176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x562308430b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 -- 172.21.15.37:0/1066594176 >> 172.21.15.37:0/1066594176 conn(0x5623080543c0 msgr2=0x562308406b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 -- 172.21.15.37:0/1066594176 shutdown_connections 2024-09-21T06:33:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.770+0000 7fc6583d6740 1 -- 172.21.15.37:0/1066594176 wait complete. 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.771+0000 7fc6583d6740 1 -- start start 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.771+0000 7fc6583d6740 1 Processor -- start 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230842bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x5623084271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562308310330 con 0x56230803f590 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56230836d810 con 0x562308060270 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5623080da620 con 0x5623080373b0 2024-09-21T06:33:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc6583d6740 10 client.0 _ll_register_callbacks cb 0x562308315420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39496/0 (socket says 172.21.15.37:39496) 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc641e00640 1 -- 172.21.15.37:0/1018323959 learned_addr learned my addr 172.21.15.37:0/1018323959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc64a800640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230842bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.773+0000 7fc64b200640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x5623084271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 -- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 msgr2=0x56230842bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230842bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 -- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 msgr2=0x5623084271c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x5623084271c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 -- 172.21.15.37:0/1018323959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623080cb850 con 0x56230803f590 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc641e00640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc634083ef0 tx=0x7fc6340844c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5054da9da4672306 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc62f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:53.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340836d0 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc634089dd0 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc644004a40 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.774+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc6340317a0 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 10 client.7503 fetch_fsmap learned FSMap version 89 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x5623081426b0 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc644004a40 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 89) ==== 3502+0+0 (secure 0 0 0) 0x7fc63404b030 con 0x56230803f590 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 10 client.7503 fetch_fsmap finished waiting for FSMap version 89 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 10 client.7503 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 20 client.7503 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230801ee40 0x562308429d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 4 client.7503 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:53.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56230837c520 con 0x56230801ee40 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.775+0000 7fc64a800640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230801ee40 0x562308429d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.776+0000 7fc64a800640 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230801ee40 0x562308429d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cc6939fcf16c87 server_cookie=c9398b339ce0f648 in_seq=0 out_seq=0 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.776+0000 7fc62f400640 10 client.7503 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc62f400640 1 -- 172.21.15.37:0/1018323959 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56230837c520 con 0x56230801ee40 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc62f400640 10 client.7503 handle_command_reply: tid=0 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 client.7503 shutdown 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 msgr2=0x56230845eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc634083ef0 tx=0x7fc6340844c0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 shutdown_connections 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230801ee40 0x562308429d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845eda0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x5623084271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 --2- 172.21.15.37:0/1018323959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230842bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc62f400640 0 client.7503 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.777+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 >> 172.21.15.37:0/1018323959 conn(0x5623080543c0 msgr2=0x562308402530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.778+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 shutdown_connections 2024-09-21T06:33:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.778+0000 7fc6583d6740 1 -- 172.21.15.37:0/1018323959 wait complete. 2024-09-21T06:33:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.778+0000 7fc6583d6740 20 client.7503 trim_cache size 0 max 16384 2024-09-21T06:33:53.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.797+0000 7fc6583d6740 1 Processor -- start 2024-09-21T06:33:53.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.797+0000 7fc6583d6740 1 -- start start 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230846f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x56230844d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c2013c0 con 0x56230803f590 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c1f94c0 con 0x562308060270 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.798+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc64c200d70 con 0x5623080373b0 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39502/0 (socket says 172.21.15.37:39502) 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 -- 172.21.15.37:0/3447752526 learned_addr learned my addr 172.21.15.37:0/3447752526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:53.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc64a800640 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230846f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 -- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 msgr2=0x56230846f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230846f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 -- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 msgr2=0x56230844d640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x56230844d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 -- 172.21.15.37:0/3447752526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562308011930 con 0x56230803f590 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc641e00640 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6340811d0 tx=0x7fc6340aa160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ca7cb41e59dabd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.799+0000 7fc63a600640 1 -- 172.21.15.37:0/3447752526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63409d070 con 0x56230803f590 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc63a600640 1 -- 172.21.15.37:0/3447752526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6340317a0 con 0x56230803f590 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc63a600640 1 -- 172.21.15.37:0/3447752526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634089dd0 con 0x56230803f590 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 -- 172.21.15.37:0/3447752526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 msgr2=0x56230842d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6340811d0 tx=0x7fc6340aa160 comp rx=0 tx=0).stop 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 -- 172.21.15.37:0/3447752526 shutdown_connections 2024-09-21T06:33:53.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230842d8d0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562308060270 0x56230844d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 --2- 172.21.15.37:0/3447752526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623080373b0 0x56230846f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 -- 172.21.15.37:0/3447752526 >> 172.21.15.37:0/3447752526 conn(0x5623080543c0 msgr2=0x56230845df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 -- 172.21.15.37:0/3447752526 shutdown_connections 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.800+0000 7fc6583d6740 1 -- 172.21.15.37:0/3447752526 wait complete. 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.801+0000 7fc6583d6740 1 -- start start 2024-09-21T06:33:53.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.802+0000 7fc6583d6740 1 Processor -- start 2024-09-21T06:33:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 0x5623084cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 0x562308453ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56230845aff0 con 0x56230803f590 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562308446060 con 0x5623080373b0 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562308427700 con 0x562308060270 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc6583d6740 10 client.0 _ll_register_callbacks cb 0x7fc63407fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc641e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39508/0 (socket says 172.21.15.37:39508) 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.803+0000 7fc641e00640 1 -- 172.21.15.37:0/2506724422 learned_addr learned my addr 172.21.15.37:0/2506724422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc64b200640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 0x562308453ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc64a800640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 0x5623084cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 -- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 msgr2=0x562308453ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 0x562308453ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 -- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 msgr2=0x5623084cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 0x5623084cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 -- 172.21.15.37:0/2506724422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230846f450 con 0x56230803f590 2024-09-21T06:33:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc64b200640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 0x562308453ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc64a800640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 0x5623084cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc641e00640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc6340804e0 tx=0x7fc634004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d65b1bf495b2222 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc62f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634098060 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc634089dd0 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6340317a0 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.804+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc63409d040 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 10 client.7509 fetch_fsmap learned FSMap version 89 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x56230846f450 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6340317a0 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 89) ==== 3502+0+0 (secure 0 0 0) 0x7fc63404b070 con 0x56230803f590 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 10 client.7509 fetch_fsmap finished waiting for FSMap version 89 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 10 client.7509 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:33:53.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 20 client.7509 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:53.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230842ae30 0x562308454530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 4 client.7509 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:53.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "status"}) -- 0x5623084061a0 con 0x56230842ae30 2024-09-21T06:33:53.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.805+0000 7fc64a800640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230842ae30 0x562308454530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc64a800640 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230842ae30 0x562308454530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08e1440fd728434 server_cookie=64c350a892bd39cf in_seq=0 out_seq=0 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc62f400640 10 client.7509 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc62f400640 1 -- 172.21.15.37:0/2506724422 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5623084061a0 con 0x56230842ae30 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc62f400640 10 client.7509 handle_command_reply: tid=0 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc6583d6740 1 client.7509 shutdown 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 msgr2=0x56230845f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.806+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc6340804e0 tx=0x7fc634004410 comp rx=0 tx=0).stop 2024-09-21T06:33:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 shutdown_connections 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56230842ae30 0x562308454530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56230803f590 0x56230845f7a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562308060270 0x562308453ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 --2- 172.21.15.37:0/2506724422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623080373b0 0x5623084cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc62f400640 0 client.7509 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 >> 172.21.15.37:0/2506724422 conn(0x5623080543c0 msgr2=0x56230844eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 shutdown_connections 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 1 -- 172.21.15.37:0/2506724422 wait complete. 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc6583d6740 20 client.7509 trim_cache size 0 max 16384 2024-09-21T06:33:53.812 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7392, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 6.7752975849999997, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 89, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 91, 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.734040630000001 2024-09-21T06:33:53.813 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 msgr2=0x7fc62807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.807+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 0x7fc62807b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000f60 tx=0x7fc63c009040 comp rx=0 tx=0).stop 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 msgr2=0x7fc64c141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc634009950 tx=0x7fc634033c20 comp rx=0 tx=0).stop 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 shutdown_connections 2024-09-21T06:33:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc628078b90 0x7fc62807b0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64c0cf6d0 0x7fc64c141130 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64c141670 0x7fc64c141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 --2- 172.21.15.37:0/2044844062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc64c0cc0e0 0x7fc64c13c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 >> 172.21.15.37:0/2044844062 conn(0x7fc64c0d05c0 msgr2=0x7fc64c13ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 shutdown_connections 2024-09-21T06:33:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:53.808+0000 7fc650a00640 1 -- 172.21.15.37:0/2044844062 wait complete. 2024-09-21T06:33:53.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 -- 172.21.15.37:0/3154168911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc0b9e40 msgr2=0x7f00bc1622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 --2- 172.21.15.37:0/3154168911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc0b9e40 0x7f00bc1622b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 -- 172.21.15.37:0/3154168911 shutdown_connections 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 --2- 172.21.15.37:0/3154168911 >> v2:172.21.15.150:3300/0 conn(0x7f00bc1627f0 0x7f00bc06d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 --2- 172.21.15.37:0/3154168911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc0b9e40 0x7f00bc1622b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 -- 172.21.15.37:0/3154168911 >> 172.21.15.37:0/3154168911 conn(0x7f00bc0d1930 msgr2=0x7f00bc0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 -- 172.21.15.37:0/3154168911 shutdown_connections 2024-09-21T06:33:54.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.060+0000 7f00c0e00640 1 -- 172.21.15.37:0/3154168911 wait complete. 2024-09-21T06:33:54.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.063+0000 7f00c0e00640 1 Processor -- start 2024-09-21T06:33:54.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.063+0000 7f00c0e00640 1 -- start start 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc06d7f0 0x7f00bc1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00bc0b9e40 0x7f00bc1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1557c0 con 0x7f00bc1627f0 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06e7f0 con 0x7f00bc0b9e40 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.064+0000 7f00c0e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc155940 con 0x7f00bc06d7f0 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:54.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42124/0 (socket says 172.21.15.37:42124) 2024-09-21T06:33:54.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 -- 172.21.15.37:0/2644176858 learned_addr learned my addr 172.21.15.37:0/2644176858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:54.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00baa00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00bc0b9e40 0x7f00bc1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:54.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bb400640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc06d7f0 0x7f00bc1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 -- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc06d7f0 msgr2=0x7f00bc1f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc06d7f0 0x7f00bc1f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 -- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00bc0b9e40 msgr2=0x7f00bc1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00bc0b9e40 0x7f00bc1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.065+0000 7f00bbe00640 1 -- 172.21.15.37:0/2644176858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc1f6220 con 0x7f00bc1627f0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00bbe00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c700 tx=0x7f00b400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4003b70 con 0x7f00bc1627f0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00b4003d10 con 0x7f00bc1627f0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1f69c0 con 0x7f00bc1627f0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b400aab0 con 0x7f00bc1627f0 2024-09-21T06:33:54.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.066+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1f6f80 con 0x7f00bc1627f0 2024-09-21T06:33:54.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.067+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc1622b0 con 0x7f00bc1627f0 2024-09-21T06:33:54.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.068+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f00b400ac50 con 0x7f00bc1627f0 2024-09-21T06:33:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.070+0000 7f00b3400640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 0x7f009c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.070+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f00b409abc0 con 0x7f00bc1627f0 2024-09-21T06:33:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.070+0000 7f00bb400640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 0x7f009c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.071+0000 7f00bb400640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 0x7f009c07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00ac033000 tx=0x7f00ac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:54.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.075+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00bc1f6220 con 0x7f00bc1627f0 2024-09-21T06:33:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.314+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00bc154e50 con 0x7f00bc1627f0 2024-09-21T06:33:54.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.315+0000 7f00b3400640 1 -- 172.21.15.37:0/2644176858 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7f00bc163170 con 0x7f00bc1627f0 2024-09-21T06:33:54.317 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 msgr2=0x7f009c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 0x7f009c07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00ac033000 tx=0x7f00ac004730 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 msgr2=0x7f00bc0d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c700 tx=0x7f00b400cbc0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 shutdown_connections 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f009c078b90 0x7f009c07b0a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00bc1627f0 0x7f00bc0d2830 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00bc0b9e40 0x7f00bc1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 --2- 172.21.15.37:0/2644176858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00bc06d7f0 0x7f00bc1f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 >> 172.21.15.37:0/2644176858 conn(0x7f00bc0d1930 msgr2=0x7f00bc15cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.317+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 shutdown_connections 2024-09-21T06:33:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54.318+0000 7f00c0e00640 1 -- 172.21.15.37:0/2644176858 wait complete. 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:33:54.343 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:54.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.344 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:54.380 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42357sec preferred_lft 42357sec 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:33:54.381 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:33:54.382 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:33:54.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.383 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:33:54.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.476 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:33:54.513 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:33:54.513 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:33:54.513 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:33:54.513 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:33:54.513 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:33:54.513 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:33:54.513 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:33:54.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:33:54.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:33:54.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:33:54.514 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:33:54.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:33:54.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:33:54.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.710 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:33:54.746 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:33:54.746 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:33:54.746 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:54.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:33:54.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:33:54.843 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:54.878 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:33:54.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:33:54.878 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:54.952 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:33:54.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:54.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:55.152 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:55.152 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:55.152 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:55.173 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:55.173 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:55.173 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:55.173 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:33:55.173 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:33:55.204 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:33:55.205 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:33:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:33:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:33:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:33:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:33:55.209 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:33:55.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:33:55.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:33:55.266 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:33:55.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:33:55.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:33:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:client7515 2024-09-21T06:33:55.371 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:33:55.372 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:33:55.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:33:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 -- 172.21.15.37:0/2046779016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 msgr2=0x7fcc40003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2046779016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc40003e10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009b50 tx=0x7fcc40005830 comp rx=0 tx=0).stop 2024-09-21T06:33:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc60a00640 1 -- 172.21.15.37:0/2046779016 reap_dead start 2024-09-21T06:33:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 -- 172.21.15.37:0/2046779016 shutdown_connections 2024-09-21T06:33:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 -- 172.21.15.37:0/2046779016 >> 172.21.15.37:0/2046779016 conn(0x7fcc5c0b1b90 msgr2=0x7fcc5c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:55.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 -- 172.21.15.37:0/2046779016 shutdown_connections 2024-09-21T06:33:55.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.675+0000 7fcc62a00640 1 -- 172.21.15.37:0/2046779016 wait complete. 2024-09-21T06:33:55.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.677+0000 7fcc62a00640 1 Processor -- start 2024-09-21T06:33:55.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.677+0000 7fcc62a00640 1 -- start start 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c1e8e00 0x7fcc5c1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc5c1f8680 0x7fcc5c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c0747a0 con 0x7fcc400039e0 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c074620 con 0x7fcc5c1e8e00 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc62a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c074920 con 0x7fcc5c1f8680 2024-09-21T06:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:55.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42150/0 (socket says 172.21.15.37:42150) 2024-09-21T06:33:55.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 -- 172.21.15.37:0/2819820272 learned_addr learned my addr 172.21.15.37:0/2819820272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc53e00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c1e8e00 0x7fcc5c1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc60a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc5c1f8680 0x7fcc5c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 -- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc5c1f8680 msgr2=0x7fcc5c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc5c1f8680 0x7fcc5c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 -- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c1e8e00 msgr2=0x7fcc5c1f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.679+0000 7fcc5be00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c1e8e00 0x7fcc5c1f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc5be00640 1 -- 172.21.15.37:0/2819820272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1f8a90 con 0x7fcc400039e0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc5be00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036040 tx=0x7fcc4c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c038040 con 0x7fcc400039e0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc4c004020 con 0x7fcc400039e0 2024-09-21T06:33:55.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c004470 con 0x7fcc400039e0 2024-09-21T06:33:55.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1e6490 con 0x7fcc400039e0 2024-09-21T06:33:55.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.680+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1e8040 con 0x7fcc400039e0 2024-09-21T06:33:55.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.681+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c0cf890 con 0x7fcc400039e0 2024-09-21T06:33:55.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.683+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc4c045020 con 0x7fcc400039e0 2024-09-21T06:33:55.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.683+0000 7fcc58c00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 0x7fcc3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:55.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.684+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fcc4c0bb870 con 0x7fcc400039e0 2024-09-21T06:33:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.684+0000 7fcc53e00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 0x7fcc3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.685+0000 7fcc53e00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 0x7fcc3807b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c160b60 tx=0x7fcc44000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.688+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc5c1f8a90 con 0x7fcc400039e0 2024-09-21T06:33:55.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:55.926+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fcc5c06d160 con 0x7fcc400039e0 2024-09-21T06:33:56.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.096+0000 7fcc58c00640 1 -- 172.21.15.37:0/2819820272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v90) ==== 105+0+0 (secure 0 0 0) 0x7fcc4c083ba0 con 0x7fcc400039e0 2024-09-21T06:33:56.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 msgr2=0x7fcc3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 0x7fcc3807b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c160b60 tx=0x7fcc44000ed0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 msgr2=0x7fcc5c1e5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036040 tx=0x7fcc4c033670 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 shutdown_connections 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcc38078b90 0x7fcc3807b0a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc5c1f8680 0x7fcc5c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c1e8e00 0x7fcc5c1f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 --2- 172.21.15.37:0/2819820272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc400039e0 0x7fcc5c1e5f50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 >> 172.21.15.37:0/2819820272 conn(0x7fcc5c0b1b90 msgr2=0x7fcc5c0c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 shutdown_connections 2024-09-21T06:33:56.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.098+0000 7fcc62a00640 1 -- 172.21.15.37:0/2819820272 wait complete. 2024-09-21T06:33:56.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 -- 172.21.15.37:0/2087108018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200afb80 msgr2=0x7f2b200ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 --2- 172.21.15.37:0/2087108018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200afb80 0x7f2b200ce960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005fd0 tx=0x7f2b1802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 -- 172.21.15.37:0/2087108018 shutdown_connections 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 --2- 172.21.15.37:0/2087108018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200afb80 0x7f2b200ce960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 --2- 172.21.15.37:0/2087108018 >> v2:172.21.15.150:3300/0 conn(0x7f2b2006e500 0x7f2b2015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 -- 172.21.15.37:0/2087108018 >> 172.21.15.37:0/2087108018 conn(0x7f2b20073400 msgr2=0x7f2b20073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 -- 172.21.15.37:0/2087108018 shutdown_connections 2024-09-21T06:33:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.356+0000 7f2b26800640 1 -- 172.21.15.37:0/2087108018 wait complete. 2024-09-21T06:33:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.359+0000 7f2b26800640 1 Processor -- start 2024-09-21T06:33:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.359+0000 7f2b26800640 1 -- start start 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b20073f20 0x7f2b20150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 0x7f2b200cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200cf450 con 0x7f2b200afb80 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200cf2d0 con 0x7f2b200cbd70 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.360+0000 7f2b26800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2b200cf5d0 con 0x7f2b20073f20 2024-09-21T06:33:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42156/0 (socket says 172.21.15.37:42156) 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 -- 172.21.15.37:0/182683578 learned_addr learned my addr 172.21.15.37:0/182683578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b25e00640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 0x7f2b200cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b25400640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b20073f20 0x7f2b20150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 -- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b20073f20 msgr2=0x7f2b20150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b20073f20 0x7f2b20150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 -- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 msgr2=0x7f2b200cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 0x7f2b200cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 -- 172.21.15.37:0/182683578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b20159d40 con 0x7f2b200afb80 2024-09-21T06:33:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b25e00640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 0x7f2b200cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.361+0000 7f2b24a00640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00e9f0 tx=0x7f2b0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.362+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00cd80 con 0x7f2b200afb80 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.362+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b0c010520 con 0x7f2b200afb80 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.362+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c010950 con 0x7f2b200afb80 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.362+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b20150960 con 0x7f2b200afb80 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.362+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201f8310 con 0x7f2b200afb80 2024-09-21T06:33:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.363+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2015c010 con 0x7f2b200afb80 2024-09-21T06:33:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.364+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b0c01f020 con 0x7f2b200afb80 2024-09-21T06:33:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.365+0000 7f2b1d600640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 0x7f2b0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.365+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f2b0c09b780 con 0x7f2b200afb80 2024-09-21T06:33:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.366+0000 7f2b25400640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 0x7f2b0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.367+0000 7f2b25400640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 0x7f2b0007b0a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b14001610 tx=0x7f2b14009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.370+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b20150960 con 0x7f2b200afb80 2024-09-21T06:33:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.606+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b200cfa80 con 0x7f2b200afb80 2024-09-21T06:33:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.607+0000 7f2b1d600640 1 -- 172.21.15.37:0/182683578 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5244 (secure 0 0 0) 0x7f2b20159d40 con 0x7f2b200afb80 2024-09-21T06:33:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:56.609 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"btime":"2024-09-21T06:33:56:099755+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:33:56.099741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7392,"mds_1":7389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:creating","state_seq":1,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392]},"id":6}]} 2024-09-21T06:33:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 91 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.609+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 msgr2=0x7f2b0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.609+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 0x7f2b0007b0a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b14001610 tx=0x7f2b14009040 comp rx=0 tx=0).stop 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.609+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 msgr2=0x7f2b200c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.609+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00e9f0 tx=0x7f2b0c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 shutdown_connections 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2b00078b90 0x7f2b0007b0a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b200afb80 0x7f2b200c6630 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200cbd70 0x7f2b200cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 --2- 172.21.15.37:0/182683578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b20073f20 0x7f2b20150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 >> 172.21.15.37:0/182683578 conn(0x7f2b20073400 msgr2=0x7f2b200b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 shutdown_connections 2024-09-21T06:33:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:56.610+0000 7f2b26800640 1 -- 172.21.15.37:0/182683578 wait complete. 2024-09-21T06:33:57.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:33:46.562653+0000', 'modified': '2024-09-21T06:33:56.099741+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7392, 'mds_1': 7389}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7389': {'gid': 7389, 'name': 'a', 'rank': 1, 'incarnation': 91, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.37:6835/1748999037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 1748999037}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1748999037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7392': {'gid': 7392, 'name': 'd', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6836/2979710501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 2979710501}, {'type': 'v1', 'addr': '172.21.15.150:6836', 'nonce': 2979710501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7392, 'qdb_cluster': [7392]} 2024-09-21T06:33:57.037 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7389:up:creating 2024-09-21T06:33:58.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:33:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 -- 172.21.15.37:0/2138792653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 msgr2=0x7fc10415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 --2- 172.21.15.37:0/2138792653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc10415e5e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005fd0 tx=0x7fc0f402a4b0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 -- 172.21.15.37:0/2138792653 shutdown_connections 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 --2- 172.21.15.37:0/2138792653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040cfbc0 0x7fc1040b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 --2- 172.21.15.37:0/2138792653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc10415e5e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 -- 172.21.15.37:0/2138792653 >> 172.21.15.37:0/2138792653 conn(0x7fc1040d4360 msgr2=0x7fc1040d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 -- 172.21.15.37:0/2138792653 shutdown_connections 2024-09-21T06:33:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.277+0000 7fc103e00640 1 -- 172.21.15.37:0/2138792653 wait complete. 2024-09-21T06:33:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.279+0000 7fc103e00640 1 Processor -- start 2024-09-21T06:33:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.279+0000 7fc103e00640 1 -- start start 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040d2520 0x7fc104154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1041531e0 0x7fc1040cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1040b8db0 con 0x7fc1040ceff0 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1040b8c30 con 0x7fc1041531e0 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1040b8f30 con 0x7fc1040d2520 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42164/0 (socket says 172.21.15.37:42164) 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 -- 172.21.15.37:0/3031428582 learned_addr learned my addr 172.21.15.37:0/3031428582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc0fbe00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040d2520 0x7fc104154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc103400640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1041531e0 0x7fc1040cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:58.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 -- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040d2520 msgr2=0x7fc104154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040d2520 0x7fc104154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 -- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1041531e0 msgr2=0x7fc1040cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1041531e0 0x7fc1040cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.281+0000 7fc102a00640 1 -- 172.21.15.37:0/3031428582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc104154ff0 con 0x7fc1040ceff0 2024-09-21T06:33:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc102a00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006010 tx=0x7fc0f40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f403e030 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f4003d90 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4035040 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1040cd400 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.282+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1040cd9c0 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.283+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1040b94c0 con 0x7fc1040ceff0 2024-09-21T06:33:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.284+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0f4041030 con 0x7fc1040ceff0 2024-09-21T06:33:58.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.285+0000 7fc0fb400640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:58.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.286+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc0f40bb680 con 0x7fc1040ceff0 2024-09-21T06:33:58.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.286+0000 7fc0fbe00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:58.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.287+0000 7fc0fbe00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1041e9440 tx=0x7fc0ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:58.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.290+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc0f4083240 con 0x7fc1040ceff0 2024-09-21T06:33:58.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.525+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1041eac20 con 0x7fc1040ceff0 2024-09-21T06:33:58.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.526+0000 7fc0fb400640 1 -- 172.21.15.37:0/3031428582 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5266 (secure 0 0 0) 0x7fc0f4087c70 con 0x7fc1040ceff0 2024-09-21T06:33:58.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:33:58.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"btime":"2024-09-21T06:33:58:111823+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:33:58.111812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7096,7389,7392]},"id":6}]} 2024-09-21T06:33:58.529 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 94 2024-09-21T06:33:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 msgr2=0x7fc0e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1041e9440 tx=0x7fc0ec009040 comp rx=0 tx=0).stop 2024-09-21T06:33:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 msgr2=0x7fc104152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006010 tx=0x7fc0f40038a0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 shutdown_connections 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1041531e0 0x7fc1040cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1040d2520 0x7fc104154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 --2- 172.21.15.37:0/3031428582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1040ceff0 0x7fc104152ae0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 >> 172.21.15.37:0/3031428582 conn(0x7fc1040d4360 msgr2=0x7fc10406d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 shutdown_connections 2024-09-21T06:33:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:58.529+0000 7fc103e00640 1 -- 172.21.15.37:0/3031428582 wait complete. 2024-09-21T06:33:58.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:33:46.562653+0000', 'modified': '2024-09-21T06:33:58.111812+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7392, 'mds_1': 7389, 'mds_2': 7096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7096': {'gid': 7096, 'name': 'b', 'rank': 2, 'incarnation': 93, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.150:6837/3722358982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 3722358982}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 3722358982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7389': {'gid': 7389, 'name': 'a', 'rank': 1, 'incarnation': 91, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6835/1748999037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 1748999037}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1748999037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7392': {'gid': 7392, 'name': 'd', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6836/2979710501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 2979710501}, {'type': 'v1', 'addr': '172.21.15.150:6836', 'nonce': 2979710501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7392, 'qdb_cluster': [7096, 7389, 7392]} 2024-09-21T06:33:58.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:33:58.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:33:59.015 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:33:59.014+0000 7fa6d7a00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 -- 172.21.15.37:0/1461749788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7690 msgr2=0x7ff8640d33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 --2- 172.21.15.37:0/1461749788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7690 0x7ff8640d33a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff8640d1dc0 tx=0x7ff85402a4d0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 -- 172.21.15.37:0/1461749788 shutdown_connections 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 --2- 172.21.15.37:0/1461749788 >> v2:172.21.15.150:3300/0 conn(0x7ff8640d38e0 0x7ff86406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 --2- 172.21.15.37:0/1461749788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7690 0x7ff8640d33a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 -- 172.21.15.37:0/1461749788 >> 172.21.15.37:0/1461749788 conn(0x7ff8640d0c70 msgr2=0x7ff8640d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 -- 172.21.15.37:0/1461749788 shutdown_connections 2024-09-21T06:33:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.188+0000 7ff869e00640 1 -- 172.21.15.37:0/1461749788 wait complete. 2024-09-21T06:33:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.190+0000 7ff869e00640 1 Processor -- start 2024-09-21T06:33:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.191+0000 7ff869e00640 1 -- start start 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8640b7690 0x7ff86415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8640d38e0 0x7ff8640cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8641625c0 con 0x7ff8640b7aa0 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff864162440 con 0x7ff8640d38e0 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff869e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff864162740 con 0x7ff8640b7690 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff85be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff85be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42184/0 (socket says 172.21.15.37:42184) 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff85be00640 1 -- 172.21.15.37:0/3666597115 learned_addr learned my addr 172.21.15.37:0/3666597115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.192+0000 7ff868a00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8640b7690 0x7ff86415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff869400640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8640d38e0 0x7ff8640cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 -- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8640b7690 msgr2=0x7ff86415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8640b7690 0x7ff86415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 -- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8640d38e0 msgr2=0x7ff8640cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8640d38e0 0x7ff8640cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 -- 172.21.15.37:0/3666597115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8641f97e0 con 0x7ff8640b7aa0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff85be00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b420 tx=0x7ff84c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff861600640 1 -- 172.21.15.37:0/3666597115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c011830 con 0x7ff8640b7aa0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff861600640 1 -- 172.21.15.37:0/3666597115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff84c016070 con 0x7ff8640b7aa0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff861600640 1 -- 172.21.15.37:0/3666597115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c011d60 con 0x7ff8640b7aa0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.193+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8640c7ad0 con 0x7ff8640b7aa0 2024-09-21T06:33:59.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.194+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8640cbe20 con 0x7ff8640b7aa0 2024-09-21T06:33:59.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.196+0000 7ff861600640 1 -- 172.21.15.37:0/3666597115 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff84c01b460 con 0x7ff8640b7aa0 2024-09-21T06:33:59.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.197+0000 7ff861600640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 0x7ff84007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.197+0000 7ff861600640 1 -- 172.21.15.37:0/3666597115 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff84c09a8b0 con 0x7ff8640b7aa0 2024-09-21T06:33:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.198+0000 7ff87241d740 1 Processor -- start 2024-09-21T06:33:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.198+0000 7ff868a00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 0x7ff84007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.198+0000 7ff87241d740 1 -- start start 2024-09-21T06:33:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.199+0000 7ff868a00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 0x7ff84007b0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff854033000 tx=0x7ff854004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.199+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7ff270 0x56466fbcfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.199+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7d63b0 0x56466fbd00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff87241d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff87241d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56466fb29430 con 0x56466f7de590 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56466fb195d0 con 0x56466f7ff270 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56466fb08a00 con 0x56466f7d63b0 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42200/0 (socket says 172.21.15.37:42200) 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 -- 172.21.15.37:0/2924145890 learned_addr learned my addr 172.21.15.37:0/2924145890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff85be00640 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7d63b0 0x56466fbd00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff868a00640 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7ff270 0x56466fbcfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 -- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7d63b0 msgr2=0x56466fbd00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7d63b0 0x56466fbd00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 -- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7ff270 msgr2=0x56466fbcfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7ff270 0x56466fbcfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.200+0000 7ff869400640 1 -- 172.21.15.37:0/2924145890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466fad1520 con 0x56466f7de590 2024-09-21T06:33:59.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.201+0000 7ff869400640 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009f20 tx=0x7ff85c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3b11174cd2a8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.201+0000 7ff859200640 1 -- 172.21.15.37:0/2924145890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c016070 con 0x56466f7de590 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.201+0000 7ff859200640 1 -- 172.21.15.37:0/2924145890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff85c004e10 con 0x56466f7de590 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.201+0000 7ff859200640 1 -- 172.21.15.37:0/2924145890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c005240 con 0x56466f7de590 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 -- 172.21.15.37:0/2924145890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 msgr2=0x56466fba0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009f20 tx=0x7ff85c010800 comp rx=0 tx=0).stop 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 -- 172.21.15.37:0/2924145890 shutdown_connections 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fba0150 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7d63b0 0x56466fbd00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 --2- 172.21.15.37:0/2924145890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7ff270 0x56466fbcfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 -- 172.21.15.37:0/2924145890 >> 172.21.15.37:0/2924145890 conn(0x56466f7f33c0 msgr2=0x56466fba5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 -- 172.21.15.37:0/2924145890 shutdown_connections 2024-09-21T06:33:59.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.202+0000 7ff87241d740 1 -- 172.21.15.37:0/2924145890 wait complete. 2024-09-21T06:33:59.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.203+0000 7ff87241d740 1 -- start start 2024-09-21T06:33:59.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.203+0000 7ff87241d740 1 Processor -- start 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fbfa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7de590 0x56466fbfd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56466faaf330 con 0x56466f7ff270 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56466fb0c810 con 0x56466f7d63b0 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56466f879620 con 0x56466f7de590 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff87241d740 10 client.0 _ll_register_callbacks cb 0x56466fab4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff869400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff869400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42214/0 (socket says 172.21.15.37:42214) 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff869400640 1 -- 172.21.15.37:0/2248573546 learned_addr learned my addr 172.21.15.37:0/2248573546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff85be00640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7de590 0x56466fbfd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.205+0000 7ff868a00640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fbfa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 -- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7de590 msgr2=0x56466fbfd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7de590 0x56466fbfd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 -- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 msgr2=0x56466fbfa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fbfa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 -- 172.21.15.37:0/2248573546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466f86a850 con 0x56466f7ff270 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff868a00640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fbfa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff869400640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009ef0 tx=0x7ff85c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412e8ff32b203eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff851600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c016070 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c004e10 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff85c025c60 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff85c025e60 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff87241d740 10 client.7545 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.206+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x56466f8e16b0 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff85c004e10 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7ff85c02c050 con 0x56466f7ff270 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 10 client.7545 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 10 client.7545 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 20 client.7545 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466f7bde40 0x56466fc07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 4 client.7545 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56466fb1b520 con 0x56466f7bde40 2024-09-21T06:33:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.207+0000 7ff868a00640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466f7bde40 0x56466fc07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.208+0000 7ff868a00640 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466f7bde40 0x56466fc07790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea370b4e5eb63293 server_cookie=dc45ca3062a6b5f4 in_seq=0 out_seq=0 2024-09-21T06:33:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.208+0000 7ff851600640 10 client.7545 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.209+0000 7ff851600640 1 -- 172.21.15.37:0/2248573546 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56466fb1b520 con 0x56466f7bde40 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.209+0000 7ff851600640 10 client.7545 handle_command_reply: tid=0 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.209+0000 7ff87241d740 1 client.7545 shutdown 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 msgr2=0x56466fbfd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009ef0 tx=0x7ff85c004bb0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 shutdown_connections 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466f7bde40 0x56466fc07790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7ff270 0x56466fbfd840 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7de590 0x56466fbfd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 --2- 172.21.15.37:0/2248573546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fbfa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff851600640 0 client.7545 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:33:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 >> 172.21.15.37:0/2248573546 conn(0x56466f7f33c0 msgr2=0x56466fbedef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 shutdown_connections 2024-09-21T06:33:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 1 -- 172.21.15.37:0/2248573546 wait complete. 2024-09-21T06:33:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.210+0000 7ff87241d740 20 client.7545 trim_cache size 0 max 16384 2024-09-21T06:33:59.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.229+0000 7ff87241d740 1 Processor -- start 2024-09-21T06:33:59.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.229+0000 7ff87241d740 1 -- start start 2024-09-21T06:33:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7de590 0x56466fbee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fbc5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8641f5e50 con 0x56466f7d63b0 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff864201eb0 con 0x56466f7de590 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff8641506c0 con 0x56466f7ff270 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42216/0 (socket says 172.21.15.37:42216) 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 -- 172.21.15.37:0/795596604 learned_addr learned my addr 172.21.15.37:0/795596604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff85be00640 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7de590 0x56466fbee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff869400640 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fbc5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 -- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 msgr2=0x56466fbc5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fbc5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 -- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7de590 msgr2=0x56466fbee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7de590 0x56466fbee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.231+0000 7ff868a00640 1 -- 172.21.15.37:0/795596604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466fbeab60 con 0x56466f7d63b0 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.232+0000 7ff868a00640 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff854009230 tx=0x7ff85403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6f702bd29f1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.232+0000 7ff853e00640 1 -- 172.21.15.37:0/795596604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854035040 con 0x56466f7d63b0 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.232+0000 7ff853e00640 1 -- 172.21.15.37:0/795596604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff854058070 con 0x56466f7d63b0 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.232+0000 7ff853e00640 1 -- 172.21.15.37:0/795596604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854003dc0 con 0x56466f7d63b0 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 -- 172.21.15.37:0/795596604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 msgr2=0x56466fbf98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff854009230 tx=0x7ff85403d000 comp rx=0 tx=0).stop 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 -- 172.21.15.37:0/795596604 shutdown_connections 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fbc5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7de590 0x56466fbee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 --2- 172.21.15.37:0/795596604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7d63b0 0x56466fbf98f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 -- 172.21.15.37:0/795596604 >> 172.21.15.37:0/795596604 conn(0x56466f7f33c0 msgr2=0x56466fbc7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 -- 172.21.15.37:0/795596604 shutdown_connections 2024-09-21T06:33:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.233+0000 7ff87241d740 1 -- 172.21.15.37:0/795596604 wait complete. 2024-09-21T06:33:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.234+0000 7ff87241d740 1 -- start start 2024-09-21T06:33:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.234+0000 7ff87241d740 1 Processor -- start 2024-09-21T06:33:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fba0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fc6dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56466fbb70e0 con 0x56466f7de590 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56466fbc89a0 con 0x56466f7d63b0 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56466fbc94b0 con 0x56466f7ff270 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.235+0000 7ff87241d740 10 client.0 _ll_register_callbacks cb 0x7ff85c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42232/0 (socket says 172.21.15.37:42232) 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 -- 172.21.15.37:0/3750883084 learned_addr learned my addr 172.21.15.37:0/3750883084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff868a00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fba0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff869400640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fc6dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 -- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 msgr2=0x56466fc6dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fc6dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 -- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 msgr2=0x56466fba0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fba0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 -- 172.21.15.37:0/3750883084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466fba6840 con 0x56466f7de590 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff868a00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fba0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff85be00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff84c062360 tx=0x7ff84c05ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f7503c558d2e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.236+0000 7ff851600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c066f30 con 0x56466f7de590 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff84c061760 con 0x56466f7de590 2024-09-21T06:33:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff854058070 con 0x56466f7de590 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff84c083050 con 0x56466f7de590 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 10 client.7551 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x56466fc6d840 con 0x56466f7de590 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff854058070 con 0x56466f7de590 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7ff84c069130 con 0x56466f7de590 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 10 client.7551 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 10 client.7551 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 20 client.7551 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466fbfd250 0x56466fbb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 4 client.7551 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.237+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "status"}) -- 0x56466fb16fd0 con 0x56466fbfd250 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.238+0000 7ff868a00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466fbfd250 0x56466fbb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.238+0000 7ff868a00640 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466fbfd250 0x56466fbb4d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c946286e14fb58be server_cookie=ddb3463652c3a327 in_seq=0 out_seq=0 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.238+0000 7ff851600640 10 client.7551 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff851600640 1 -- 172.21.15.37:0/3750883084 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56466fb16fd0 con 0x56466fbfd250 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff851600640 10 client.7551 handle_command_reply: tid=0 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 client.7551 shutdown 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 msgr2=0x56466fc63d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff84c062360 tx=0x7ff84c05ea40 comp rx=0 tx=0).stop 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 shutdown_connections 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x56466fbfd250 0x56466fbb4d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56466f7de590 0x56466fc63d70 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff851600640 0 client.7551 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56466f7ff270 0x56466fc6dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 --2- 172.21.15.37:0/3750883084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56466f7d63b0 0x56466fba0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 >> 172.21.15.37:0/3750883084 conn(0x56466f7f33c0 msgr2=0x56466fbcfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 shutdown_connections 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 1 -- 172.21.15.37:0/3750883084 wait complete. 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.239+0000 7ff87241d740 20 client.7551 trim_cache size 0 max 16384 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7096, 2024-09-21T06:33:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1255266050000001, 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 94, 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 91, 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.164122479 2024-09-21T06:33:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 msgr2=0x7ff84007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 0x7ff84007b0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff854033000 tx=0x7ff854004730 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 msgr2=0x7ff8640cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b420 tx=0x7ff84c00b8e0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 shutdown_connections 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff840078bc0 0x7ff84007b0d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640b7aa0 0x7ff8640cb230 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8640d38e0 0x7ff8640cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 --2- 172.21.15.37:0/3666597115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8640b7690 0x7ff86415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 >> 172.21.15.37:0/3666597115 conn(0x7ff8640d0c70 msgr2=0x7ff86415c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 shutdown_connections 2024-09-21T06:33:59.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.240+0000 7ff869e00640 1 -- 172.21.15.37:0/3666597115 wait complete. 2024-09-21T06:33:59.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:33:59.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 -- 172.21.15.37:0/979383097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc0b9dd0 msgr2=0x7fd4bc162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 --2- 172.21.15.37:0/979383097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc0b9dd0 0x7fd4bc162230 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009830 tx=0x7fd4a402a5c0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 -- 172.21.15.37:0/979383097 shutdown_connections 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 --2- 172.21.15.37:0/979383097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc0b9dd0 0x7fd4bc162230 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 -- 172.21.15.37:0/979383097 >> 172.21.15.37:0/979383097 conn(0x7fd4bc0d42e0 msgr2=0x7fd4bc0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 -- 172.21.15.37:0/979383097 shutdown_connections 2024-09-21T06:33:59.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.494+0000 7fd4c1000640 1 -- 172.21.15.37:0/979383097 wait complete. 2024-09-21T06:33:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.496+0000 7fd4c1000640 1 Processor -- start 2024-09-21T06:33:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.496+0000 7fd4c1000640 1 -- start start 2024-09-21T06:33:59.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0b9dd0 0x7fd4bc1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc0d2ba0 0x7fd4bc0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1555b0 con 0x7fd4bc162770 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc06e000 con 0x7fd4bc0d2ba0 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4c1000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd4bc155730 con 0x7fd4bc0b9dd0 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42246/0 (socket says 172.21.15.37:42246) 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4bbe00640 1 -- 172.21.15.37:0/1585240853 learned_addr learned my addr 172.21.15.37:0/1585240853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc0d2ba0 0x7fd4bc0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.498+0000 7fd4bb400640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0b9dd0 0x7fd4bc1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 -- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0b9dd0 msgr2=0x7fd4bc1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0b9dd0 0x7fd4bc1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 -- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc0d2ba0 msgr2=0x7fd4bc0cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc0d2ba0 0x7fd4bc0cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 -- 172.21.15.37:0/1585240853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc0cc180 con 0x7fd4bc162770 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cb10 tx=0x7fd4b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4b3400640 1 -- 172.21.15.37:0/1585240853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4016020 con 0x7fd4bc162770 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4b3400640 1 -- 172.21.15.37:0/1585240853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b400ad60 con 0x7fd4bc162770 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4b3400640 1 -- 172.21.15.37:0/1585240853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4012680 con 0x7fd4bc162770 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.499+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc142930 con 0x7fd4bc162770 2024-09-21T06:33:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.500+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc142ef0 con 0x7fd4bc162770 2024-09-21T06:33:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.502+0000 7fd4b3400640 1 -- 172.21.15.37:0/1585240853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4b4019020 con 0x7fd4bc162770 2024-09-21T06:33:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.503+0000 7fd4b3400640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 0x7fd49807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.503+0000 7fd4b3400640 1 -- 172.21.15.37:0/1585240853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd4b409b020 con 0x7fd4bc162770 2024-09-21T06:33:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.503+0000 7fd4c9674740 1 Processor -- start 2024-09-21T06:33:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.503+0000 7fd4bb400640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 0x7fd49807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.504+0000 7fd4c9674740 1 -- start start 2024-09-21T06:33:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.504+0000 7fd4bb400640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 0x7fd49807b0d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001720 tx=0x7fd4ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 0x55eef0558b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eef04b2430 con 0x55eef0167590 2024-09-21T06:33:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eef04a25d0 con 0x55eef015f3b0 2024-09-21T06:33:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eef0491a00 con 0x55eef0188270 2024-09-21T06:33:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42258/0 (socket says 172.21.15.37:42258) 2024-09-21T06:33:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.505+0000 7fd4bbe00640 1 -- 172.21.15.37:0/2565832885 learned_addr learned my addr 172.21.15.37:0/2565832885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bb400640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 0x55eef0558b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 -- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 msgr2=0x55eef0558b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 0x55eef0558b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 -- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 msgr2=0x55eef05590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 -- 172.21.15.37:0/2565832885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef045a520 con 0x55eef0167590 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bb400640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 0x55eef0558b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4062970 tx=0x7fd4b4062780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce6d941dbe9ef27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4abe00640 1 -- 172.21.15.37:0/2565832885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b40676a0 con 0x55eef0167590 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4abe00640 1 -- 172.21.15.37:0/2565832885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b4067070 con 0x55eef0167590 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.506+0000 7fd4abe00640 1 -- 172.21.15.37:0/2565832885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4067460 con 0x55eef0167590 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 -- 172.21.15.37:0/2565832885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 msgr2=0x55eef0529150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4062970 tx=0x7fd4b4062780 comp rx=0 tx=0).stop 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 -- 172.21.15.37:0/2565832885 shutdown_connections 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0529150 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 --2- 172.21.15.37:0/2565832885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0188270 0x55eef0558b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 -- 172.21.15.37:0/2565832885 >> 172.21.15.37:0/2565832885 conn(0x55eef017c3c0 msgr2=0x55eef052eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 -- 172.21.15.37:0/2565832885 shutdown_connections 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.507+0000 7fd4c9674740 1 -- 172.21.15.37:0/2565832885 wait complete. 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.508+0000 7fd4c9674740 1 -- start start 2024-09-21T06:33:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.509+0000 7fd4c9674740 1 Processor -- start 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 0x55eef054f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eef0438330 con 0x55eef0167590 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eef0495810 con 0x55eef0188270 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eef0202620 con 0x55eef015f3b0 2024-09-21T06:33:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4c9674740 10 client.0 _ll_register_callbacks cb 0x55eef043d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42270/0 (socket says 172.21.15.37:42270) 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.510+0000 7fd4b3e00640 1 -- 172.21.15.37:0/3702678556 learned_addr learned my addr 172.21.15.37:0/3702678556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 0x55eef054f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 -- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 msgr2=0x55eef056df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 -- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 msgr2=0x55eef054f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 0x55eef054f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 -- 172.21.15.37:0/3702678556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef01f3850 con 0x55eef0167590 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 0x55eef054f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40095f0 tx=0x7fd4a4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6d002a5e9920c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.511+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4005020 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b4067070 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd4a40051c0 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd4a404c050 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 10 client.7566 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55eef026a6b0 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b4067070 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7fd4a404f020 con 0x55eef0167590 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 10 client.7566 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 10 client.7566 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:33:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 20 client.7566 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef0146e40 0x55eef0582dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 4 client.7566 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.512+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef04a4520 con 0x55eef0146e40 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.513+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef0146e40 0x55eef0582dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.513+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef0146e40 0x55eef0582dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=868b2c8f1c0a63e3 server_cookie=eb9a69101e69526e in_seq=0 out_seq=0 2024-09-21T06:33:59.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.513+0000 7fd4a8c00640 10 client.7566 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.514+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3702678556 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eef04a4520 con 0x55eef0146e40 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.514+0000 7fd4a8c00640 10 client.7566 handle_command_reply: tid=0 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.514+0000 7fd4c9674740 1 client.7566 shutdown 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.514+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 msgr2=0x55eef0554810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.514+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40095f0 tx=0x7fd4a4003820 comp rx=0 tx=0).stop 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 shutdown_connections 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef0146e40 0x55eef0582dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0167590 0x55eef0554810 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0188270 0x55eef054f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3702678556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 >> 172.21.15.37:0/3702678556 conn(0x55eef017c3c0 msgr2=0x55eef0558b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4a8c00640 0 client.7566 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 shutdown_connections 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 1 -- 172.21.15.37:0/3702678556 wait complete. 2024-09-21T06:33:59.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.515+0000 7fd4c9674740 20 client.7566 trim_cache size 0 max 16384 2024-09-21T06:33:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.534+0000 7fd4c9674740 1 Processor -- start 2024-09-21T06:33:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.534+0000 7fd4c9674740 1 -- start start 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc2018a0 con 0x55eef0188270 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc201f70 con 0x55eef0167590 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd4bc200ab0 con 0x55eef015f3b0 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42284/0 (socket says 172.21.15.37:42284) 2024-09-21T06:33:59.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3829241 learned_addr learned my addr 172.21.15.37:0/3829241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 msgr2=0x55eef056dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 msgr2=0x55eef0551590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3829241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef052ab60 con 0x55eef0188270 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.536+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd4b405fee0 tx=0x7fd4b40991f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e81be6d75f4de29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4ab400640 1 -- 172.21.15.37:0/3829241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4080030 con 0x55eef0188270 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4ab400640 1 -- 172.21.15.37:0/3829241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b4067040 con 0x55eef0188270 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4ab400640 1 -- 172.21.15.37:0/3829241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b40671f0 con 0x55eef0188270 2024-09-21T06:33:59.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 -- 172.21.15.37:0/3829241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 msgr2=0x55eef0551ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd4b405fee0 tx=0x7fd4b40991f0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 -- 172.21.15.37:0/3829241 shutdown_connections 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.537+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3829241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef015f3b0 0x55eef056dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.538+0000 7fd4c9674740 1 -- 172.21.15.37:0/3829241 >> 172.21.15.37:0/3829241 conn(0x55eef017c3c0 msgr2=0x55eef0524660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.538+0000 7fd4c9674740 1 -- 172.21.15.37:0/3829241 shutdown_connections 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.538+0000 7fd4c9674740 1 -- 172.21.15.37:0/3829241 wait complete. 2024-09-21T06:33:59.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.539+0000 7fd4c9674740 1 -- start start 2024-09-21T06:33:59.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.539+0000 7fd4c9674740 1 Processor -- start 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eef05870d0 con 0x55eef0188270 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eef0554f10 con 0x55eef015f3b0 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55eef05862c0 con 0x55eef0167590 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.540+0000 7fd4c9674740 10 client.0 _ll_register_callbacks cb 0x7fd4a4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42300/0 (socket says 172.21.15.37:42300) 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3911175544 learned_addr learned my addr 172.21.15.37:0/3911175544 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4b3e00640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0167590 msgr2=0x55eef0551590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 msgr2=0x55eef05ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 -- 172.21.15.37:0/3911175544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef052c930 con 0x55eef0188270 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4bbe00640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cae0 tx=0x7fd4b4067480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647e6d0fbe5c244e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.541+0000 7fd4a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4080030 con 0x55eef0188270 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd4b405eee0 con 0x55eef0188270 2024-09-21T06:33:59.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b4067040 con 0x55eef0188270 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd4b40620e0 con 0x55eef0188270 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 10 client.7572 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55eef052df40 con 0x55eef0188270 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b4067040 con 0x55eef0188270 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7fd4b4078ce0 con 0x55eef0188270 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 10 client.7572 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 10 client.7572 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 20 client.7572 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef05f3440 0x55eef05f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 4 client.7572 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "status"}) -- 0x55eef049ffd0 con 0x55eef05f3440 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.542+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef05f3440 0x55eef05f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4bb400640 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef05f3440 0x55eef05f3850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9439b0da5e91cc server_cookie=54e1e8f1a38247f9 in_seq=0 out_seq=0 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4a8c00640 10 client.7572 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4a8c00640 1 -- 172.21.15.37:0/3911175544 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55eef049ffd0 con 0x55eef05f3440 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4a8c00640 10 client.7572 handle_command_reply: tid=0 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4c9674740 1 client.7572 shutdown 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 msgr2=0x55eef0551ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cae0 tx=0x7fd4b4067480 comp rx=0 tx=0).stop 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.543+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 shutdown_connections 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x55eef05f3440 0x55eef05f3850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eef0188270 0x55eef0551ad0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eef0167590 0x55eef0551590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 --2- 172.21.15.37:0/3911175544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eef015f3b0 0x55eef05ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4a8c00640 0 client.7572 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 >> 172.21.15.37:0/3911175544 conn(0x55eef017c3c0 msgr2=0x55eef054efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 shutdown_connections 2024-09-21T06:33:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 1 -- 172.21.15.37:0/3911175544 wait complete. 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c9674740 20 client.7572 trim_cache size 0 max 16384 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7389, 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.4417788809999998, 2024-09-21T06:33:59.549 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 94, 2024-09-21T06:33:59.550 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-09-21T06:33:59.550 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 91, 2024-09-21T06:33:59.550 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.471352842999998 2024-09-21T06:33:59.550 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:59.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 msgr2=0x7fd49807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 0x7fd49807b0d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001720 tx=0x7fd4ac009040 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 msgr2=0x7fd4bc1f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.544+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cb10 tx=0x7fd4b400a060 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 shutdown_connections 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd498078bc0 0x7fd49807b0d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4bc162770 0x7fd4bc1f8720 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc0d2ba0 0x7fd4bc0cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 --2- 172.21.15.37:0/1585240853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0b9dd0 0x7fd4bc1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 >> 172.21.15.37:0/1585240853 conn(0x7fd4bc0d42e0 msgr2=0x7fd4bc15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 shutdown_connections 2024-09-21T06:33:59.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.545+0000 7fd4c1000640 1 -- 172.21.15.37:0/1585240853 wait complete. 2024-09-21T06:33:59.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:33:59.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 -- 172.21.15.37:0/807394904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01006ddf0 msgr2=0x7fb01006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 --2- 172.21.15.37:0/807394904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01006ddf0 0x7fb01006e200 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7faff8009830 tx=0x7faff802a5c0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 -- 172.21.15.37:0/807394904 shutdown_connections 2024-09-21T06:33:59.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 --2- 172.21.15.37:0/807394904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01006ddf0 0x7fb01006e200 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 -- 172.21.15.37:0/807394904 >> 172.21.15.37:0/807394904 conn(0x7fb0100b1b90 msgr2=0x7fb0100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 -- 172.21.15.37:0/807394904 shutdown_connections 2024-09-21T06:33:59.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.792+0000 7fb017400640 1 -- 172.21.15.37:0/807394904 wait complete. 2024-09-21T06:33:59.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.794+0000 7fb017400640 1 Processor -- start 2024-09-21T06:33:59.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.794+0000 7fb017400640 1 -- start start 2024-09-21T06:33:59.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01006ddf0 0x7fb0101571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0100cef00 0x7fb0100b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb010160fd0 con 0x7fb010156a50 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb010160e50 con 0x7fb0100cef00 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb017400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100cfa20 con 0x7fb01006ddf0 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb015400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb015400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42318/0 (socket says 172.21.15.37:42318) 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb015400640 1 -- 172.21.15.37:0/3860147628 learned_addr learned my addr 172.21.15.37:0/3860147628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb007e00640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0100cef00 0x7fb0100b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.796+0000 7fb014a00640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01006ddf0 0x7fb0101571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 -- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01006ddf0 msgr2=0x7fb0101571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01006ddf0 0x7fb0101571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 -- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0100cef00 msgr2=0x7fb0100b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0100cef00 0x7fb0100b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 -- 172.21.15.37:0/3860147628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0101f9d70 con 0x7fb010156a50 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb015400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c730 tx=0x7fb00800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb00d600640 1 -- 172.21.15.37:0/3860147628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008003b70 con 0x7fb010156a50 2024-09-21T06:33:59.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb00d600640 1 -- 172.21.15.37:0/3860147628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb008003d30 con 0x7fb010156a50 2024-09-21T06:33:59.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101f8fb0 con 0x7fb010156a50 2024-09-21T06:33:59.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb00d600640 1 -- 172.21.15.37:0/3860147628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0080052d0 con 0x7fb010156a50 2024-09-21T06:33:59.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.797+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101f9570 con 0x7fb010156a50 2024-09-21T06:33:59.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.801+0000 7fb00d600640 1 -- 172.21.15.37:0/3860147628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb008005680 con 0x7fb010156a50 2024-09-21T06:33:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.802+0000 7fb00d600640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 0x7fafec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.802+0000 7fb01e408740 1 Processor -- start 2024-09-21T06:33:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.802+0000 7fb00d600640 1 -- 172.21.15.37:0/3860147628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb00809ad10 con 0x7fb010156a50 2024-09-21T06:33:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.802+0000 7fb01e408740 1 -- start start 2024-09-21T06:33:59.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.803+0000 7fb014a00640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 0x7fafec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.803+0000 7fb014a00640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 0x7fafec07b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb000000f60 tx=0x7fb000009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad39970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55897ac93430 con 0x55897a9403b0 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55897ac835d0 con 0x55897a948590 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55897ac72a00 con 0x55897a969270 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42330/0 (socket says 172.21.15.37:42330) 2024-09-21T06:33:59.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 -- 172.21.15.37:0/1478562078 learned_addr learned my addr 172.21.15.37:0/1478562078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb015400640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb014a00640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad39970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 -- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 msgr2=0x55897ad39970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad39970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 -- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 msgr2=0x55897ad09f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad09f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb007e00640 1 -- 172.21.15.37:0/1478562078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897ac3b520 con 0x55897a9403b0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.804+0000 7fb015400640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad09f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.805+0000 7fb007e00640 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7faff8009950 tx=0x7faff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaa0d1401dbc422 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.805+0000 7fb005200640 1 -- 172.21.15.37:0/1478562078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8005220 con 0x55897a9403b0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.805+0000 7fb005200640 1 -- 172.21.15.37:0/1478562078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faff8040030 con 0x55897a9403b0 2024-09-21T06:33:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.805+0000 7fb005200640 1 -- 172.21.15.37:0/1478562078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80047a0 con 0x55897a9403b0 2024-09-21T06:33:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 -- 172.21.15.37:0/1478562078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 msgr2=0x55897ad39eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7faff8009950 tx=0x7faff80046c0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 -- 172.21.15.37:0/1478562078 shutdown_connections 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39eb0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad09f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 --2- 172.21.15.37:0/1478562078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad39970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 -- 172.21.15.37:0/1478562078 >> 172.21.15.37:0/1478562078 conn(0x55897a95d3c0 msgr2=0x55897ad0f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 -- 172.21.15.37:0/1478562078 shutdown_connections 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.806+0000 7fb01e408740 1 -- 172.21.15.37:0/1478562078 wait complete. 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.807+0000 7fb01e408740 1 -- start start 2024-09-21T06:33:59.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.807+0000 7fb01e408740 1 Processor -- start 2024-09-21T06:33:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897ad30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a969270 0x55897ad36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55897ac19330 con 0x55897a9403b0 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55897ac76810 con 0x55897a969270 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55897a9e3620 con 0x55897a948590 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb01e408740 10 client.0 _ll_register_callbacks cb 0x55897ac1e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42344/0 (socket says 172.21.15.37:42344) 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 -- 172.21.15.37:0/2240492450 learned_addr learned my addr 172.21.15.37:0/2240492450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb015400640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a969270 0x55897ad36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb007e00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897ad30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 -- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 msgr2=0x55897ad30280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897ad30280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 -- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a969270 msgr2=0x55897ad36b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a969270 0x55897ad36b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.809+0000 7fb014a00640 1 -- 172.21.15.37:0/2240492450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897a9d4850 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7fb014a00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb000014700 tx=0x7fb000001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c3162a48be3bea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7faffd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000013680 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff8040030 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb000013820 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb00002e050 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7fb01e408740 10 client.7587 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.810+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55897aa4b6b0 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faff8040030 con 0x55897a9403b0 2024-09-21T06:33:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7fb000031020 con 0x55897a9403b0 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 10 client.7587 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 10 client.7587 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 20 client.7587 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897a927e40 0x55897ad716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 4 client.7587 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897ac85520 con 0x55897a927e40 2024-09-21T06:33:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.811+0000 7fb007e00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897a927e40 0x55897ad716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.812+0000 7fb007e00640 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897a927e40 0x55897ad716c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805cfca2b02a7daf server_cookie=3a2258c0e15929b8 in_seq=0 out_seq=0 2024-09-21T06:33:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.812+0000 7faffd600640 10 client.7587 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.813+0000 7faffd600640 1 -- 172.21.15.37:0/2240492450 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55897ac85520 con 0x55897a927e40 2024-09-21T06:33:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.813+0000 7faffd600640 10 client.7587 handle_command_reply: tid=0 2024-09-21T06:33:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.813+0000 7fb01e408740 1 client.7587 shutdown 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 msgr2=0x55897ad4bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb000014700 tx=0x7fb000001080 comp rx=0 tx=0).stop 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 shutdown_connections 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897a927e40 0x55897ad716c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a969270 0x55897ad36b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897ad30280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 --2- 172.21.15.37:0/2240492450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad4bab0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 >> 172.21.15.37:0/2240492450 conn(0x55897a95d3c0 msgr2=0x55897ad4af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7faffd600640 0 client.7587 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 shutdown_connections 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 1 -- 172.21.15.37:0/2240492450 wait complete. 2024-09-21T06:33:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.814+0000 7fb01e408740 20 client.7587 trim_cache size 0 max 16384 2024-09-21T06:33:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.833+0000 7fb01e408740 1 Processor -- start 2024-09-21T06:33:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.833+0000 7fb01e408740 1 -- start start 2024-09-21T06:33:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad39530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad33470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb010201a60 con 0x55897a9403b0 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb010202130 con 0x55897a948590 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100a0ab0 con 0x55897a969270 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42348/0 (socket says 172.21.15.37:42348) 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 -- 172.21.15.37:0/3728369 learned_addr learned my addr 172.21.15.37:0/3728369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb007e00640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad39530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb015400640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad33470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 -- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 msgr2=0x55897ad33470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad33470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 -- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 msgr2=0x55897ad39530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad39530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb014a00640 1 -- 172.21.15.37:0/3728369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897ad0bcf0 con 0x55897a9403b0 2024-09-21T06:33:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb007e00640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad39530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.835+0000 7fb015400640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.836+0000 7fb014a00640 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb000007ab0 tx=0x7fb000007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5def6e1cac9095b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.836+0000 7fafffe00640 1 -- 172.21.15.37:0/3728369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000036020 con 0x55897a9403b0 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.836+0000 7fafffe00640 1 -- 172.21.15.37:0/3728369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb000026070 con 0x55897a9403b0 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.836+0000 7fafffe00640 1 -- 172.21.15.37:0/3728369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000013a30 con 0x55897a9403b0 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 -- 172.21.15.37:0/3728369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 msgr2=0x55897ad39c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb000007ab0 tx=0x7fb000007df0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 -- 172.21.15.37:0/3728369 shutdown_connections 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a969270 0x55897ad33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a948590 0x55897ad39530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 --2- 172.21.15.37:0/3728369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a9403b0 0x55897ad39c80 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 -- 172.21.15.37:0/3728369 >> 172.21.15.37:0/3728369 conn(0x55897a95d3c0 msgr2=0x55897ad34180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 -- 172.21.15.37:0/3728369 shutdown_connections 2024-09-21T06:33:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.837+0000 7fb01e408740 1 -- 172.21.15.37:0/3728369 wait complete. 2024-09-21T06:33:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.838+0000 7fb01e408740 1 -- start start 2024-09-21T06:33:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.838+0000 7fb01e408740 1 Processor -- start 2024-09-21T06:33:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a9403b0 0x55897add3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897adca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55897ad32d30 con 0x55897a969270 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55897ad4c0a0 con 0x55897a9403b0 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.839+0000 7fb01e408740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55897ad672e0 con 0x55897a948590 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb01e408740 10 client.0 _ll_register_callbacks cb 0x7fb00000fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42352/0 (socket says 172.21.15.37:42352) 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 -- 172.21.15.37:0/1367169236 learned_addr learned my addr 172.21.15.37:0/1367169236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb014a00640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a9403b0 0x55897add3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb007e00640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897adca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 -- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 msgr2=0x55897adca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897adca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 -- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a9403b0 msgr2=0x55897add3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a9403b0 0x55897add3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 -- 172.21.15.37:0/1367169236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897ad10650 con 0x55897a969270 2024-09-21T06:33:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb007e00640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897adca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7fb015400640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb00805fbd0 tx=0x7fb0080620d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2284f16b81f87248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.840+0000 7faffd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008081060 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb008061bc0 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb000026070 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb008067390 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 10 client.7593 fetch_fsmap learned FSMap version 94 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55897add45a0 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb000026070 con 0x55897a969270 2024-09-21T06:33:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 94) ==== 3542+0+0 (secure 0 0 0) 0x7fb00805e2b0 con 0x55897a969270 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 10 client.7593 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 10 client.7593 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 20 client.7593 populate_metadata read hostname 'smithi037' 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897ad4efd0 0x55897ad4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 4 client.7593 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.841+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "status"}) -- 0x55897ac80fd0 con 0x55897ad4efd0 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.842+0000 7fb014a00640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897ad4efd0 0x55897ad4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.842+0000 7fb014a00640 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897ad4efd0 0x55897ad4c400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7026b06a37b12e2 server_cookie=b95ac29a05d3d9d5 in_seq=0 out_seq=0 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.842+0000 7faffd600640 10 client.7593 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7faffd600640 1 -- 172.21.15.37:0/1367169236 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55897ac80fd0 con 0x55897ad4efd0 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7faffd600640 10 client.7593 handle_command_reply: tid=0 2024-09-21T06:33:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 client.7593 shutdown 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 msgr2=0x55897add7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb00805fbd0 tx=0x7fb0080620d0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 shutdown_connections 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55897ad4efd0 0x55897ad4c400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55897a969270 0x55897add7a90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55897a948590 0x55897adca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 --2- 172.21.15.37:0/1367169236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55897a9403b0 0x55897add3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 >> 172.21.15.37:0/1367169236 conn(0x55897a95d3c0 msgr2=0x55897ad4b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7faffd600640 0 client.7593 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 shutdown_connections 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 1 -- 172.21.15.37:0/1367169236 wait complete. 2024-09-21T06:33:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.843+0000 7fb01e408740 20 client.7593 trim_cache size 0 max 16384 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7392, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 12.811745484999999, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 94, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 91, 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.770491402000001 2024-09-21T06:33:59.849 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:33:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 msgr2=0x7fafec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 0x7fafec07b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb000000f60 tx=0x7fb000009040 comp rx=0 tx=0).stop 2024-09-21T06:33:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 msgr2=0x7fb01015dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c730 tx=0x7fb00800cbf0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 shutdown_connections 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fafec078b90 0x7fafec07b0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb010156a50 0x7fb01015dce0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0100cef00 0x7fb0100b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 --2- 172.21.15.37:0/3860147628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01006ddf0 0x7fb0101571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 >> 172.21.15.37:0/3860147628 conn(0x7fb0100b1b90 msgr2=0x7fb0100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 shutdown_connections 2024-09-21T06:33:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:33:59.844+0000 7fb017400640 1 -- 172.21.15.37:0/3860147628 wait complete. 2024-09-21T06:33:59.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 -- 172.21.15.37:0/3923592669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d0070030 msgr2=0x7f46d006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 --2- 172.21.15.37:0/3923592669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d0070030 0x7f46d006ddf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009830 tx=0x7f46b802a520 comp rx=0 tx=0).stop 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 -- 172.21.15.37:0/3923592669 shutdown_connections 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 --2- 172.21.15.37:0/3923592669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d0070030 0x7f46d006ddf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 --2- 172.21.15.37:0/3923592669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0074d90 0x7f46d006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 -- 172.21.15.37:0/3923592669 >> 172.21.15.37:0/3923592669 conn(0x7f46d00cca40 msgr2=0x7f46d00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 -- 172.21.15.37:0/3923592669 shutdown_connections 2024-09-21T06:34:00.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.092+0000 7f46d7800640 1 -- 172.21.15.37:0/3923592669 wait complete. 2024-09-21T06:34:00.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.095+0000 7f46d7800640 1 Processor -- start 2024-09-21T06:34:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.095+0000 7f46d7800640 1 -- start start 2024-09-21T06:34:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d006e330 0x7f46d009ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0070030 0x7f46d00c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015ad50 con 0x7f46d0074d90 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015abd0 con 0x7f46d006e330 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d7800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f46d0070e80 con 0x7f46d0070030 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.096+0000 7f46d5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42368/0 (socket says 172.21.15.37:42368) 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 -- 172.21.15.37:0/1095553429 learned_addr learned my addr 172.21.15.37:0/1095553429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d4e00640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d006e330 0x7f46d009ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46c7e00640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0070030 0x7f46d00c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 -- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0070030 msgr2=0x7f46d00c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0070030 0x7f46d00c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 -- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d006e330 msgr2=0x7f46d009ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d006e330 0x7f46d009ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 -- 172.21.15.37:0/1095553429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d0074910 con 0x7f46d0074d90 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.097+0000 7f46d5800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f46c800cad0 tx=0x7f46c800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.098+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8016020 con 0x7f46d0074d90 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.098+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c800ac80 con 0x7f46d0074d90 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.098+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8012680 con 0x7f46d0074d90 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.098+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d0074670 con 0x7f46d0074d90 2024-09-21T06:34:00.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.098+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01f58e0 con 0x7f46d0074d90 2024-09-21T06:34:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.099+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d006ddf0 con 0x7f46d0074d90 2024-09-21T06:34:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.100+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46c8019020 con 0x7f46d0074d90 2024-09-21T06:34:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.101+0000 7f46cd600640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 0x7f46ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.101+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f46c809b390 con 0x7f46d0074d90 2024-09-21T06:34:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.102+0000 7f46d4e00640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 0x7f46ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.102+0000 7f46d4e00640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 0x7f46ac07b0d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46c00095f0 tx=0x7f46c0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.106+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46c8020020 con 0x7f46d0074d90 2024-09-21T06:34:00.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.345+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d00b5f40 con 0x7f46d0074d90 2024-09-21T06:34:00.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.345+0000 7f46cd600640 1 -- 172.21.15.37:0/1095553429 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5266 (secure 0 0 0) 0x7f46c8062fe0 con 0x7f46d0074d90 2024-09-21T06:34:00.348 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:00.348 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"btime":"2024-09-21T06:33:58:111823+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:33:58.111812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7096,7389,7392]},"id":6}]} 2024-09-21T06:34:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 94 2024-09-21T06:34:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 msgr2=0x7f46ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 0x7f46ac07b0d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46c00095f0 tx=0x7f46c0005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 msgr2=0x7f46d01f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f46c800cad0 tx=0x7f46c800ced0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 shutdown_connections 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f46ac078bc0 0x7f46ac07b0d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46d0074d90 0x7f46d01f5dc0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d0070030 0x7f46d00c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 --2- 172.21.15.37:0/1095553429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d006e330 0x7f46d009ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 >> 172.21.15.37:0/1095553429 conn(0x7f46d00cca40 msgr2=0x7f46d00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 shutdown_connections 2024-09-21T06:34:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:00.348+0000 7f46d7800640 1 -- 172.21.15.37:0/1095553429 wait complete. 2024-09-21T06:34:00.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:00.373 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:34:00.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:00.426 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:34:00.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:00.470 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:34:00.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:00.547 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:34:00.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:00.627 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:34:00.706 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:34:00.955 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:34:00.956 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:00.956 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:34:02.957 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:34:03.213 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:34:03.214 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:03.214 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:34:05.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:34:05.472 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:34:05.472 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:05.472 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:34:07.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-21T06:34:07.723 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:34:07.723 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:34:07.723 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:34:07.980 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:34:07.981 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:07.981 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:34:09.982 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:34:10.241 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:34:10.242 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:10.242 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:34:12.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:34:12.500 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T06:34:12.501 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:34:12.501 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:34:12.501 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:34:12.750 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:34:12.750 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:34:12.751 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:34:12.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.979+0000 7ff189400640 1 -- 172.21.15.37:0/4106601479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841456e0 msgr2=0x7ff1840b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.979+0000 7ff189400640 1 --2- 172.21.15.37:0/4106601479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841456e0 0x7ff1840b4f30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff174009830 tx=0x7ff17402a510 comp rx=0 tx=0).stop 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 -- 172.21.15.37:0/4106601479 shutdown_connections 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 --2- 172.21.15.37:0/4106601479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841429b0 0x7ff18406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 --2- 172.21.15.37:0/4106601479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841456e0 0x7ff1840b4f30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 -- 172.21.15.37:0/4106601479 >> 172.21.15.37:0/4106601479 conn(0x7ff18406f920 msgr2=0x7ff1840b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:12.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 -- 172.21.15.37:0/4106601479 shutdown_connections 2024-09-21T06:34:12.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.980+0000 7ff189400640 1 -- 172.21.15.37:0/4106601479 wait complete. 2024-09-21T06:34:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.982+0000 7ff189400640 1 Processor -- start 2024-09-21T06:34:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.982+0000 7ff189400640 1 -- start start 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1840cc8d0 0x7ff1840a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 0x7ff1840cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840ce150 con 0x7ff1841429b0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840cdfd0 con 0x7ff1840cc8d0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.983+0000 7ff189400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff1840ce2d0 con 0x7ff1841456e0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56654/0 (socket says 172.21.15.37:56654) 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 -- 172.21.15.37:0/1382922971 learned_addr learned my addr 172.21.15.37:0/1382922971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff183e00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1840cc8d0 0x7ff1840a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff188a00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 0x7ff1840cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 -- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 msgr2=0x7ff1840cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 0x7ff1840cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 -- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1840cc8d0 msgr2=0x7ff1840a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1840cc8d0 0x7ff1840a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 -- 172.21.15.37:0/1382922971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1840a9980 con 0x7ff1841429b0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff188a00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 0x7ff1840cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.984+0000 7ff17be00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e7c0 tx=0x7ff16c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.985+0000 7ff180c00640 1 -- 172.21.15.37:0/1382922971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00ccc0 con 0x7ff1841429b0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.985+0000 7ff180c00640 1 -- 172.21.15.37:0/1382922971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c00ee70 con 0x7ff1841429b0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.985+0000 7ff180c00640 1 -- 172.21.15.37:0/1382922971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c010690 con 0x7ff1841429b0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.985+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1840a9e00 con 0x7ff1841429b0 2024-09-21T06:34:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.985+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1840aa3c0 con 0x7ff1841429b0 2024-09-21T06:34:12.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.988+0000 7ff180c00640 1 -- 172.21.15.37:0/1382922971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff16c010830 con 0x7ff1841429b0 2024-09-21T06:34:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.989+0000 7ff180c00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 0x7ff16007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.990+0000 7ff180c00640 1 -- 172.21.15.37:0/1382922971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff16c09b850 con 0x7ff1841429b0 2024-09-21T06:34:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.990+0000 7ff191a72740 1 Processor -- start 2024-09-21T06:34:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.990+0000 7ff191a72740 1 -- start start 2024-09-21T06:34:12.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.990+0000 7ff183e00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 0x7ff16007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff183e00640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 0x7ff16007b0a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff174004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71728730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713196b0 0x557c7170b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c716411c0 con 0x557c713114d0 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c71654870 con 0x557c7133a390 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c71643ca0 con 0x557c713196b0 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.991+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56662/0 (socket says 172.21.15.37:56662) 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 -- 172.21.15.37:0/891734776 learned_addr learned my addr 172.21.15.37:0/891734776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff188a00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713196b0 0x557c7170b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff183e00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71728730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 -- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713196b0 msgr2=0x557c7170b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713196b0 0x557c7170b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 -- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 msgr2=0x557c71728730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71728730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 -- 172.21.15.37:0/891734776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7141c7d0 con 0x557c713114d0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff183e00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71728730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff17be00640 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff16c063430 tx=0x7ff16c062fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c633ef4bde959a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff173e00640 1 -- 172.21.15.37:0/891734776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c067ed0 con 0x557c713114d0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff173e00640 1 -- 172.21.15.37:0/891734776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c06cd80 con 0x557c713114d0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.992+0000 7ff173e00640 1 -- 172.21.15.37:0/891734776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c085da0 con 0x557c713114d0 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 -- 172.21.15.37:0/891734776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 msgr2=0x557c7170b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff16c063430 tx=0x7ff16c062fb0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 -- 172.21.15.37:0/891734776 shutdown_connections 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713114d0 0x557c7170b400 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713196b0 0x557c7170b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 --2- 172.21.15.37:0/891734776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71728730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 -- 172.21.15.37:0/891734776 >> 172.21.15.37:0/891734776 conn(0x557c7132e4e0 msgr2=0x557c716e1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 -- 172.21.15.37:0/891734776 shutdown_connections 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.993+0000 7ff191a72740 1 -- 172.21.15.37:0/891734776 wait complete. 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.994+0000 7ff191a72740 1 -- start start 2024-09-21T06:34:12.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.995+0000 7ff191a72740 1 Processor -- start 2024-09-21T06:34:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71725680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71725fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c715ea410 con 0x557c713196b0 2024-09-21T06:34:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c71647ab0 con 0x557c7133a390 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c713b4740 con 0x557c713114d0 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff191a72740 10 client.0 _ll_register_callbacks cb 0x557c715c7360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.996+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff183e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71725680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56676/0 (socket says 172.21.15.37:56676) 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff183e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71725680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:45994/0 (socket says 172.21.15.37:45994) 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 -- 172.21.15.37:0/474132488 learned_addr learned my addr 172.21.15.37:0/474132488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff188a00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71725fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 -- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 msgr2=0x557c71725680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71725680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 -- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 msgr2=0x557c71725fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71725fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 -- 172.21.15.37:0/474132488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c713a5970 con 0x557c713196b0 2024-09-21T06:34:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff188a00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71725fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff17be00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff16c087040 tx=0x7ff16c062930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d831535dbbeea16c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.997+0000 7ff170c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c067ed0 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff16c010d30 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c06cd80 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7ff16c06a0e0 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 10 client.7614 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x557c713836a0 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c06cd80 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7ff16c06a280 con 0x557c713196b0 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 10 client.7614 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 10 client.7614 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 20 client.7614 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c712f8f60 0x557c71723ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 4 client.7614 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.998+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c716567c0 con 0x557c712f8f60 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.999+0000 7ff183e00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c712f8f60 0x557c71723ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.999+0000 7ff183e00640 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c712f8f60 0x557c71723ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecdcfb338c84c3b server_cookie=9c33fc164108cdc6 in_seq=0 out_seq=0 2024-09-21T06:34:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:12.999+0000 7ff170c00640 10 client.7614 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.000+0000 7ff170c00640 1 -- 172.21.15.37:0/474132488 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c716567c0 con 0x557c712f8f60 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.000+0000 7ff170c00640 10 client.7614 handle_command_reply: tid=0 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 client.7614 shutdown 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 msgr2=0x557c71722020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff16c087040 tx=0x7ff16c062930 comp rx=0 tx=0).stop 2024-09-21T06:34:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 shutdown_connections 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c712f8f60 0x557c71723ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c71722020 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71725fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff170c00640 0 client.7614 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 --2- 172.21.15.37:0/474132488 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71725680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 >> 172.21.15.37:0/474132488 conn(0x557c7132e4e0 msgr2=0x557c71729fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 shutdown_connections 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 1 -- 172.21.15.37:0/474132488 wait complete. 2024-09-21T06:34:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.001+0000 7ff191a72740 20 client.7614 trim_cache size 0 max 16384 2024-09-21T06:34:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.017+0000 7ff191a72740 1 Processor -- start 2024-09-21T06:34:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.017+0000 7ff191a72740 1 -- start start 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71729b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71749130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff184202450 con 0x557c713196b0 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff184201620 con 0x557c7133a390 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.018+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff184201d20 con 0x557c713114d0 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56682/0 (socket says 172.21.15.37:56682) 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 -- 172.21.15.37:0/3297468750 learned_addr learned my addr 172.21.15.37:0/3297468750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff188a00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71749130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff183e00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71729b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 -- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 msgr2=0x557c71729b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71729b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 -- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 msgr2=0x557c71749130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71749130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 -- 172.21.15.37:0/3297468750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7172fb40 con 0x557c713196b0 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff188a00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71749130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff183e00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71729b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff17be00640 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff16c05fa20 tx=0x7ff16c05faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293234601950f14e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff173400640 1 -- 172.21.15.37:0/3297468750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c067ed0 con 0x557c713196b0 2024-09-21T06:34:13.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff173400640 1 -- 172.21.15.37:0/3297468750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c082070 con 0x557c713196b0 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.019+0000 7ff173400640 1 -- 172.21.15.37:0/3297468750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c06a280 con 0x557c713196b0 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 -- 172.21.15.37:0/3297468750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 msgr2=0x557c7172a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff16c05fa20 tx=0x7ff16c05faf0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 -- 172.21.15.37:0/3297468750 shutdown_connections 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c713196b0 0x557c7172a070 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c7133a390 0x557c71749130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 --2- 172.21.15.37:0/3297468750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c71729b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 -- 172.21.15.37:0/3297468750 >> 172.21.15.37:0/3297468750 conn(0x557c7132e4e0 msgr2=0x557c716e24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 -- 172.21.15.37:0/3297468750 shutdown_connections 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.020+0000 7ff191a72740 1 -- 172.21.15.37:0/3297468750 wait complete. 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.021+0000 7ff191a72740 1 -- start start 2024-09-21T06:34:13.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.021+0000 7ff191a72740 1 Processor -- start 2024-09-21T06:34:13.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c717a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c713196b0 0x557c71729a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c71735ce0 con 0x557c7133a390 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840c98d0 con 0x557c713196b0 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557c7171e6f0 con 0x557c713114d0 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff191a72740 10 client.0 _ll_register_callbacks cb 0x7ff17c001a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff188a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff188a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56686/0 (socket says 172.21.15.37:56686) 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff188a00640 1 -- 172.21.15.37:0/2013746191 learned_addr learned my addr 172.21.15.37:0/2013746191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff17be00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c713196b0 0x557c71729a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.023+0000 7ff183e00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c717a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 -- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 msgr2=0x557c717a44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c717a44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 -- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c713196b0 msgr2=0x557c71729a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c713196b0 0x557c71729a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 -- 172.21.15.37:0/2013746191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c716fabb0 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff188a00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0017a0 tx=0x7ff17c016790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c728f3698a30c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff170c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c004a40 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff17c004be0 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c082070 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7ff17c028050 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff191a72740 10 client.7620 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.024+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x557c717a4ae0 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c082070 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7ff17c02b020 con 0x557c7133a390 2024-09-21T06:34:13.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 10 client.7620 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 10 client.7620 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 20 client.7620 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c716fb290 0x557c716df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 4 client.7620 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c71652270 con 0x557c716fb290 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.025+0000 7ff183e00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c716fb290 0x557c716df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.026+0000 7ff183e00640 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c716fb290 0x557c716df600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0adede57306abcb server_cookie=20f4c5f6c80f439 in_seq=0 out_seq=0 2024-09-21T06:34:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.026+0000 7ff170c00640 10 client.7620 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:13.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.040+0000 7ff170c00640 1 -- 172.21.15.37:0/2013746191 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c71652270 con 0x557c716fb290 2024-09-21T06:34:13.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.040+0000 7ff170c00640 10 client.7620 handle_command_reply: tid=0 2024-09-21T06:34:13.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.040+0000 7ff191a72740 1 client.7620 shutdown 2024-09-21T06:34:13.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 msgr2=0x557c71729f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0017a0 tx=0x7ff17c016790 comp rx=0 tx=0).stop 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 shutdown_connections 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x557c716fb290 0x557c716df600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c7133a390 0x557c71729f40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c713196b0 0x557c71729a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 --2- 172.21.15.37:0/2013746191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557c713114d0 0x557c717a44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff170c00640 0 client.7620 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 >> 172.21.15.37:0/2013746191 conn(0x557c7132e4e0 msgr2=0x557c71707cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 shutdown_connections 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 1 -- 172.21.15.37:0/2013746191 wait complete. 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.041+0000 7ff191a72740 20 client.7620 trim_cache size 0 max 16384 2024-09-21T06:34:13.044 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:34:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:34:13.045 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:13.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 msgr2=0x7ff16007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 0x7ff16007b0a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff174004730 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 msgr2=0x7ff18415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e7c0 tx=0x7ff16c00ec80 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 shutdown_connections 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff160078b90 0x7ff16007b0a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1841429b0 0x7ff18415bdf0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1841456e0 0x7ff1840cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 --2- 172.21.15.37:0/1382922971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1840cc8d0 0x7ff1840a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 >> 172.21.15.37:0/1382922971 conn(0x7ff18406f920 msgr2=0x7ff184143030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 shutdown_connections 2024-09-21T06:34:13.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.042+0000 7ff189400640 1 -- 172.21.15.37:0/1382922971 wait complete. 2024-09-21T06:34:13.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2024-09-21T06:34:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 -- 172.21.15.37:0/957807666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380b9c20 msgr2=0x7f5938162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 --2- 172.21.15.37:0/957807666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380b9c20 0x7f5938162280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5924009830 tx=0x7f592402a510 comp rx=0 tx=0).stop 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 -- 172.21.15.37:0/957807666 shutdown_connections 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 --2- 172.21.15.37:0/957807666 >> v2:172.21.15.150:3300/0 conn(0x7f59380ba030 0x7f59381627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 --2- 172.21.15.37:0/957807666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380b9c20 0x7f5938162280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 -- 172.21.15.37:0/957807666 >> 172.21.15.37:0/957807666 conn(0x7f59380d4360 msgr2=0x7f59380d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 -- 172.21.15.37:0/957807666 shutdown_connections 2024-09-21T06:34:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.294+0000 7f593f000640 1 -- 172.21.15.37:0/957807666 wait complete. 2024-09-21T06:34:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.296+0000 7f593f000640 1 Processor -- start 2024-09-21T06:34:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.297+0000 7f593f000640 1 -- start start 2024-09-21T06:34:13.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59380b9c20 0x7f5938151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380d2bd0 0x7f59380b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5938155660 con 0x7f59380ba030 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59381554e0 con 0x7f59380b9c20 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593f000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f59381557e0 con 0x7f59380d2bd0 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56698/0 (socket says 172.21.15.37:56698) 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593d200640 1 -- 172.21.15.37:0/3725289148 learned_addr learned my addr 172.21.15.37:0/3725289148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.298+0000 7f593e600640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380d2bd0 0x7f59380b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593dc00640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59380b9c20 0x7f5938151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 -- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380d2bd0 msgr2=0x7f59380b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380d2bd0 0x7f59380b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 -- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59380b9c20 msgr2=0x7f5938151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59380b9c20 0x7f5938151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 -- 172.21.15.37:0/3725289148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5938155f80 con 0x7f59380ba030 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f593d200640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5924009950 tx=0x7f5924033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.299+0000 7f5935600640 1 -- 172.21.15.37:0/3725289148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924005c90 con 0x7f59380ba030 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.300+0000 7f5935600640 1 -- 172.21.15.37:0/3725289148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5924031e50 con 0x7f59380ba030 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.300+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59380c85c0 con 0x7f59380ba030 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.300+0000 7f5935600640 1 -- 172.21.15.37:0/3725289148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59240332f0 con 0x7f59380ba030 2024-09-21T06:34:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.300+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5938157070 con 0x7f59380ba030 2024-09-21T06:34:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.302+0000 7f5935600640 1 -- 172.21.15.37:0/3725289148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5924033490 con 0x7f59380ba030 2024-09-21T06:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.304+0000 7f5935600640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 0x7f591807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.304+0000 7f5935600640 1 -- 172.21.15.37:0/3725289148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f59240bc100 con 0x7f59380ba030 2024-09-21T06:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.304+0000 7f593dc00640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 0x7f591807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.304+0000 7f594765e740 1 Processor -- start 2024-09-21T06:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.304+0000 7f594765e740 1 -- start start 2024-09-21T06:34:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.305+0000 7f593dc00640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 0x7f591807b0d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f592c007fd0 tx=0x7f592c001370 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 0x557492e3b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e3b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557492d711c0 con 0x557492a6a390 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557492d84870 con 0x557492a414d0 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557492d73ca0 con 0x557492a496b0 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56702/0 (socket says 172.21.15.37:56702) 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 -- 172.21.15.37:0/554500571 learned_addr learned my addr 172.21.15.37:0/554500571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593d200640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 0x557492e3b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593e600640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e3b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 -- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 msgr2=0x557492e3b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e3b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 -- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 msgr2=0x557492e3b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 0x557492e3b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593dc00640 1 -- 172.21.15.37:0/554500571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557492b4c7d0 con 0x557492a6a390 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593e600640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e3b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.306+0000 7f593d200640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 0x557492e3b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.307+0000 7f593dc00640 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f592c00f9f0 tx=0x7f592c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad7e8679e5777c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.307+0000 7f592a000640 1 -- 172.21.15.37:0/554500571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c01b050 con 0x557492a6a390 2024-09-21T06:34:13.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.307+0000 7f592a000640 1 -- 172.21.15.37:0/554500571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c025070 con 0x557492a6a390 2024-09-21T06:34:13.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.307+0000 7f592a000640 1 -- 172.21.15.37:0/554500571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c014de0 con 0x557492a6a390 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 -- 172.21.15.37:0/554500571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 msgr2=0x557492e58730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f592c00f9f0 tx=0x7f592c012000 comp rx=0 tx=0).stop 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 -- 172.21.15.37:0/554500571 shutdown_connections 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e3b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a414d0 0x557492e3b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 --2- 172.21.15.37:0/554500571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e58730 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 -- 172.21.15.37:0/554500571 >> 172.21.15.37:0/554500571 conn(0x557492a5e4e0 msgr2=0x557492e11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 -- 172.21.15.37:0/554500571 shutdown_connections 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.308+0000 7f594765e740 1 -- 172.21.15.37:0/554500571 wait complete. 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.309+0000 7f594765e740 1 -- start start 2024-09-21T06:34:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.309+0000 7f594765e740 1 Processor -- start 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492e55eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e4fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557492d1a410 con 0x557492a6a390 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557492d77ab0 con 0x557492a496b0 2024-09-21T06:34:13.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557492ae4740 con 0x557492a414d0 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f594765e740 10 client.0 _ll_register_callbacks cb 0x557492cf7360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593dc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492e55eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e4fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56718/0 (socket says 172.21.15.37:56718) 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593e600640 1 -- 172.21.15.37:0/1830041718 learned_addr learned my addr 172.21.15.37:0/1830041718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.311+0000 7f593dc00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492e55eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:46046/0 (socket says 172.21.15.37:46046) 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 -- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 msgr2=0x557492e55eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492e55eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 -- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 msgr2=0x557492e4fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e4fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 -- 172.21.15.37:0/1830041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557492ad5970 con 0x557492a6a390 2024-09-21T06:34:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593d200640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e4fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f593e600640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5930012a10 tx=0x7f5930012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1dcdad0cddb9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f591ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930010e00 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f593001dc40 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c025070 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f593001de40 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f594765e740 10 client.7632 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.312+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x557492ab36a0 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c025070 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f5930024050 con 0x557492a6a390 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 10 client.7632 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 10 client.7632 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 20 client.7632 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492a28f60 0x557492e54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 4 client.7632 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557492d867c0 con 0x557492a28f60 2024-09-21T06:34:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.313+0000 7f593dc00640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492a28f60 0x557492e54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.314+0000 7f591ea00640 10 client.7632 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.314+0000 7f593dc00640 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492a28f60 0x557492e54d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2173a64dd9886083 server_cookie=4fa761e12d79c657 in_seq=0 out_seq=0 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.314+0000 7f591ea00640 1 -- 172.21.15.37:0/1830041718 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557492d867c0 con 0x557492a28f60 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.314+0000 7f591ea00640 10 client.7632 handle_command_reply: tid=0 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.314+0000 7f594765e740 1 client.7632 shutdown 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 msgr2=0x557492e54330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5930012a10 tx=0x7f5930012ed0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 shutdown_connections 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492a28f60 0x557492e54d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e54330 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e4fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 --2- 172.21.15.37:0/1830041718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492e55eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 >> 172.21.15.37:0/1830041718 conn(0x557492a5e4e0 msgr2=0x557492e5a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f591ea00640 0 client.7632 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 shutdown_connections 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 1 -- 172.21.15.37:0/1830041718 wait complete. 2024-09-21T06:34:13.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.315+0000 7f594765e740 20 client.7632 trim_cache size 0 max 16384 2024-09-21T06:34:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.331+0000 7f594765e740 1 Processor -- start 2024-09-21T06:34:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.331+0000 7f594765e740 1 -- start start 2024-09-21T06:34:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e55280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 0x557492e37ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59382024e0 con 0x557492a414d0 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59382016b0 con 0x557492a6a390 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5938201db0 con 0x557492a496b0 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f593dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f593dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56720/0 (socket says 172.21.15.37:56720) 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.332+0000 7f593dc00640 1 -- 172.21.15.37:0/1343599570 learned_addr learned my addr 172.21.15.37:0/1343599570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593e600640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 0x557492e37ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593d200640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e55280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 -- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 msgr2=0x557492e55280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e55280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 -- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 msgr2=0x557492e37ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 0x557492e37ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 -- 172.21.15.37:0/1343599570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557492e11460 con 0x557492a414d0 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593e600640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 0x557492e37ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f593dc00640 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f592c014d70 tx=0x7f592c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e122dd476b530497 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f5929600640 1 -- 172.21.15.37:0/1343599570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c01b050 con 0x557492a414d0 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f5929600640 1 -- 172.21.15.37:0/1343599570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c025040 con 0x557492a414d0 2024-09-21T06:34:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.333+0000 7f5929600640 1 -- 172.21.15.37:0/1343599570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c005280 con 0x557492a414d0 2024-09-21T06:34:13.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 -- 172.21.15.37:0/1343599570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 msgr2=0x557492e54d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f592c014d70 tx=0x7f592c004620 comp rx=0 tx=0).stop 2024-09-21T06:34:13.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 -- 172.21.15.37:0/1343599570 shutdown_connections 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a6a390 0x557492e37ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a496b0 0x557492e55280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 --2- 172.21.15.37:0/1343599570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a414d0 0x557492e54d40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 -- 172.21.15.37:0/1343599570 >> 172.21.15.37:0/1343599570 conn(0x557492a5e4e0 msgr2=0x557492e2a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 -- 172.21.15.37:0/1343599570 shutdown_connections 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.334+0000 7f594765e740 1 -- 172.21.15.37:0/1343599570 wait complete. 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.335+0000 7f594765e740 1 -- start start 2024-09-21T06:34:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.335+0000 7f594765e740 1 Processor -- start 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492ecb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e3b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557492e56e30 con 0x557492a6a390 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59381f93a0 con 0x557492a496b0 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557492e12ec0 con 0x557492a414d0 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f594765e740 10 client.0 _ll_register_callbacks cb 0x7f5930009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56726/0 (socket says 172.21.15.37:56726) 2024-09-21T06:34:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593e600640 1 -- 172.21.15.37:0/3359332819 learned_addr learned my addr 172.21.15.37:0/3359332819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593d200640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e3b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593dc00640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492ecb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593e600640 1 -- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 msgr2=0x557492ecb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.337+0000 7f593e600640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492ecb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f593e600640 1 -- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 msgr2=0x557492e3b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f593e600640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e3b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f593e600640 1 -- 172.21.15.37:0/3359332819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557492e4e450 con 0x557492a6a390 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f593e600640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5930004f20 tx=0x7f59300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463436c2831eaa55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f591ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930026880 con 0x557492a6a390 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f5930026a20 con 0x557492a6a390 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c025040 con 0x557492a6a390 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f5930035050 con 0x557492a6a390 2024-09-21T06:34:13.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f594765e740 10 client.7638 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.338+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x557492ed8460 con 0x557492a6a390 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c025040 con 0x557492a6a390 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f5930038020 con 0x557492a6a390 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 10 client.7638 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 10 client.7638 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 20 client.7638 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492ed7f30 0x557492e78210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 4 client.7638 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557492d82270 con 0x557492ed7f30 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.339+0000 7f593dc00640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492ed7f30 0x557492e78210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.340+0000 7f593dc00640 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492ed7f30 0x557492e78210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9e2a7f6177b3127 server_cookie=ccdbd6863655f75b in_seq=0 out_seq=0 2024-09-21T06:34:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.340+0000 7f591ea00640 10 client.7638 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.353+0000 7f591ea00640 1 -- 172.21.15.37:0/3359332819 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557492d82270 con 0x557492ed7f30 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.353+0000 7f591ea00640 10 client.7638 handle_command_reply: tid=0 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.353+0000 7f594765e740 1 client.7638 shutdown 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.353+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 msgr2=0x557492e3ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.353+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5930004f20 tx=0x7f59300039c0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 shutdown_connections 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x557492ed7f30 0x557492e78210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557492a6a390 0x557492e3ba20 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557492a496b0 0x557492e3b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 --2- 172.21.15.37:0/3359332819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557492a414d0 0x557492ecb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f591ea00640 0 client.7638 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 >> 172.21.15.37:0/3359332819 conn(0x557492a5e4e0 msgr2=0x557492e4f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 shutdown_connections 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 1 -- 172.21.15.37:0/3359332819 wait complete. 2024-09-21T06:34:13.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.354+0000 7f594765e740 20 client.7638 trim_cache size 0 max 16384 2024-09-21T06:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:13.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 msgr2=0x7f591807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 0x7f591807b0d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f592c007fd0 tx=0x7f592c001370 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 msgr2=0x7f59380c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5924009950 tx=0x7f5924033c20 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 shutdown_connections 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5918078bc0 0x7f591807b0d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59380ba030 0x7f59380c79e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59380d2bd0 0x7f59380b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 --2- 172.21.15.37:0/3725289148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59380b9c20 0x7f5938151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 >> 172.21.15.37:0/3725289148 conn(0x7f59380d4360 msgr2=0x7f59381545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 shutdown_connections 2024-09-21T06:34:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.355+0000 7f593f000640 1 -- 172.21.15.37:0/3725289148 wait complete. 2024-09-21T06:34:13.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 -- 172.21.15.37:0/1094741130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb8154f20 msgr2=0x7f0eb8155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1094741130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb8154f20 0x7f0eb8155330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009830 tx=0x7f0ea402a510 comp rx=0 tx=0).stop 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 -- 172.21.15.37:0/1094741130 shutdown_connections 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1094741130 >> v2:172.21.15.150:3300/0 conn(0x7f0eb80cecc0 0x7f0eb80cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1094741130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb8154f20 0x7f0eb8155330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 -- 172.21.15.37:0/1094741130 >> 172.21.15.37:0/1094741130 conn(0x7f0eb80d4360 msgr2=0x7f0eb80d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 -- 172.21.15.37:0/1094741130 shutdown_connections 2024-09-21T06:34:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.608+0000 7f0ebe600640 1 -- 172.21.15.37:0/1094741130 wait complete. 2024-09-21T06:34:13.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.610+0000 7f0ebe600640 1 Processor -- start 2024-09-21T06:34:13.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.611+0000 7f0ebe600640 1 -- start start 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb80cecc0 0x7f0eb81520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb80d2760 0x7f0eb81f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80cf790 con 0x7f0eb8154f20 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80cf610 con 0x7f0eb80cecc0 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebe600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb80cf910 con 0x7f0eb80d2760 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56736/0 (socket says 172.21.15.37:56736) 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebdc00640 1 -- 172.21.15.37:0/1546559361 learned_addr learned my addr 172.21.15.37:0/1546559361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb80d2760 0x7f0eb81f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.612+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb80cecc0 0x7f0eb81520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 -- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb80d2760 msgr2=0x7f0eb81f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb80d2760 0x7f0eb81f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 -- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb80cecc0 msgr2=0x7f0eb81520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb80cecc0 0x7f0eb81520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 -- 172.21.15.37:0/1546559361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8154ae0 con 0x7f0eb8154f20 2024-09-21T06:34:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000ea80 tx=0x7f0eb000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0eb4c00640 1 -- 172.21.15.37:0/1546559361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004b20 con 0x7f0eb8154f20 2024-09-21T06:34:13.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0eb4c00640 1 -- 172.21.15.37:0/1546559361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0004ce0 con 0x7f0eb8154f20 2024-09-21T06:34:13.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81f9560 con 0x7f0eb8154f20 2024-09-21T06:34:13.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.613+0000 7f0eb4c00640 1 -- 172.21.15.37:0/1546559361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0005280 con 0x7f0eb8154f20 2024-09-21T06:34:13.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.614+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb80c80c0 con 0x7f0eb8154f20 2024-09-21T06:34:13.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.616+0000 7f0eb4c00640 1 -- 172.21.15.37:0/1546559361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0eb0005420 con 0x7f0eb8154f20 2024-09-21T06:34:13.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.618+0000 7f0eb4c00640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 0x7f0e9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.618+0000 7f0eb4c00640 1 -- 172.21.15.37:0/1546559361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0eb009bb20 con 0x7f0eb8154f20 2024-09-21T06:34:13.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.618+0000 7f0ec6c72740 1 Processor -- start 2024-09-21T06:34:13.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.618+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 0x7f0e9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.618+0000 7f0ec6c72740 1 -- start start 2024-09-21T06:34:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.619+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 0x7f0e9807b0d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007fd0 tx=0x7f0eac001370 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae275730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade5e4d0 0x558cae258400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cae18e1c0 con 0x558cade666b0 2024-09-21T06:34:13.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558cae1a1870 con 0x558cade5e4d0 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558cae190ca0 con 0x558cade87390 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56740/0 (socket says 172.21.15.37:56740) 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0ebdc00640 1 -- 172.21.15.37:0/2655132057 learned_addr learned my addr 172.21.15.37:0/2655132057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.620+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade5e4d0 0x558cae258400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebd200640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae275730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 -- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 msgr2=0x558cae275730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae275730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 -- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade5e4d0 msgr2=0x558cae258400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade5e4d0 0x558cae258400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 -- 172.21.15.37:0/2655132057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cadf697d0 con 0x558cade666b0 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebd200640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae275730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000f010 tx=0x7f0eb0063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a069c311fd67094a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ea9600640 1 -- 172.21.15.37:0/2655132057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00636f0 con 0x558cade666b0 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ea9600640 1 -- 172.21.15.37:0/2655132057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0068120 con 0x558cade666b0 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.621+0000 7f0ea9600640 1 -- 172.21.15.37:0/2655132057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb006cda0 con 0x558cade666b0 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2655132057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 msgr2=0x558cae258940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000f010 tx=0x7f0eb0063040 comp rx=0 tx=0).stop 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2655132057 shutdown_connections 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae258940 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade5e4d0 0x558cae258400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2655132057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae275730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2655132057 >> 172.21.15.37:0/2655132057 conn(0x558cade7b4e0 msgr2=0x558cae22e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2655132057 shutdown_connections 2024-09-21T06:34:13.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.622+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2655132057 wait complete. 2024-09-21T06:34:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.623+0000 7f0ec6c72740 1 -- start start 2024-09-21T06:34:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.623+0000 7f0ec6c72740 1 Processor -- start 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae245fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade87390 0x558cae22def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cae137410 con 0x558cade666b0 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558cae194ab0 con 0x558cade87390 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558cadf01740 con 0x558cade5e4d0 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ec6c72740 10 client.0 _ll_register_callbacks cb 0x558cae114360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56742/0 (socket says 172.21.15.37:56742) 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0eb7e00640 1 -- 172.21.15.37:0/1763882552 learned_addr learned my addr 172.21.15.37:0/1763882552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae245fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.625+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade87390 0x558cae22def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 -- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 msgr2=0x558cae245fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae245fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 -- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade87390 msgr2=0x558cae22def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade87390 0x558cae22def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 -- 172.21.15.37:0/1763882552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cadef2970 con 0x558cade666b0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009780 tx=0x7f0ea40044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd746b0a96a2426 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0e9e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003b40 con 0x558cade666b0 2024-09-21T06:34:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0ea4003ce0 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.626+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0068120 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f0ea404c050 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 10 client.7650 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x558caded06a0 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0068120 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f0ea404f020 con 0x558cade666b0 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 10 client.7650 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 10 client.7650 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 20 client.7650 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cade45f60 0x558cae26d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 4 client.7650 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cae1a37c0 con 0x558cade45f60 2024-09-21T06:34:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.627+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cade45f60 0x558cae26d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.628+0000 7f0ebd200640 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cade45f60 0x558cae26d7d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f948f9f867063d2c server_cookie=f5cda028c669151f in_seq=0 out_seq=0 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.628+0000 7f0e9e000640 10 client.7650 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0e9e000640 1 -- 172.21.15.37:0/1763882552 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558cae1a37c0 con 0x558cade45f60 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0e9e000640 10 client.7650 handle_command_reply: tid=0 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 client.7650 shutdown 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 msgr2=0x558cae22d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009780 tx=0x7f0ea40044d0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 shutdown_connections 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cade45f60 0x558cae26d7d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade666b0 0x558cae22d9b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade87390 0x558cae22def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/1763882552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae245fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0e9e000640 0 client.7650 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 >> 172.21.15.37:0/1763882552 conn(0x558cade7b4e0 msgr2=0x558cae26cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 shutdown_connections 2024-09-21T06:34:13.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.629+0000 7f0ec6c72740 1 -- 172.21.15.37:0/1763882552 wait complete. 2024-09-21T06:34:13.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.630+0000 7f0ec6c72740 20 client.7650 trim_cache size 0 max 16384 2024-09-21T06:34:13.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.645+0000 7f0ec6c72740 1 Processor -- start 2024-09-21T06:34:13.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.645+0000 7f0ec6c72740 1 -- start start 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae26cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae276790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8202440 con 0x558cade87390 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8201610 con 0x558cade666b0 2024-09-21T06:34:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb8201d10 con 0x558cade5e4d0 2024-09-21T06:34:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebdc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56750/0 (socket says 172.21.15.37:56750) 2024-09-21T06:34:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebdc00640 1 -- 172.21.15.37:0/3477197499 learned_addr learned my addr 172.21.15.37:0/3477197499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae276790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebd200640 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae26cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebdc00640 1 -- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 msgr2=0x558cae26cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.647+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae26cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ebdc00640 1 -- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 msgr2=0x558cae276790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae276790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ebdc00640 1 -- 172.21.15.37:0/3477197499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cae240960 con 0x558cade87390 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ebdc00640 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0004ff0 tx=0x7f0eb0099cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ece9a0d00b01e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ea8c00640 1 -- 172.21.15.37:0/3477197499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0083070 con 0x558cade87390 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ea8c00640 1 -- 172.21.15.37:0/3477197499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0068120 con 0x558cade87390 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.648+0000 7f0ea8c00640 1 -- 172.21.15.37:0/3477197499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb006cd80 con 0x558cade87390 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 -- 172.21.15.37:0/3477197499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 msgr2=0x558cae276cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0004ff0 tx=0x7f0eb0099cf0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 -- 172.21.15.37:0/3477197499 shutdown_connections 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade87390 0x558cae276cd0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae276790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/3477197499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade5e4d0 0x558cae26cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 -- 172.21.15.37:0/3477197499 >> 172.21.15.37:0/3477197499 conn(0x558cade7b4e0 msgr2=0x558cae2721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 -- 172.21.15.37:0/3477197499 shutdown_connections 2024-09-21T06:34:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.649+0000 7f0ec6c72740 1 -- 172.21.15.37:0/3477197499 wait complete. 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.650+0000 7f0ec6c72740 1 -- start start 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.650+0000 7f0ec6c72740 1 Processor -- start 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae282160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae2954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cae1a1510 con 0x558cade5e4d0 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8156880 con 0x558cade666b0 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558cae273f70 con 0x558cade87390 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ec6c72740 10 client.0 _ll_register_callbacks cb 0x7f0eb00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56756/0 (socket says 172.21.15.37:56756) 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 -- 172.21.15.37:0/2798600679 learned_addr learned my addr 172.21.15.37:0/2798600679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 -- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 msgr2=0x558cae2954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae2954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 -- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 msgr2=0x558cae282160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae282160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.652+0000 7f0ebd200640 1 -- 172.21.15.37:0/2798600679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cae2770f0 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0ebd200640 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00fdd0 tx=0x7f0eac007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6fb4064b6a0f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac02b460 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0eac010040 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0068120 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f0eac038050 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0ec6c72740 10 client.7656 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x558cae2826a0 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0068120 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.653+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f0eac03b020 con 0x558cade5e4d0 2024-09-21T06:34:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 10 client.7656 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 10 client.7656 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 20 client.7656 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cae240140 0x558cae295a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 4 client.7656 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cae19f270 con 0x558cae240140 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cae240140 0x558cae295a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0e9e000640 10 client.7656 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.654+0000 7f0eb7e00640 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cae240140 0x558cae295a10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51b8ea2cbf4a126f server_cookie=ede4756fac25c6e2 in_seq=0 out_seq=0 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0e9e000640 1 -- 172.21.15.37:0/2798600679 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cae19f270 con 0x558cae240140 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0e9e000640 10 client.7656 handle_command_reply: tid=0 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 client.7656 shutdown 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 msgr2=0x558cae28c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00fdd0 tx=0x7f0eac007ce0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 shutdown_connections 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x558cae240140 0x558cae295a10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558cade87390 0x558cae2954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cade666b0 0x558cae282160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 --2- 172.21.15.37:0/2798600679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cade5e4d0 0x558cae28c990 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0e9e000640 0 client.7656 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.664+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 >> 172.21.15.37:0/2798600679 conn(0x558cade7b4e0 msgr2=0x558cae276790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 shutdown_connections 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ec6c72740 1 -- 172.21.15.37:0/2798600679 wait complete. 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ec6c72740 20 client.7656 trim_cache size 0 max 16384 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 msgr2=0x7f0e9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 0x7f0e9807b0d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007fd0 tx=0x7f0eac001370 comp rx=0 tx=0).stop 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 msgr2=0x7f0eb8157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000ea80 tx=0x7f0eb000ef40 comp rx=0 tx=0).stop 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 shutdown_connections 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e98078bc0 0x7f0e9807b0d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eb8154f20 0x7f0eb8157220 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb80d2760 0x7f0eb81f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 --2- 172.21.15.37:0/1546559361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb80cecc0 0x7f0eb81520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.665+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 >> 172.21.15.37:0/1546559361 conn(0x7f0eb80d4360 msgr2=0x7f0eb8162350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.666+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 shutdown_connections 2024-09-21T06:34:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13.666+0000 7f0ebe600640 1 -- 172.21.15.37:0/1546559361 wait complete. 2024-09-21T06:34:13.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:13.694 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/ 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-21T06:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-21T06:34:13.738 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:13.739 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:13.739 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:13.739 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:13.739 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-21T06:34:13.739 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-09-21T06:34:13.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-21T06:34:13.890 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-09-21T06:34:14.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:14.041 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:14.041 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:14.042 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:14.042 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:14.042 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-21T06:34:14.042 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-09-21T06:34:14.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-21T06:34:14.194 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:14.334 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627781 2024-09-21T06:34:14.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:14.345 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:14.346 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:14.346 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:14.346 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:14.346 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-21T06:34:14.346 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627782 2024-09-21T06:34:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-21T06:34:14.498 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627783 2024-09-21T06:34:14.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:14.649 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-21T06:34:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.833+0000 7fe7db9f8f40 1 Processor -- start 2024-09-21T06:34:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.833+0000 7fe7db9f8f40 1 -- start start 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c8714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c880bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e42c7561d0 con 0x55e42c87af90 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e42c762f90 con 0x55e42c87ba80 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55e42c6f7c60 con 0x55e42c871bd0 2024-09-21T06:34:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57070/0 (socket says 172.21.15.37:57070) 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 -- 172.21.15.37:0/3825217379 learned_addr learned my addr 172.21.15.37:0/3825217379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d4c00640 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c8714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7cfe00640 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c880bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 -- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 msgr2=0x55e42c8714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c8714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 -- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 msgr2=0x55e42c880bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c880bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.835+0000 7fe7d5600640 1 -- 172.21.15.37:0/3825217379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42c7f2270 con 0x55e42c87af90 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.836+0000 7fe7d5600640 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x55e42c796a40 tx=0x7fe7d0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b079cfeb58c4947 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.836+0000 7fe7cea00640 1 -- 172.21.15.37:0/3825217379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0005820 con 0x55e42c87af90 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.836+0000 7fe7cea00640 1 -- 172.21.15.37:0/3825217379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7d00059c0 con 0x55e42c87af90 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.836+0000 7fe7cea00640 1 -- 172.21.15.37:0/3825217379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0005dd0 con 0x55e42c87af90 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/3825217379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 msgr2=0x55e42c87e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x55e42c796a40 tx=0x7fe7d0009620 comp rx=0 tx=0).stop 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/3825217379 shutdown_connections 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c87e680 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c880bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/3825217379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c8714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/3825217379 >> 172.21.15.37:0/3825217379 conn(0x55e42c6c8ac0 msgr2=0x55e42c7f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/3825217379 shutdown_connections 2024-09-21T06:34:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.837+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/3825217379 wait complete. 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.838+0000 7fe7db9f8f40 1 Processor -- start 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.838+0000 7fe7db9f8f40 1 -- start start 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c85d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c85ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c85f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e42c880770 con 0x55e42c87af90 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e42c87c7a0 con 0x55e42c87ba80 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.839+0000 7fe7db9f8f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55e42c87c2b0 con 0x55e42c871bd0 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c85ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c85ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57084/0 (socket says 172.21.15.37:57084) 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 -- 172.21.15.37:0/172857296 learned_addr learned my addr 172.21.15.37:0/172857296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7d4c00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c85d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7d5600640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c85f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 -- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 msgr2=0x55e42c85d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e42c871bd0 0x55e42c85d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 -- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 msgr2=0x55e42c85f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e42c87ba80 0x55e42c85f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 -- 172.21.15.37:0/172857296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42c85ffb0 con 0x55e42c87af90 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.840+0000 7fe7cfe00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c85ccd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001d80 tx=0x7fe7c000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.841+0000 7fe7cd600640 1 -- 172.21.15.37:0/172857296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0010850 con 0x55e42c87af90 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.841+0000 7fe7cd600640 1 -- 172.21.15.37:0/172857296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7c0014030 con 0x55e42c87af90 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.841+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e42c860290 con 0x55e42c87af90 2024-09-21T06:34:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.841+0000 7fe7cd600640 1 -- 172.21.15.37:0/172857296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0010c50 con 0x55e42c87af90 2024-09-21T06:34:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.841+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42c87e060 con 0x55e42c87af90 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.843+0000 7fe7cd600640 1 -- 172.21.15.37:0/172857296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe7c00183d0 con 0x55e42c87af90 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.844+0000 7fe7cd600640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe7b4077570 0x7fe7b4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.844+0000 7fe7cd600640 1 -- 172.21.15.37:0/172857296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe7c0099ee0 con 0x55e42c87af90 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.844+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55e42c86b320 0x55e42c86b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.844+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 13.12 13:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55e42c9234d0 con 0x55e42c86b320 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.845+0000 7fe7d4c00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe7b4077570 0x7fe7b4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.845+0000 7fe7d5600640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55e42c86b320 0x55e42c86b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.845+0000 7fe7d4c00640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe7b4077570 0x7fe7b4079a80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8055000 tx=0x7fe7c8002fb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.846+0000 7fe7d5600640 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55e42c86b320 0x55e42c86b730 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:14.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7d5600640 1 -- 172.21.15.37:0/172857296 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v91'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe7d0003040 con 0x55e42c86b320 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55e42c86b320 msgr2=0x55e42c86b730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55e42c86b320 0x55e42c86b730 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe7b4077570 msgr2=0x7fe7b4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe7b4077570 0x7fe7b4079a80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8055000 tx=0x7fe7c8002fb0 comp rx=0 tx=0).stop 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 msgr2=0x55e42c85ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 --2- 172.21.15.37:0/172857296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e42c87af90 0x55e42c85ccd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001d80 tx=0x7fe7c000d720 comp rx=0 tx=0).stop 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7d5600640 1 -- 172.21.15.37:0/172857296 reap_dead start 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 shutdown_connections 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 >> 172.21.15.37:0/172857296 conn(0x55e42c6c8ac0 msgr2=0x55e42c7aa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 shutdown_connections 2024-09-21T06:34:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:14.848+0000 7fe7db9f8f40 1 -- 172.21.15.37:0/172857296 wait complete. 2024-09-21T06:34:14.861 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-21T06:34:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.003+0000 7f031665bf40 1 Processor -- start 2024-09-21T06:34:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.003+0000 7f031665bf40 1 -- start start 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1c329b0 0x5619b1d016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619b1b7ec60 con 0x5619b1ced5c0 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5619b1bdd1d0 con 0x5619b1c7a630 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f031665bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5619b1c7a110 con 0x5619b1c329b0 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.004+0000 7f0310200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57104/0 (socket says 172.21.15.37:57104) 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 -- 172.21.15.37:0/3938548980 learned_addr learned my addr 172.21.15.37:0/3938548980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f030ee00640 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f030f800640 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1c329b0 0x5619b1d016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 -- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1c329b0 msgr2=0x5619b1d016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1c329b0 0x5619b1d016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 -- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 msgr2=0x5619b1c5f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c5f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 -- 172.21.15.37:0/3938548980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b1c2e410 con 0x5619b1ced5c0 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f0310200640 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f030800b7b0 tx=0x7f030800bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4288576c802715b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f030da00640 1 -- 172.21.15.37:0/3938548980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030800db30 con 0x5619b1ced5c0 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f030da00640 1 -- 172.21.15.37:0/3938548980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f030800ddc0 con 0x5619b1ced5c0 2024-09-21T06:34:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.005+0000 7f030da00640 1 -- 172.21.15.37:0/3938548980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030800f690 con 0x5619b1ced5c0 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 -- 172.21.15.37:0/3938548980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 msgr2=0x5619b1c5f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f030800b7b0 tx=0x7f030800bc70 comp rx=0 tx=0).stop 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 -- 172.21.15.37:0/3938548980 shutdown_connections 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1ced5c0 0x5619b1c5f720 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c5f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 --2- 172.21.15.37:0/3938548980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1c329b0 0x5619b1d016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 -- 172.21.15.37:0/3938548980 >> 172.21.15.37:0/3938548980 conn(0x5619b1b4fac0 msgr2=0x5619b1d06c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 -- 172.21.15.37:0/3938548980 shutdown_connections 2024-09-21T06:34:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.006+0000 7f031665bf40 1 -- 172.21.15.37:0/3938548980 wait complete. 2024-09-21T06:34:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.007+0000 7f031665bf40 1 Processor -- start 2024-09-21T06:34:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.007+0000 7f031665bf40 1 -- start start 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 0x5619b1c72220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c72760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1ced5c0 0x5619b1c6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619b1c5a8d0 con 0x5619b1c329b0 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5619b1d055b0 con 0x5619b1c7a630 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f031665bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5619b1d05890 con 0x5619b1ced5c0 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f030f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 0x5619b1c72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f030f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 0x5619b1c72220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57116/0 (socket says 172.21.15.37:57116) 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f030f800640 1 -- 172.21.15.37:0/4180196356 learned_addr learned my addr 172.21.15.37:0/4180196356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f030ee00640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.009+0000 7f0310200640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1ced5c0 0x5619b1c6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 -- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1ced5c0 msgr2=0x5619b1c6fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619b1ced5c0 0x5619b1c6fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 -- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 msgr2=0x5619b1c72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5619b1c7a630 0x5619b1c72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 -- 172.21.15.37:0/4180196356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b1c2f360 con 0x5619b1c329b0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f030f800640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 0x5619b1c72220 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f030000df70 tx=0x7f030000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f0307e00640 1 -- 172.21.15.37:0/4180196356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000be60 con 0x5619b1c329b0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f0307e00640 1 -- 172.21.15.37:0/4180196356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0300016070 con 0x5619b1c329b0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5619b1c2f640 con 0x5619b1c329b0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f0307e00640 1 -- 172.21.15.37:0/4180196356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000116c0 con 0x5619b1c329b0 2024-09-21T06:34:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.010+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b1c2fc00 con 0x5619b1c329b0 2024-09-21T06:34:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.012+0000 7f0307e00640 1 -- 172.21.15.37:0/4180196356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0300011860 con 0x5619b1c329b0 2024-09-21T06:34:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.013+0000 7f0307e00640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02f0077570 0x7f02f0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.013+0000 7f0307e00640 1 -- 172.21.15.37:0/4180196356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0300099a00 con 0x5619b1c329b0 2024-09-21T06:34:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.014+0000 7f031665bf40 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x5619b1c707c0 0x5619b1dae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.014+0000 7f030ee00640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02f0077570 0x7f02f0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.014+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 13.12 13:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x5619b1d06f80 con 0x5619b1c707c0 2024-09-21T06:34:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.014+0000 7f0310200640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x5619b1c707c0 0x5619b1dae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.015+0000 7f030ee00640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02f0077570 0x7f02f0079a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f02fc055000 tx=0x7f02fc003020 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.015+0000 7f0310200640 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x5619b1c707c0 0x5619b1dae550 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.016+0000 7f0310200640 1 -- 172.21.15.37:0/4180196356 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v91'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030800b040 con 0x5619b1c707c0 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x5619b1c707c0 msgr2=0x5619b1dae550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x5619b1c707c0 0x5619b1dae550 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02f0077570 msgr2=0x7f02f0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f02f0077570 0x7f02f0079a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f02fc055000 tx=0x7f02fc003020 comp rx=0 tx=0).stop 2024-09-21T06:34:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 msgr2=0x5619b1c72220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 --2- 172.21.15.37:0/4180196356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b1c329b0 0x5619b1c72220 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f030000df70 tx=0x7f030000b5b0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f0310200640 1 -- 172.21.15.37:0/4180196356 reap_dead start 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 shutdown_connections 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 >> 172.21.15.37:0/4180196356 conn(0x5619b1b4fac0 msgr2=0x5619b1c5f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 shutdown_connections 2024-09-21T06:34:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.017+0000 7f031665bf40 1 -- 172.21.15.37:0/4180196356 wait complete. 2024-09-21T06:34:15.030 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-21T06:34:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.171+0000 7f6f70b46f40 1 Processor -- start 2024-09-21T06:34:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.172+0000 7f6f70b46f40 1 -- start start 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ece5a0 0x557ea2e89660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ed1630 0x557ea2eb2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ea2dd5c60 con 0x557ea2eb11e0 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557ea2e341d0 con 0x557ea2ed1630 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557ea2eb61e0 con 0x557ea2ece5a0 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f6a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f6a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57134/0 (socket says 172.21.15.37:57134) 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f6a600640 1 -- 172.21.15.37:0/3351401572 learned_addr learned my addr 172.21.15.37:0/3351401572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f69c00640 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ece5a0 0x557ea2e89660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.173+0000 7f6f69200640 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ed1630 0x557ea2eb2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 -- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ece5a0 msgr2=0x557ea2e89660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ece5a0 0x557ea2e89660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 -- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ed1630 msgr2=0x557ea2eb2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ed1630 0x557ea2eb2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 -- 172.21.15.37:0/3351401572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea2ecd120 con 0x557ea2eb11e0 2024-09-21T06:34:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f6a600640 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c6d0 tx=0x7f6f6400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fe96f040d77df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f63400640 1 -- 172.21.15.37:0/3351401572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64004020 con 0x557ea2eb11e0 2024-09-21T06:34:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f63400640 1 -- 172.21.15.37:0/3351401572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f640042b0 con 0x557ea2eb11e0 2024-09-21T06:34:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.174+0000 7f6f63400640 1 -- 172.21.15.37:0/3351401572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400aa80 con 0x557ea2eb11e0 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 -- 172.21.15.37:0/3351401572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 msgr2=0x557ea2eb2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c6d0 tx=0x7f6f6400cb90 comp rx=0 tx=0).stop 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 -- 172.21.15.37:0/3351401572 shutdown_connections 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2eb2a70 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ed1630 0x557ea2eb2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/3351401572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ece5a0 0x557ea2e89660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 -- 172.21.15.37:0/3351401572 >> 172.21.15.37:0/3351401572 conn(0x557ea2da6ac0 msgr2=0x557ea2f5c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 -- 172.21.15.37:0/3351401572 shutdown_connections 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.175+0000 7f6f70b46f40 1 -- 172.21.15.37:0/3351401572 wait complete. 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.176+0000 7f6f70b46f40 1 Processor -- start 2024-09-21T06:34:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.176+0000 7f6f70b46f40 1 -- start start 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2f3fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ece5a0 0x557ea2eb2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ed1630 0x557ea2f3c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ea2f5dec0 con 0x557ea2eb11e0 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557ea2ecd390 con 0x557ea2ece5a0 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f70b46f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557ea2ecc3a0 con 0x557ea2ed1630 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f69c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2f3fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f69c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2f3fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57136/0 (socket says 172.21.15.37:57136) 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f69c00640 1 -- 172.21.15.37:0/653570131 learned_addr learned my addr 172.21.15.37:0/653570131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f69200640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ece5a0 0x557ea2eb2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f6a600640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ed1630 0x557ea2f3c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.178+0000 7f6f69c00640 1 -- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ed1630 msgr2=0x557ea2f3c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f69c00640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557ea2ed1630 0x557ea2f3c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f69c00640 1 -- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ece5a0 msgr2=0x557ea2eb2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f69c00640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557ea2ece5a0 0x557ea2eb2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f69c00640 1 -- 172.21.15.37:0/653570131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea2f3bcc0 con 0x557ea2eb11e0 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f69c00640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2f3fd80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c025f00 tx=0x7f6f5c04d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f62000640 1 -- 172.21.15.37:0/653570131 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c068070 con 0x557ea2eb11e0 2024-09-21T06:34:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f62000640 1 -- 172.21.15.37:0/653570131 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f5c058280 con 0x557ea2eb11e0 2024-09-21T06:34:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ea2f58720 con 0x557ea2eb11e0 2024-09-21T06:34:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f62000640 1 -- 172.21.15.37:0/653570131 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c058770 con 0x557ea2eb11e0 2024-09-21T06:34:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.179+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea2f3bf60 con 0x557ea2eb11e0 2024-09-21T06:34:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.181+0000 7f6f62000640 1 -- 172.21.15.37:0/653570131 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f5c06b070 con 0x557ea2eb11e0 2024-09-21T06:34:15.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.182+0000 7f6f62000640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c0775a0 0x7f6f4c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.182+0000 7f6f62000640 1 -- 172.21.15.37:0/653570131 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f6f5c0dd900 con 0x557ea2eb11e0 2024-09-21T06:34:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.183+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x557ea2f3c9c0 0x557ea3001060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.183+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- osd_op(unknown.0.0:1 13.5 13:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x557ea2f5d620 con 0x557ea2f3c9c0 2024-09-21T06:34:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.183+0000 7f6f69200640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c0775a0 0x7f6f4c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.183+0000 7f6f6a600640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x557ea2f3c9c0 0x557ea3001060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.184+0000 7f6f69200640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c0775a0 0x7f6f4c079ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x557ea2f5bc80 tx=0x7f6f58001750 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.184+0000 7f6f6a600640 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x557ea2f3c9c0 0x557ea3001060 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f6a600640 1 -- 172.21.15.37:0/653570131 <== osd.2 v2:172.21.15.37:6800/2037276396 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v91'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6f6400b040 con 0x557ea2f3c9c0 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x557ea2f3c9c0 msgr2=0x557ea3001060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x557ea2f3c9c0 0x557ea3001060 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c0775a0 msgr2=0x7f6f4c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6f4c0775a0 0x7f6f4c079ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x557ea2f5bc80 tx=0x7f6f58001750 comp rx=0 tx=0).stop 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 msgr2=0x557ea2f3fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 --2- 172.21.15.37:0/653570131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea2eb11e0 0x557ea2f3fd80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c025f00 tx=0x7f6f5c04d940 comp rx=0 tx=0).stop 2024-09-21T06:34:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f6a600640 1 -- 172.21.15.37:0/653570131 reap_dead start 2024-09-21T06:34:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 shutdown_connections 2024-09-21T06:34:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 >> 172.21.15.37:0/653570131 conn(0x557ea2da6ac0 msgr2=0x557ea2ecd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 shutdown_connections 2024-09-21T06:34:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.186+0000 7f6f70b46f40 1 -- 172.21.15.37:0/653570131 wait complete. 2024-09-21T06:34:15.199 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-21T06:34:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.343+0000 7f8e3d15ff40 1 Processor -- start 2024-09-21T06:34:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.343+0000 7f8e3d15ff40 1 -- start start 2024-09-21T06:34:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.344+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.344+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ada00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d7ad876c60 con 0x55d7ad96c240 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d7ad8d51d0 con 0x55d7ad924c20 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d7ad96c650 con 0x55d7ada019b0 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57148/0 (socket says 172.21.15.37:57148) 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 -- 172.21.15.37:0/2402461532 learned_addr learned my addr 172.21.15.37:0/2402461532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e36200640 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e36c00640 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ada00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 -- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 msgr2=0x55d7ada00940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ada00940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 -- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 msgr2=0x55d7ad9e04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9e04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.345+0000 7f8e35800640 1 -- 172.21.15.37:0/2402461532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ad973430 con 0x55d7ad96c240 2024-09-21T06:34:15.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.346+0000 7f8e35800640 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8e24025f00 tx=0x7f8e24002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a169bfa845a71152 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.346+0000 7f8e2fe00640 1 -- 172.21.15.37:0/2402461532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24056470 con 0x55d7ad96c240 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.346+0000 7f8e2fe00640 1 -- 172.21.15.37:0/2402461532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e24056610 con 0x55d7ad96c240 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.346+0000 7f8e2fe00640 1 -- 172.21.15.37:0/2402461532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24056a60 con 0x55d7ad96c240 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/2402461532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 msgr2=0x55d7ada00400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8e24025f00 tx=0x7f8e24002fb0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/2402461532 shutdown_connections 2024-09-21T06:34:15.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ada00400 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ada00940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/2402461532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/2402461532 >> 172.21.15.37:0/2402461532 conn(0x55d7ad847ac0 msgr2=0x55d7ad9e0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/2402461532 shutdown_connections 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.347+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/2402461532 wait complete. 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.348+0000 7f8e3d15ff40 1 Processor -- start 2024-09-21T06:34:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.348+0000 7f8e3d15ff40 1 -- start start 2024-09-21T06:34:15.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ad970660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ad9e0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d7ad9e1a60 con 0x55d7ad96c240 2024-09-21T06:34:15.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d7ad96b7d0 con 0x55d7ad924c20 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.349+0000 7f8e3d15ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d7ad970f00 con 0x55d7ada019b0 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ad970660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ad970660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57156/0 (socket says 172.21.15.37:57156) 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 -- 172.21.15.37:0/793621097 learned_addr learned my addr 172.21.15.37:0/793621097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e36200640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e36c00640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ad9e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 -- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 msgr2=0x55d7ad9e0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7ada019b0 0x55d7ad9e0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 -- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 msgr2=0x55d7ad9f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7ad924c20 0x55d7ad9f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.350+0000 7f8e35800640 1 -- 172.21.15.37:0/793621097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ada00c90 con 0x55d7ad96c240 2024-09-21T06:34:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e35800640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ad970660 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400a3d0 tx=0x7f8e24057b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e2ea00640 1 -- 172.21.15.37:0/793621097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2406f030 con 0x55d7ad96c240 2024-09-21T06:34:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e2ea00640 1 -- 172.21.15.37:0/793621097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e24057c00 con 0x55d7ad96c240 2024-09-21T06:34:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d7ada009f0 con 0x55d7ad96c240 2024-09-21T06:34:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e2ea00640 1 -- 172.21.15.37:0/793621097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24057e50 con 0x55d7ad96c240 2024-09-21T06:34:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.351+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ad9fd930 con 0x55d7ad96c240 2024-09-21T06:34:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.353+0000 7f8e2ea00640 1 -- 172.21.15.37:0/793621097 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e24072070 con 0x55d7ad96c240 2024-09-21T06:34:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.354+0000 7f8e2ea00640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e18077570 0x7f8e18079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.354+0000 7f8e2ea00640 1 -- 172.21.15.37:0/793621097 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f8e240ded50 con 0x55d7ad96c240 2024-09-21T06:34:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.354+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x55d7ada99580 0x55d7ada99990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.354+0000 7f8e36200640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e18077570 0x7f8e18079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.354+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- osd_op(unknown.0.0:1 13.9 13:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55d7ad9e0bd0 con 0x55d7ada99580 2024-09-21T06:34:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.355+0000 7f8e36c00640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x55d7ada99580 0x55d7ada99990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.355+0000 7f8e36200640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e18077570 0x7f8e18079a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c780 tx=0x7f8e2800f040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.355+0000 7f8e36c00640 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x55d7ada99580 0x55d7ada99990 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e36c00640 1 -- 172.21.15.37:0/793621097 <== osd.0 v2:172.21.15.37:6808/787824175 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v91'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e30010550 con 0x55d7ada99580 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x55d7ada99580 msgr2=0x55d7ada99990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x55d7ada99580 0x55d7ada99990 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e18077570 msgr2=0x7f8e18079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8e18077570 0x7f8e18079a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c780 tx=0x7f8e2800f040 comp rx=0 tx=0).stop 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 msgr2=0x55d7ad970660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 --2- 172.21.15.37:0/793621097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7ad96c240 0x55d7ad970660 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400a3d0 tx=0x7f8e24057b10 comp rx=0 tx=0).stop 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e36c00640 1 -- 172.21.15.37:0/793621097 reap_dead start 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 shutdown_connections 2024-09-21T06:34:15.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 >> 172.21.15.37:0/793621097 conn(0x55d7ad847ac0 msgr2=0x55d7ad9dfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.357+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 shutdown_connections 2024-09-21T06:34:15.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.358+0000 7f8e3d15ff40 1 -- 172.21.15.37:0/793621097 wait complete. 2024-09-21T06:34:15.373 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-21T06:34:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.515+0000 7f62d2b1ff40 1 Processor -- start 2024-09-21T06:34:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.515+0000 7f62d2b1ff40 1 -- start start 2024-09-21T06:34:15.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 0x55a316e6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 0x55a316e743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a316e8ee30 con 0x55a316e8c5a0 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a316d93c60 con 0x55a316e47470 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a316e8f110 con 0x55a316e8f630 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62ca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57178/0 (socket says 172.21.15.37:57178) 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62ca800640 1 -- 172.21.15.37:0/448276729 learned_addr learned my addr 172.21.15.37:0/448276729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62cb200640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 0x55a316e6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.516+0000 7f62cbc00640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 0x55a316e743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 -- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 msgr2=0x55a316e743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 0x55a316e743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 -- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 msgr2=0x55a316e6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 0x55a316e6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 -- 172.21.15.37:0/448276729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a316d6bf10 con 0x55a316e8c5a0 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62cb200640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 0x55a316e6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62cbc00640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 0x55a316e743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62ca800640 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f62b8025e80 tx=0x7f62b8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdad35c2769dce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62c9400640 1 -- 172.21.15.37:0/448276729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b804db70 con 0x55a316e8c5a0 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62c9400640 1 -- 172.21.15.37:0/448276729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62b804dd10 con 0x55a316e8c5a0 2024-09-21T06:34:15.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.517+0000 7f62c9400640 1 -- 172.21.15.37:0/448276729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80566d0 con 0x55a316e8c5a0 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/448276729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 msgr2=0x55a316e446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f62b8025e80 tx=0x7f62b8002fb0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/448276729 shutdown_connections 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8c5a0 0x55a316e446a0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e8f630 0x55a316e743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/448276729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e47470 0x55a316e6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/448276729 >> 172.21.15.37:0/448276729 conn(0x55a316d64ac0 msgr2=0x55a316e74f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/448276729 shutdown_connections 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.518+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/448276729 wait complete. 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.519+0000 7f62d2b1ff40 1 Processor -- start 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.519+0000 7f62d2b1ff40 1 -- start start 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e47470 0x55a316efb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e8c5a0 0x55a316efd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 0x55a316efda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a316f005b0 con 0x55a316e8f630 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a316f1bef0 con 0x55a316e8c5a0 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62d2b1ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a316e8bc30 con 0x55a316e47470 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 0x55a316efda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 0x55a316efda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57180/0 (socket says 172.21.15.37:57180) 2024-09-21T06:34:15.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 -- 172.21.15.37:0/3634674863 learned_addr learned my addr 172.21.15.37:0/3634674863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cb200640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e47470 0x55a316efb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62ca800640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e8c5a0 0x55a316efd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 -- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e47470 msgr2=0x55a316efb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a316e47470 0x55a316efb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 -- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e8c5a0 msgr2=0x55a316efd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.521+0000 7f62cbc00640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a316e8c5a0 0x55a316efd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62cbc00640 1 -- 172.21.15.37:0/3634674863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a316effa30 con 0x55a316e8f630 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62cbc00640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 0x55a316efda80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d660 tx=0x7f62c400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62c3e00640 1 -- 172.21.15.37:0/3634674863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c400bab0 con 0x55a316e8f630 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62c3e00640 1 -- 172.21.15.37:0/3634674863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62c4014070 con 0x55a316e8f630 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a316efc8a0 con 0x55a316e8f630 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62c3e00640 1 -- 172.21.15.37:0/3634674863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4019470 con 0x55a316e8f630 2024-09-21T06:34:15.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.522+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a316e8b970 con 0x55a316e8f630 2024-09-21T06:34:15.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.524+0000 7f62c3e00640 1 -- 172.21.15.37:0/3634674863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62c4010440 con 0x55a316e8f630 2024-09-21T06:34:15.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.525+0000 7f62c3e00640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f62ac0775a0 0x7f62ac079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.525+0000 7f62c3e00640 1 -- 172.21.15.37:0/3634674863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f62c409af60 con 0x55a316e8f630 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x55a316f016e0 0x55a316e8ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62cb200640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f62ac0775a0 0x7f62ac079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- osd_op(unknown.0.0:1 13.5 13:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a316e8b340 con 0x55a316f016e0 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62ca800640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x55a316f016e0 0x55a316e8ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62cb200640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f62ac0775a0 0x7f62ac079ab0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55a316e756c0 tx=0x7f62bc004880 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.526+0000 7f62ca800640 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x55a316f016e0 0x55a316e8ae00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.528+0000 7f62ca800640 1 -- 172.21.15.37:0/3634674863 <== osd.2 v2:172.21.15.37:6800/2037276396 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v91'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f62b805f030 con 0x55a316f016e0 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x55a316f016e0 msgr2=0x55a316e8ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x55a316f016e0 0x55a316e8ae00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f62ac0775a0 msgr2=0x7f62ac079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f62ac0775a0 0x7f62ac079ab0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55a316e756c0 tx=0x7f62bc004880 comp rx=0 tx=0).stop 2024-09-21T06:34:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 msgr2=0x55a316efda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 --2- 172.21.15.37:0/3634674863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a316e8f630 0x55a316efda80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d660 tx=0x7f62c400db20 comp rx=0 tx=0).stop 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62cbc00640 1 -- 172.21.15.37:0/3634674863 reap_dead start 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 shutdown_connections 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 >> 172.21.15.37:0/3634674863 conn(0x55a316d64ac0 msgr2=0x55a316e446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 shutdown_connections 2024-09-21T06:34:15.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.529+0000 7f62d2b1ff40 1 -- 172.21.15.37:0/3634674863 wait complete. 2024-09-21T06:34:15.542 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-21T06:34:15.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.684+0000 7f8b6f3eef40 1 Processor -- start 2024-09-21T06:34:15.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.684+0000 7f8b6f3eef40 1 -- start start 2024-09-21T06:34:15.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026a879b0 0x563026b5d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026acf360 0x563026b5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563026b5c160 con 0x563026b55280 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630269d3c60 con 0x563026acf360 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563026b55690 con 0x563026a879b0 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b69000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.685+0000 7f8b69000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57188/0 (socket says 172.21.15.37:57188) 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 -- 172.21.15.37:0/2386585387 learned_addr learned my addr 172.21.15.37:0/2386585387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b67c00640 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026acf360 0x563026b5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b68600640 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026a879b0 0x563026b5d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 -- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026a879b0 msgr2=0x563026b5d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026a879b0 0x563026b5d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 -- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026acf360 msgr2=0x563026b5c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026acf360 0x563026b5c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 -- 172.21.15.37:0/2386585387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630269abf10 con 0x563026b55280 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b69000640 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d760 tx=0x7f8b6000dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a975e667d0fe4b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b66800640 1 -- 172.21.15.37:0/2386585387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ba50 con 0x563026b55280 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b66800640 1 -- 172.21.15.37:0/2386585387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b6000bce0 con 0x563026b55280 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.686+0000 7f8b66800640 1 -- 172.21.15.37:0/2386585387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000f690 con 0x563026b55280 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2386585387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 msgr2=0x563026ab2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d760 tx=0x7f8b6000dc20 comp rx=0 tx=0).stop 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2386585387 shutdown_connections 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026b55280 0x563026ab2db0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026acf360 0x563026b5c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2386585387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026a879b0 0x563026b5d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2386585387 >> 172.21.15.37:0/2386585387 conn(0x5630269a4ac0 msgr2=0x563026ab1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2386585387 shutdown_connections 2024-09-21T06:34:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.687+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2386585387 wait complete. 2024-09-21T06:34:15.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.688+0000 7f8b6f3eef40 1 Processor -- start 2024-09-21T06:34:15.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.688+0000 7f8b6f3eef40 1 -- start start 2024-09-21T06:34:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026a879b0 0x563026b51a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 0x563026bf2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026b55280 0x563026b53130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563026b5c6e0 con 0x563026acf360 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563026ace590 con 0x563026a879b0 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b6f3eef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x563026ace370 con 0x563026b55280 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b67c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 0x563026bf2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b67c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 0x563026bf2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57198/0 (socket says 172.21.15.37:57198) 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b67c00640 1 -- 172.21.15.37:0/2053022794 learned_addr learned my addr 172.21.15.37:0/2053022794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b68600640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026a879b0 0x563026b51a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.690+0000 7f8b69000640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026b55280 0x563026b53130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 -- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026b55280 msgr2=0x563026b53130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563026b55280 0x563026b53130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 -- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026a879b0 msgr2=0x563026b51a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563026a879b0 0x563026b51a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 -- 172.21.15.37:0/2053022794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563026b52360 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b67c00640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 0x563026bf2d40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400a6e0 tx=0x7f8b5400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b65400640 1 -- 172.21.15.37:0/2053022794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5401b070 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b65400640 1 -- 172.21.15.37:0/2053022794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b54009ba0 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b65400640 1 -- 172.21.15.37:0/2053022794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54016460 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563026b52640 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.691+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563026b52c00 con 0x563026acf360 2024-09-21T06:34:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.694+0000 7f8b65400640 1 -- 172.21.15.37:0/2053022794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b5401e070 con 0x563026acf360 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.694+0000 7f8b65400640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b48077570 0x7f8b48079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.695+0000 7f8b65400640 1 -- 172.21.15.37:0/2053022794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f8b5409a4f0 con 0x563026acf360 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.695+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x563026bf23b0 0x563026b4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.695+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- osd_op(unknown.0.0:1 13.1d 13:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x563026b4f470 con 0x563026bf23b0 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.695+0000 7f8b68600640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b48077570 0x7f8b48079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.695+0000 7f8b69000640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x563026bf23b0 0x563026b4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.696+0000 7f8b68600640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b48077570 0x7f8b48079a80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b58055000 tx=0x7f8b58003020 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.696+0000 7f8b69000640 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x563026bf23b0 0x563026b4ef30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b69000640 1 -- 172.21.15.37:0/2053022794 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v91'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8b60015020 con 0x563026bf23b0 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x563026bf23b0 msgr2=0x563026b4ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x563026bf23b0 0x563026b4ef30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b48077570 msgr2=0x7f8b48079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8b48077570 0x7f8b48079a80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b58055000 tx=0x7f8b58003020 comp rx=0 tx=0).stop 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 msgr2=0x563026bf2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 --2- 172.21.15.37:0/2053022794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563026acf360 0x563026bf2d40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400a6e0 tx=0x7f8b5400aba0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b69000640 1 -- 172.21.15.37:0/2053022794 reap_dead start 2024-09-21T06:34:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 shutdown_connections 2024-09-21T06:34:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 >> 172.21.15.37:0/2053022794 conn(0x5630269a4ac0 msgr2=0x563026ab0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 shutdown_connections 2024-09-21T06:34:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.698+0000 7f8b6f3eef40 1 -- 172.21.15.37:0/2053022794 wait complete. 2024-09-21T06:34:15.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:15.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2024-09-21T06:34:15.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 -- 172.21.15.37:0/3122860493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc0b49b0 msgr2=0x7f55fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 --2- 172.21.15.37:0/3122860493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc0b49b0 0x7f55fc0b4dc0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005fd0 tx=0x7f55f402aae0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 -- 172.21.15.37:0/3122860493 shutdown_connections 2024-09-21T06:34:15.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 --2- 172.21.15.37:0/3122860493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc0b49b0 0x7f55fc0b4dc0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 --2- 172.21.15.37:0/3122860493 >> v2:172.21.15.150:3300/0 conn(0x7f55fc15f720 0x7f55fc155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 -- 172.21.15.37:0/3122860493 >> 172.21.15.37:0/3122860493 conn(0x7f55fc0d0460 msgr2=0x7f55fc06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 -- 172.21.15.37:0/3122860493 shutdown_connections 2024-09-21T06:34:15.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.943+0000 7f5602c00640 1 -- 172.21.15.37:0/3122860493 wait complete. 2024-09-21T06:34:15.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.945+0000 7f5602c00640 1 Processor -- start 2024-09-21T06:34:15.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.946+0000 7f5602c00640 1 -- start start 2024-09-21T06:34:15.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55fc0732a0 0x7f55fc145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55fc0b49b0 0x7f55fc1e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc0ae180 con 0x7f55fc15f720 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc0ae000 con 0x7f55fc0b49b0 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc0ae300 con 0x7f55fc0732a0 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57202/0 (socket says 172.21.15.37:57202) 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5602200640 1 -- 172.21.15.37:0/3541899494 learned_addr learned my addr 172.21.15.37:0/3541899494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5600e00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55fc0b49b0 0x7f55fc1e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.947+0000 7f5601800640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55fc0732a0 0x7f55fc145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 -- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55fc0732a0 msgr2=0x7f55fc145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55fc0732a0 0x7f55fc145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 -- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55fc0b49b0 msgr2=0x7f55fc1e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55fc0b49b0 0x7f55fc1e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 -- 172.21.15.37:0/3541899494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc1eb190 con 0x7f55fc15f720 2024-09-21T06:34:15.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602200640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009d90 tx=0x7f55f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f55f9600640 1 -- 172.21.15.37:0/3541899494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4044070 con 0x7f55fc15f720 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f55f9600640 1 -- 172.21.15.37:0/3541899494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f4003e00 con 0x7f55fc15f720 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f55f9600640 1 -- 172.21.15.37:0/3541899494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40044b0 con 0x7f55fc15f720 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.948+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1e8b70 con 0x7f55fc15f720 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.949+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1e9350 con 0x7f55fc15f720 2024-09-21T06:34:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.951+0000 7f55f9600640 1 -- 172.21.15.37:0/3541899494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55f4047070 con 0x7f55fc15f720 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.952+0000 7f55f9600640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 0x7f55dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.952+0000 7f55f9600640 1 -- 172.21.15.37:0/3541899494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f55f40bbe00 con 0x7f55fc15f720 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.952+0000 7f5601800640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 0x7f55dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.952+0000 7f560b1ea740 1 Processor -- start 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.953+0000 7f560b1ea740 1 -- start start 2024-09-21T06:34:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.953+0000 7f5601800640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 0x7f55dc07b0a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009b30 tx=0x7f55f0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f981316680 0x55f981705330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f98161d0c0 con 0x55f9812f59a0 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f981630f90 con 0x55f9812ed7c0 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f9816203c0 con 0x55f981316680 2024-09-21T06:34:15.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57218/0 (socket says 172.21.15.37:57218) 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f5602200640 1 -- 172.21.15.37:0/1029508103 learned_addr learned my addr 172.21.15.37:0/1029508103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f5601800640 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f981316680 0x55f981705330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.954+0000 7f5600e00640 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 -- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f981316680 msgr2=0x55f981705330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f981316680 0x55f981705330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 -- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 msgr2=0x55f9816e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 -- 172.21.15.37:0/1029508103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9813f6a50 con 0x55f9812f59a0 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f5602200640 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f55f4083390 tx=0x7f55f4083130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf5e8a02b222de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f55ee000640 1 -- 172.21.15.37:0/1029508103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40a8070 con 0x55f9812f59a0 2024-09-21T06:34:15.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f55ee000640 1 -- 172.21.15.37:0/1029508103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f4088070 con 0x55f9812f59a0 2024-09-21T06:34:15.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.955+0000 7f55ee000640 1 -- 172.21.15.37:0/1029508103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004850 con 0x55f9812f59a0 2024-09-21T06:34:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 -- 172.21.15.37:0/1029508103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 msgr2=0x55f9816e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f55f4083390 tx=0x7f55f4083130 comp rx=0 tx=0).stop 2024-09-21T06:34:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 -- 172.21.15.37:0/1029508103 shutdown_connections 2024-09-21T06:34:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816e8540 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1029508103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f981316680 0x55f981705330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 -- 172.21.15.37:0/1029508103 >> 172.21.15.37:0/1029508103 conn(0x55f98130a7d0 msgr2=0x55f9816be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 -- 172.21.15.37:0/1029508103 shutdown_connections 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.956+0000 7f560b1ea740 1 -- 172.21.15.37:0/1029508103 wait complete. 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.957+0000 7f560b1ea740 1 -- start start 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.958+0000 7f560b1ea740 1 Processor -- start 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f9816ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f9815c5af0 con 0x55f981316680 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f9816241d0 con 0x55f9812ed7c0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f981390a30 con 0x55f9812f59a0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f560b1ea740 10 client.0 _ll_register_callbacks cb 0x55f9815cbb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.959+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57232/0 (socket says 172.21.15.37:57232) 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 -- 172.21.15.37:0/2296228147 learned_addr learned my addr 172.21.15.37:0/2296228147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5601800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5600e00640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f9816ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 -- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 msgr2=0x55f9816ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f9816ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 -- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 msgr2=0x55f9816d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 -- 172.21.15.37:0/2296228147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98135f990 con 0x55f981316680 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f5602200640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f55f40ba070 tx=0x7f55f4004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3cb12eee38af5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f55e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f409de60 con 0x55f981316680 2024-09-21T06:34:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f55f4082e30 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.960+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f4088070 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f55f40a5050 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 10 client.7719 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55f9814571e0 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f4088070 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f55f407f310 con 0x55f981316680 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 10 client.7719 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 10 client.7719 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 20 client.7719 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f9812d5250 0x55f981715750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 4 client.7719 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.961+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f981632ee0 con 0x55f9812d5250 2024-09-21T06:34:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.962+0000 7f5601800640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f9812d5250 0x55f981715750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.962+0000 7f5601800640 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f9812d5250 0x55f981715750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d6159230f00434 server_cookie=f8bd7a595c64fba7 in_seq=0 out_seq=0 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.962+0000 7f55e2a00640 10 client.7719 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.963+0000 7f55e2a00640 1 -- 172.21.15.37:0/2296228147 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f981632ee0 con 0x55f9812d5250 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.963+0000 7f55e2a00640 10 client.7719 handle_command_reply: tid=0 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.963+0000 7f560b1ea740 1 client.7719 shutdown 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 msgr2=0x55f9816ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f55f40ba070 tx=0x7f55f4004d70 comp rx=0 tx=0).stop 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 shutdown_connections 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f9812d5250 0x55f981715750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f981316680 0x55f9816ffe20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f9816ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 --2- 172.21.15.37:0/2296228147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9812ed7c0 0x55f9816d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f55e2a00640 0 client.7719 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 >> 172.21.15.37:0/2296228147 conn(0x55f98130a7d0 msgr2=0x55f9816be290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 shutdown_connections 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 1 -- 172.21.15.37:0/2296228147 wait complete. 2024-09-21T06:34:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.964+0000 7f560b1ea740 20 client.7719 trim_cache size 0 max 16384 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.983+0000 7f560b1ea740 1 Processor -- start 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.983+0000 7f560b1ea740 1 -- start start 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f981724ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f981725020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc202420 con 0x55f9812ed7c0 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc2015f0 con 0x55f981316680 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc201cf0 con 0x55f9812f59a0 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f5601800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f5601800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57244/0 (socket says 172.21.15.37:57244) 2024-09-21T06:34:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.984+0000 7f5601800640 1 -- 172.21.15.37:0/1246449244 learned_addr learned my addr 172.21.15.37:0/1246449244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 -- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 msgr2=0x55f981724ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f981724ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 -- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 msgr2=0x55f981725020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f981725020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 -- 172.21.15.37:0/1246449244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9813f4d50 con 0x55f9812ed7c0 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f5601800640 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x55f9816d6c90 tx=0x7f55f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66313a96bb633c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f55ed600640 1 -- 172.21.15.37:0/1246449244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0035070 con 0x55f9812ed7c0 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f55ed600640 1 -- 172.21.15.37:0/1246449244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f0003bc0 con 0x55f9812ed7c0 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.985+0000 7f55ed600640 1 -- 172.21.15.37:0/1246449244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0030470 con 0x55f9812ed7c0 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 -- 172.21.15.37:0/1246449244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 msgr2=0x55f981712940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x55f9816d6c90 tx=0x7f55f0014000 comp rx=0 tx=0).stop 2024-09-21T06:34:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 -- 172.21.15.37:0/1246449244 shutdown_connections 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f981725020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812f59a0 0x55f981724ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 --2- 172.21.15.37:0/1246449244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812ed7c0 0x55f981712940 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 -- 172.21.15.37:0/1246449244 >> 172.21.15.37:0/1246449244 conn(0x55f98130a7d0 msgr2=0x55f9816fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 -- 172.21.15.37:0/1246449244 shutdown_connections 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.986+0000 7f560b1ea740 1 -- 172.21.15.37:0/1246449244 wait complete. 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.987+0000 7f560b1ea740 1 -- start start 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.987+0000 7f560b1ea740 1 Processor -- start 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 0x55f9817854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f9816fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:15.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1f9450 con 0x55f9812f59a0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f98171a600 con 0x55f981316680 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f9816fef80 con 0x55f9812ed7c0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f560b1ea740 10 client.0 _ll_register_callbacks cb 0x7f55f4083820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57260/0 (socket says 172.21.15.37:57260) 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 -- 172.21.15.37:0/3556644708 learned_addr learned my addr 172.21.15.37:0/3556644708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5602200640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f9816fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5601800640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 0x55f9817854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 -- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 msgr2=0x55f9817854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 0x55f9817854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 -- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 msgr2=0x55f9816fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f9816fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5600e00640 1 -- 172.21.15.37:0/3556644708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f981785e50 con 0x55f9812f59a0 2024-09-21T06:34:15.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.989+0000 7f5601800640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 0x55f9817854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f5600e00640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f55fc072b60 tx=0x7f55e8015840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9b28ee54e3447 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f55e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e80052e0 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f55e801ba60 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0003bc0 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f55e8026050 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f560b1ea740 10 client.7725 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55f9816f9d30 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.990+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f0003bc0 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f55e8029020 con 0x55f9812f59a0 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 10 client.7725 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 10 client.7725 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 20 client.7725 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:15.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f981719e20 0x55f981717ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 4 client.7725 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-21T06:34:16.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55f9816d75f0 con 0x55f981719e20 2024-09-21T06:34:16.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.991+0000 7f5601800640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f981719e20 0x55f981717ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.992+0000 7f5601800640 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f981719e20 0x55f981717ed0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160ac6c462d45a90 server_cookie=ef66535e90413c84 in_seq=0 out_seq=0 2024-09-21T06:34:16.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.992+0000 7f55e2a00640 10 client.7725 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.001 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:34:15.993+0000 7fa6d2a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1) n(v3 rc2024-09-21T06:34:00.701908+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5600e7f5f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:34:00.701908+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.001 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:34:15.994+0000 7f2617600640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1) n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e14e422000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.001 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:34:15.995+0000 7f2617600640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/AsLsXsFsx@2},l=7515 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e14e422580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.002 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-09-21T06:34:15.995+0000 7fa6dac00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1) n(v4 rc2024-09-21T06:34:00.701908+0000 5=0+5) (inest sync->mix g=1,2) caps={7515=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5600e7f5f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:34:00.701908+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f55e2a00640 1 -- 172.21.15.37:0/3556644708 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9816d75f0 con 0x55f981719e20 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f55e2a00640 10 client.7725 handle_command_reply: tid=0 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 client.7725 shutdown 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 msgr2=0x55f9816fa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f55fc072b60 tx=0x7f55e8015840 comp rx=0 tx=0).stop 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 shutdown_connections 2024-09-21T06:34:16.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55f981719e20 0x55f981717ed0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9812f59a0 0x55f9816fa940 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f981316680 0x55f9816fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 --2- 172.21.15.37:0/3556644708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9812ed7c0 0x55f9817854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 >> 172.21.15.37:0/3556644708 conn(0x55f98130a7d0 msgr2=0x55f981712940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f55e2a00640 0 client.7725 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.993+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 shutdown_connections 2024-09-21T06:34:16.003 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:34:15.994+0000 7f7e2bc00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1) n(v2 rc2024-09-21T06:34:00.701908+0000 4=0+4) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55606b8f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:34:00.701908+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.994+0000 7f560b1ea740 1 -- 172.21.15.37:0/3556644708 wait complete. 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.994+0000 7f560b1ea740 20 client.7725 trim_cache size 0 max 16384 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-21T06:34:16.004 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 msgr2=0x7f55dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 0x7f55dc07b0a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009b30 tx=0x7f55f0005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 msgr2=0x7f55fc1e7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009d90 tx=0x7f55f4003c10 comp rx=0 tx=0).stop 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 shutdown_connections 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f55dc078b90 0x7f55dc07b0a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55fc15f720 0x7f55fc1e7240 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55fc0b49b0 0x7f55fc1e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 --2- 172.21.15.37:0/3541899494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55fc0732a0 0x7f55fc145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 >> 172.21.15.37:0/3541899494 conn(0x7f55fc0d0460 msgr2=0x7f55fc154250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 shutdown_connections 2024-09-21T06:34:16.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:15.995+0000 7f5602c00640 1 -- 172.21.15.37:0/3541899494 wait complete. 2024-09-21T06:34:16.006 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:34:15.996+0000 7f7e33e00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1) n(v1 rc2024-09-21T06:34:00.701908+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7515=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55606b8f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:34:00.701908+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:34:16.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:16.024 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub pause 2024-09-21T06:34:16.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 -- 172.21.15.37:0/1058958174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574006e330 msgr2=0x7f57400cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 --2- 172.21.15.37:0/1058958174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574006e330 0x7f57400cdc70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f57400b2c70 tx=0x7f573802aa30 comp rx=0 tx=0).stop 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 -- 172.21.15.37:0/1058958174 shutdown_connections 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 --2- 172.21.15.37:0/1058958174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574006e330 0x7f57400cdc70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 --2- 172.21.15.37:0/1058958174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740074b60 0x7f574006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 --2- 172.21.15.37:0/1058958174 >> v2:172.21.15.150:3300/0 conn(0x7f5740156a50 0x7f5740074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 -- 172.21.15.37:0/1058958174 >> 172.21.15.37:0/1058958174 conn(0x7f57400b1b90 msgr2=0x7f57400b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 -- 172.21.15.37:0/1058958174 shutdown_connections 2024-09-21T06:34:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.256+0000 7f5748000640 1 -- 172.21.15.37:0/1058958174 wait complete. 2024-09-21T06:34:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.259+0000 7f5748000640 1 Processor -- start 2024-09-21T06:34:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.259+0000 7f5748000640 1 -- start start 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006e330 0x7f57401594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5740156a50 0x7f57401f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57400ce330 con 0x7f5740074b60 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57400ce1b0 con 0x7f5740156a50 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.260+0000 7f5748000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f57400ce4b0 con 0x7f574006e330 2024-09-21T06:34:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57290/0 (socket says 172.21.15.37:57290) 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 -- 172.21.15.37:0/1027157423 learned_addr learned my addr 172.21.15.37:0/1027157423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5745600640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006e330 0x7f57401594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5746000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5740156a50 0x7f57401f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 -- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006e330 msgr2=0x7f57401594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006e330 0x7f57401594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 -- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5740156a50 msgr2=0x7f57401f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5740156a50 0x7f57401f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 -- 172.21.15.37:0/1027157423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57400b5200 con 0x7f5740074b60 2024-09-21T06:34:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.261+0000 7f5744c00640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f572c00e7f0 tx=0x7f572c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.262+0000 7f573d600640 1 -- 172.21.15.37:0/1027157423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c00eea0 con 0x7f5740074b60 2024-09-21T06:34:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.262+0000 7f573d600640 1 -- 172.21.15.37:0/1027157423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f572c00ccc0 con 0x7f5740074b60 2024-09-21T06:34:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.262+0000 7f573d600640 1 -- 172.21.15.37:0/1027157423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c005280 con 0x7f5740074b60 2024-09-21T06:34:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.262+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574009edf0 con 0x7f5740074b60 2024-09-21T06:34:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.262+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57400b54a0 con 0x7f5740074b60 2024-09-21T06:34:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.264+0000 7f573d600640 1 -- 172.21.15.37:0/1027157423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f572c005420 con 0x7f5740074b60 2024-09-21T06:34:16.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.266+0000 7f573d600640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 0x7f572007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.266+0000 7f573d600640 1 -- 172.21.15.37:0/1027157423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f572c09b800 con 0x7f5740074b60 2024-09-21T06:34:16.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.267+0000 7f5745600640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 0x7f572007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.267+0000 7f574f0bc740 1 Processor -- start 2024-09-21T06:34:16.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.267+0000 7f574f0bc740 1 -- start start 2024-09-21T06:34:16.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f5745600640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 0x7f572007b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5734009b30 tx=0x7f5734005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6189370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe61898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fe6072c10 con 0x555fe5db7390 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555fe60d19f0 con 0x555fe5d8e4d0 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.268+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555fe60c0e20 con 0x555fe5d966b0 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57306/0 (socket says 172.21.15.37:57306) 2024-09-21T06:34:16.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 -- 172.21.15.37:0/4049595419 learned_addr learned my addr 172.21.15.37:0/4049595419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5744c00640 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6189370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5746000640 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe61898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 -- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 msgr2=0x555fe61898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe61898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 -- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 msgr2=0x555fe6189370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6189370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.269+0000 7f5745600640 1 -- 172.21.15.37:0/4049595419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe5e997d0 con 0x555fe5db7390 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.270+0000 7f5745600640 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5734011ec0 tx=0x7f5734014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0650b9fd22b5071 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.270+0000 7f5731c00640 1 -- 172.21.15.37:0/4049595419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400ebf0 con 0x555fe5db7390 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.270+0000 7f5731c00640 1 -- 172.21.15.37:0/4049595419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5734004020 con 0x555fe5db7390 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.270+0000 7f5731c00640 1 -- 172.21.15.37:0/4049595419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734004410 con 0x555fe5db7390 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 -- 172.21.15.37:0/4049595419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 msgr2=0x555fe61a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5734011ec0 tx=0x7f5734014000 comp rx=0 tx=0).stop 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 -- 172.21.15.37:0/4049595419 shutdown_connections 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe61898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6189370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 --2- 172.21.15.37:0/4049595419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe61a66a0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 -- 172.21.15.37:0/4049595419 >> 172.21.15.37:0/4049595419 conn(0x555fe5dab4e0 msgr2=0x555fe615f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 -- 172.21.15.37:0/4049595419 shutdown_connections 2024-09-21T06:34:16.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.271+0000 7f574f0bc740 1 -- 172.21.15.37:0/4049595419 wait complete. 2024-09-21T06:34:16.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.272+0000 7f574f0bc740 1 -- start start 2024-09-21T06:34:16.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.272+0000 7f574f0bc740 1 Processor -- start 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6177c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe619b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fe6066790 con 0x555fe5db7390 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555fe60c4c30 con 0x555fe5d8e4d0 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555fe5e31740 con 0x555fe5d966b0 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f574f0bc740 10 client.0 _ll_register_callbacks cb 0x555fe6076260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5746000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5746000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57312/0 (socket says 172.21.15.37:57312) 2024-09-21T06:34:16.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5746000640 1 -- 172.21.15.37:0/522569600 learned_addr learned my addr 172.21.15.37:0/522569600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5745600640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6177c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5744c00640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe619b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5746000640 1 -- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 msgr2=0x555fe619b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.274+0000 7f5746000640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe619b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5746000640 1 -- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 msgr2=0x555fe6177c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5746000640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6177c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5746000640 1 -- 172.21.15.37:0/522569600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe5e006a0 con 0x555fe5db7390 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5745600640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6177c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5744c00640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe619b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5746000640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f573802b020 tx=0x7f5738003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f51381e9d48c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5726000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738047060 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f5738004040 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5734004020 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f573804d050 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f574f0bc740 10 client.7737 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.275+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x555fe5ef9f60 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5734004020 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f5738050020 con 0x555fe5db7390 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 10 client.7737 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 10 client.7737 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 20 client.7737 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe5d75f60 0x555fe61b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 4 client.7737 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fe60d3940 con 0x555fe5d75f60 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.276+0000 7f5745600640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe5d75f60 0x555fe61b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.277+0000 7f5745600640 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe5d75f60 0x555fe61b6c50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d70c16d15950d22 server_cookie=bf2425dd3fd61bc8 in_seq=0 out_seq=0 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.277+0000 7f5726000640 10 client.7737 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f5726000640 1 -- 172.21.15.37:0/522569600 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fe60d3940 con 0x555fe5d75f60 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f5726000640 10 client.7737 handle_command_reply: tid=0 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 client.7737 shutdown 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 msgr2=0x555fe619bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f573802b020 tx=0x7f5738003ba0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 shutdown_connections 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe5d75f60 0x555fe61b6c50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5db7390 0x555fe619bc40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe619b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 --2- 172.21.15.37:0/522569600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5d8e4d0 0x555fe6177c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f5726000640 0 client.7737 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.278+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 >> 172.21.15.37:0/522569600 conn(0x555fe5dab4e0 msgr2=0x555fe6158810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.279+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 shutdown_connections 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.279+0000 7f574f0bc740 1 -- 172.21.15.37:0/522569600 wait complete. 2024-09-21T06:34:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.279+0000 7f574f0bc740 20 client.7737 trim_cache size 0 max 16384 2024-09-21T06:34:16.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.297+0000 7f574f0bc740 1 Processor -- start 2024-09-21T06:34:16.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.297+0000 7f574f0bc740 1 -- start start 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe6160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe619cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5740202780 con 0x555fe5d8e4d0 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5740201970 con 0x555fe5db7390 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.298+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5740202050 con 0x555fe5d966b0 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57314/0 (socket says 172.21.15.37:57314) 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 -- 172.21.15.37:0/856534871 learned_addr learned my addr 172.21.15.37:0/856534871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5744c00640 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe6160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5746000640 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe619cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 -- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 msgr2=0x555fe6160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe6160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 -- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 msgr2=0x555fe619cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe619cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 -- 172.21.15.37:0/856534871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe5e97ad0 con 0x555fe5d8e4d0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5745600640 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f5734011ec0 tx=0x7f5734004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c2a393ce26d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.299+0000 7f5731200640 1 -- 172.21.15.37:0/856534871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734004040 con 0x555fe5d8e4d0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f5731200640 1 -- 172.21.15.37:0/856534871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57340041e0 con 0x555fe5d8e4d0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f5731200640 1 -- 172.21.15.37:0/856534871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340035f0 con 0x555fe5d8e4d0 2024-09-21T06:34:16.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 -- 172.21.15.37:0/856534871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 msgr2=0x555fe619a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f5734011ec0 tx=0x7f5734004730 comp rx=0 tx=0).stop 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 -- 172.21.15.37:0/856534871 shutdown_connections 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe619cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d966b0 0x555fe6160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 --2- 172.21.15.37:0/856534871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d8e4d0 0x555fe619a280 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 -- 172.21.15.37:0/856534871 >> 172.21.15.37:0/856534871 conn(0x555fe5dab4e0 msgr2=0x555fe615e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 -- 172.21.15.37:0/856534871 shutdown_connections 2024-09-21T06:34:16.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.300+0000 7f574f0bc740 1 -- 172.21.15.37:0/856534871 wait complete. 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.301+0000 7f574f0bc740 1 -- start start 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.302+0000 7f574f0bc740 1 Processor -- start 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d8e4d0 0x555fe61b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe61898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b4960 con 0x555fe5d966b0 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555fe619c910 con 0x555fe5db7390 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555fe61b4870 con 0x555fe5d8e4d0 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f574f0bc740 10 client.0 _ll_register_callbacks cb 0x7f5738009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f5744c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f5744c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57330/0 (socket says 172.21.15.37:57330) 2024-09-21T06:34:16.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f5744c00640 1 -- 172.21.15.37:0/752407490 learned_addr learned my addr 172.21.15.37:0/752407490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f5745600640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d8e4d0 0x555fe61b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.303+0000 7f5746000640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe61898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 -- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d8e4d0 msgr2=0x555fe61b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d8e4d0 0x555fe61b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 -- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 msgr2=0x555fe61898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe61898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 -- 172.21.15.37:0/752407490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe6223e10 con 0x555fe5d966b0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5744c00640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f572c00e7c0 tx=0x7f572c063940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be992440f5d48e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5726000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c06c040 con 0x555fe5d966b0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f572c085b10 con 0x555fe5d966b0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.304+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57340041e0 con 0x555fe5d966b0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f572c067df0 con 0x555fe5d966b0 2024-09-21T06:34:16.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 10 client.7743 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x555fe61b45b0 con 0x555fe5d966b0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57340041e0 con 0x555fe5d966b0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f572c05ed10 con 0x555fe5d966b0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 10 client.7743 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 10 client.7743 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 20 client.7743 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe62272f0 0x555fe6227700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 4 client.7743 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x555fe60cf3f0 con 0x555fe62272f0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.305+0000 7f5745600640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe62272f0 0x555fe6227700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.306+0000 7f5745600640 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe62272f0 0x555fe6227700 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bc03fe2c64c64c server_cookie=336e6100598c03b3 in_seq=0 out_seq=0 2024-09-21T06:34:16.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.306+0000 7f5726000640 10 client.7743 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.306+0000 7f5726000640 1 -- 172.21.15.37:0/752407490 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fe60cf3f0 con 0x555fe62272f0 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.306+0000 7f5726000640 10 client.7743 handle_command_reply: tid=0 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.306+0000 7f574f0bc740 1 client.7743 shutdown 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 msgr2=0x555fe6189370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f572c00e7c0 tx=0x7f572c063940 comp rx=0 tx=0).stop 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 shutdown_connections 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x555fe62272f0 0x555fe6227700 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fe5d966b0 0x555fe6189370 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fe5db7390 0x555fe61898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 --2- 172.21.15.37:0/752407490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555fe5d8e4d0 0x555fe61b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f5726000640 0 client.7743 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 >> 172.21.15.37:0/752407490 conn(0x555fe5dab4e0 msgr2=0x555fe61b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 shutdown_connections 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 1 -- 172.21.15.37:0/752407490 wait complete. 2024-09-21T06:34:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f574f0bc740 20 client.7743 trim_cache size 0 max 16384 2024-09-21T06:34:16.313 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:34:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:34:16.313 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.307+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 msgr2=0x7f572007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 0x7f572007b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5734009b30 tx=0x7f5734005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 msgr2=0x7f5740159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f572c00e7f0 tx=0x7f572c00ecb0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 shutdown_connections 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5720078b90 0x7f572007b0a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5740074b60 0x7f5740159a10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5740156a50 0x7f57401f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 --2- 172.21.15.37:0/1027157423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006e330 0x7f57401594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 >> 172.21.15.37:0/1027157423 conn(0x7f57400b1b90 msgr2=0x7f57401560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 shutdown_connections 2024-09-21T06:34:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.308+0000 7f5748000640 1 -- 172.21.15.37:0/1027157423 wait complete. 2024-09-21T06:34:16.331 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:16.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-21T06:34:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 -- 172.21.15.37:0/2164139403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 msgr2=0x7fe404152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 --2- 172.21.15.37:0/2164139403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe404152450 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe4040ce890 tx=0x7fe3f402a4d0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 -- 172.21.15.37:0/2164139403 shutdown_connections 2024-09-21T06:34:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 --2- 172.21.15.37:0/2164139403 >> v2:172.21.15.150:3300/0 conn(0x7fe404071160 0x7fe404071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 --2- 172.21.15.37:0/2164139403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe404152450 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 -- 172.21.15.37:0/2164139403 >> 172.21.15.37:0/2164139403 conn(0x7fe4040d1020 msgr2=0x7fe4040d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 -- 172.21.15.37:0/2164139403 shutdown_connections 2024-09-21T06:34:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.560+0000 7fe403e00640 1 -- 172.21.15.37:0/2164139403 wait complete. 2024-09-21T06:34:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.562+0000 7fe403e00640 1 Processor -- start 2024-09-21T06:34:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.562+0000 7fe403e00640 1 -- start start 2024-09-21T06:34:16.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 0x7fe4041e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 0x7fe40413fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4040b40d0 con 0x7fe404072320 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4040b3f50 con 0x7fe40415b900 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe4040b4250 con 0x7fe404071160 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57350/0 (socket says 172.21.15.37:57350) 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 -- 172.21.15.37:0/1191193930 learned_addr learned my addr 172.21.15.37:0/1191193930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe402a00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 0x7fe4041e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403400640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 0x7fe40413fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 -- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 msgr2=0x7fe4041e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 0x7fe4041e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 -- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 msgr2=0x7fe40413fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 0x7fe40413fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe3fa000640 1 -- 172.21.15.37:0/1191193930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe404153a80 con 0x7fe404072320 2024-09-21T06:34:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe402a00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 0x7fe4041e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.564+0000 7fe403400640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 0x7fe40413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe3fa000640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c390 tx=0x7fe3ec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe3fb400640 1 -- 172.21.15.37:0/1191193930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec017070 con 0x7fe404072320 2024-09-21T06:34:16.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe3fb400640 1 -- 172.21.15.37:0/1191193930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3ec0053c0 con 0x7fe404072320 2024-09-21T06:34:16.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe404153d60 con 0x7fe404072320 2024-09-21T06:34:16.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe3fb400640 1 -- 172.21.15.37:0/1191193930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec003850 con 0x7fe404072320 2024-09-21T06:34:16.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.565+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe404154320 con 0x7fe404072320 2024-09-21T06:34:16.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.568+0000 7fe3fb400640 1 -- 172.21.15.37:0/1191193930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3ec01a070 con 0x7fe404072320 2024-09-21T06:34:16.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.569+0000 7fe3fb400640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 0x7fe3e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.569+0000 7fe3fb400640 1 -- 172.21.15.37:0/1191193930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe3ec09a2a0 con 0x7fe404072320 2024-09-21T06:34:16.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.569+0000 7fe4108bc740 1 Processor -- start 2024-09-21T06:34:16.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.569+0000 7fe4108bc740 1 -- start start 2024-09-21T06:34:16.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.569+0000 7fe402a00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 0x7fe3e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.570+0000 7fe402a00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 0x7fe3e007b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf291400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf291940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630bf1c71c0 con 0x5630beec0390 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630bf1da870 con 0x5630bee974d0 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5630bf1c9ca0 con 0x5630bee9f6b0 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57362/0 (socket says 172.21.15.37:57362) 2024-09-21T06:34:16.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 -- 172.21.15.37:0/739324650 learned_addr learned my addr 172.21.15.37:0/739324650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe3fa000640 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf291400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe403400640 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf291940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 -- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 msgr2=0x5630bf291940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf291940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 -- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 msgr2=0x5630bf291400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf291400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.571+0000 7fe402a00640 1 -- 172.21.15.37:0/739324650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630befa27d0 con 0x5630beec0390 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.572+0000 7fe402a00640 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40042d0 tx=0x7fe3f403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae680393a5e652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.572+0000 7fe3f3400640 1 -- 172.21.15.37:0/739324650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4035040 con 0x5630beec0390 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.572+0000 7fe3f3400640 1 -- 172.21.15.37:0/739324650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3f404d070 con 0x5630beec0390 2024-09-21T06:34:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.572+0000 7fe3f3400640 1 -- 172.21.15.37:0/739324650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004a00 con 0x5630beec0390 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 -- 172.21.15.37:0/739324650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 msgr2=0x5630bf2ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40042d0 tx=0x7fe3f403c000 comp rx=0 tx=0).stop 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 -- 172.21.15.37:0/739324650 shutdown_connections 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf291940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 --2- 172.21.15.37:0/739324650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf2ae730 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 -- 172.21.15.37:0/739324650 >> 172.21.15.37:0/739324650 conn(0x5630beeb44e0 msgr2=0x5630bf267620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 -- 172.21.15.37:0/739324650 shutdown_connections 2024-09-21T06:34:16.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.573+0000 7fe4108bc740 1 -- 172.21.15.37:0/739324650 wait complete. 2024-09-21T06:34:16.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.574+0000 7fe4108bc740 1 -- start start 2024-09-21T06:34:16.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.574+0000 7fe4108bc740 1 Processor -- start 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf27fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf2a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630bf170410 con 0x5630bee9f6b0 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630bf1cdab0 con 0x5630bee974d0 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5630bef3a740 con 0x5630beec0390 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe4108bc740 10 client.0 _ll_register_callbacks cb 0x5630bf14d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57378/0 (socket says 172.21.15.37:57378) 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe3fa000640 1 -- 172.21.15.37:0/2272728401 learned_addr learned my addr 172.21.15.37:0/2272728401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe402a00640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf27fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.576+0000 7fe403400640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf2a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 -- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 msgr2=0x5630bf2a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf2a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 -- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 msgr2=0x5630bf27fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf27fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 -- 172.21.15.37:0/2272728401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630bef2b970 con 0x5630bee9f6b0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe402a00640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf27fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000fb0 tx=0x7fe3ec05e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a6eb1eaabe617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec064040 con 0x5630bee9f6b0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe3ec0610a0 con 0x5630bee9f6b0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f404d070 con 0x5630bee9f6b0 2024-09-21T06:34:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7fe3ec061e40 con 0x5630bee9f6b0 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe4108bc740 10 client.7758 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.577+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5630bef096a0 con 0x5630bee9f6b0 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3f404d070 con 0x5630bee9f6b0 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7fe3ec05d790 con 0x5630bee9f6b0 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 10 client.7758 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 10 client.7758 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 20 client.7758 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bee7ef60 0x5630bf2a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 4 client.7758 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630bf1dc7c0 con 0x5630bee7ef60 2024-09-21T06:34:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.578+0000 7fe402a00640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bee7ef60 0x5630bf2a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.579+0000 7fe402a00640 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bee7ef60 0x5630bf2a7500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=225732970c188b2c server_cookie=32a617aaa21fce09 in_seq=0 out_seq=0 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.579+0000 7fe3e7e00640 10 client.7758 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.579+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2272728401 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5630bf1dc7c0 con 0x5630bee7ef60 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.579+0000 7fe3e7e00640 10 client.7758 handle_command_reply: tid=0 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.579+0000 7fe4108bc740 1 client.7758 shutdown 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 msgr2=0x5630bf2a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000fb0 tx=0x7fe3ec05e490 comp rx=0 tx=0).stop 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 shutdown_connections 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bee7ef60 0x5630bf2a7500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf2a3010 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf2a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe3e7e00640 0 client.7758 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2272728401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf27fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 >> 172.21.15.37:0/2272728401 conn(0x5630beeb44e0 msgr2=0x5630bf260830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 shutdown_connections 2024-09-21T06:34:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 1 -- 172.21.15.37:0/2272728401 wait complete. 2024-09-21T06:34:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.580+0000 7fe4108bc740 20 client.7758 trim_cache size 0 max 16384 2024-09-21T06:34:16.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.599+0000 7fe4108bc740 1 Processor -- start 2024-09-21T06:34:16.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.599+0000 7fe4108bc740 1 -- start start 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.600+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf2a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.600+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf2a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.600+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe4108bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe404202280 con 0x5630beec0390 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe404201450 con 0x5630bee974d0 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe404201b50 con 0x5630bee9f6b0 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57390/0 (socket says 172.21.15.37:57390) 2024-09-21T06:34:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 -- 172.21.15.37:0/832684934 learned_addr learned my addr 172.21.15.37:0/832684934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe402a00640 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf2a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe3fa000640 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf2a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 -- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 msgr2=0x5630bf2a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf2a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 -- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 msgr2=0x5630bf2a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf2a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.601+0000 7fe403400640 1 -- 172.21.15.37:0/832684934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630bf2af1a0 con 0x5630beec0390 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.602+0000 7fe403400640 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00af80 tx=0x7fe3fc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4abcaa4a25e8e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.602+0000 7fe3f2a00640 1 -- 172.21.15.37:0/832684934 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc004b20 con 0x5630beec0390 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.602+0000 7fe3f2a00640 1 -- 172.21.15.37:0/832684934 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3fc004cc0 con 0x5630beec0390 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.602+0000 7fe3f2a00640 1 -- 172.21.15.37:0/832684934 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc003bd0 con 0x5630beec0390 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 -- 172.21.15.37:0/832684934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 msgr2=0x5630bf288510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00af80 tx=0x7fe3fc0128e0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 -- 172.21.15.37:0/832684934 shutdown_connections 2024-09-21T06:34:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630beec0390 0x5630bf288510 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630bee9f6b0 0x5630bf2a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 --2- 172.21.15.37:0/832684934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf2a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 -- 172.21.15.37:0/832684934 >> 172.21.15.37:0/832684934 conn(0x5630beeb44e0 msgr2=0x5630bf2afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 -- 172.21.15.37:0/832684934 shutdown_connections 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.603+0000 7fe4108bc740 1 -- 172.21.15.37:0/832684934 wait complete. 2024-09-21T06:34:16.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.604+0000 7fe4108bc740 1 -- start start 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.604+0000 7fe4108bc740 1 Processor -- start 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf324fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf288a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4041f92e0 con 0x5630bee9f6b0 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630bf2ada40 con 0x5630bee974d0 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5630bf32b630 con 0x5630beec0390 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.605+0000 7fe4108bc740 10 client.0 _ll_register_callbacks cb 0x7fe3ec0011d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57404/0 (socket says 172.21.15.37:57404) 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 -- 172.21.15.37:0/2138765676 learned_addr learned my addr 172.21.15.37:0/2138765676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe402a00640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf324fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe403400640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf288a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 -- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 msgr2=0x5630bf288a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf288a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 -- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 msgr2=0x5630bf324fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf324fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 -- 172.21.15.37:0/2138765676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630bf268300 con 0x5630bee9f6b0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3fa000640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec062000 tx=0x7fe3ec05ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a869f68d67ce6758 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.606+0000 7fe3e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec064040 con 0x5630bee9f6b0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe3ec0612b0 con 0x5630bee9f6b0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc004cc0 con 0x5630bee9f6b0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7fe3ec066920 con 0x5630bee9f6b0 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 10 client.7764 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5630bf2cdff0 con 0x5630bee9f6b0 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3fc004cc0 con 0x5630bee9f6b0 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7fe3ec067ab0 con 0x5630bee9f6b0 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 10 client.7764 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 10 client.7764 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 20 client.7764 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bf2afbb0 0x5630bf32e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 4 client.7764 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.607+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630bf1d8270 con 0x5630bf2afbb0 2024-09-21T06:34:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe402a00640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bf2afbb0 0x5630bf32e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe402a00640 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bf2afbb0 0x5630bf32e9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec49e0929b49a65b server_cookie=688f09f7eb747d2f in_seq=0 out_seq=0 2024-09-21T06:34:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe3e7e00640 10 client.7764 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe3e7e00640 1 -- 172.21.15.37:0/2138765676 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5630bf1d8270 con 0x5630bf2afbb0 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe3e7e00640 10 client.7764 handle_command_reply: tid=0 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.608+0000 7fe4108bc740 1 client.7764 shutdown 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 msgr2=0x5630bf288510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec062000 tx=0x7fe3ec05ddf0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 shutdown_connections 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x5630bf2afbb0 0x5630bf32e9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630bee9f6b0 0x5630bf288510 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5630beec0390 0x5630bf288a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 --2- 172.21.15.37:0/2138765676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630bee974d0 0x5630bf324fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe3e7e00640 0 client.7764 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 >> 172.21.15.37:0/2138765676 conn(0x5630beeb44e0 msgr2=0x5630bf261400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 shutdown_connections 2024-09-21T06:34:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 1 -- 172.21.15.37:0/2138765676 wait complete. 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.609+0000 7fe4108bc740 20 client.7764 trim_cache size 0 max 16384 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:16.615 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 msgr2=0x7fe3e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 0x7fe3e007b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4004730 comp rx=0 tx=0).stop 2024-09-21T06:34:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 msgr2=0x7fe40413f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c390 tx=0x7fe3ec00c850 comp rx=0 tx=0).stop 2024-09-21T06:34:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 shutdown_connections 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe3e0078b90 0x7fe3e007b0a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe404072320 0x7fe40413f7b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe40415b900 0x7fe40413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 --2- 172.21.15.37:0/1191193930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe404071160 0x7fe4041e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 >> 172.21.15.37:0/1191193930 conn(0x7fe4040d1020 msgr2=0x7fe4040d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 shutdown_connections 2024-09-21T06:34:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.610+0000 7fe403e00640 1 -- 172.21.15.37:0/1191193930 wait complete. 2024-09-21T06:34:16.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:16.637 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:16.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 -- 172.21.15.37:0/808560398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5580cf9a0 msgr2=0x7ff558072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 --2- 172.21.15.37:0/808560398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5580cf9a0 0x7ff558072d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff550005fd0 tx=0x7ff55002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 -- 172.21.15.37:0/808560398 shutdown_connections 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 --2- 172.21.15.37:0/808560398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5580cf9a0 0x7ff558072d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 --2- 172.21.15.37:0/808560398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff55815fc50 0x7ff55806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 -- 172.21.15.37:0/808560398 >> 172.21.15.37:0/808560398 conn(0x7ff5580b1b90 msgr2=0x7ff5580b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.863+0000 7ff55f800640 1 -- 172.21.15.37:0/808560398 shutdown_connections 2024-09-21T06:34:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.864+0000 7ff55f800640 1 -- 172.21.15.37:0/808560398 wait complete. 2024-09-21T06:34:16.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.866+0000 7ff55f800640 1 Processor -- start 2024-09-21T06:34:16.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.866+0000 7ff55f800640 1 -- start start 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5580cf9a0 0x7ff5581f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55815fc50 0x7ff5581f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff558070d20 con 0x7ff55806e330 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff558070ba0 con 0x7ff55815fc50 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.867+0000 7ff55f800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5580732d0 con 0x7ff5580cf9a0 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57408/0 (socket says 172.21.15.37:57408) 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 -- 172.21.15.37:0/2567685350 learned_addr learned my addr 172.21.15.37:0/2567685350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff54fe00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5580cf9a0 0x7ff5581f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55d800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55815fc50 0x7ff5581f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 -- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5580cf9a0 msgr2=0x7ff5581f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5580cf9a0 0x7ff5581f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 -- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55815fc50 msgr2=0x7ff5581f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55815fc50 0x7ff5581f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.868+0000 7ff55ce00640 1 -- 172.21.15.37:0/2567685350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55815af40 con 0x7ff55806e330 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff55ce00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e790 tx=0x7ff54800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff555600640 1 -- 172.21.15.37:0/2567685350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800eea0 con 0x7ff55806e330 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5580c8480 con 0x7ff55806e330 2024-09-21T06:34:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff555600640 1 -- 172.21.15.37:0/2567685350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff54800cdb0 con 0x7ff55806e330 2024-09-21T06:34:16.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff555600640 1 -- 172.21.15.37:0/2567685350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548010690 con 0x7ff55806e330 2024-09-21T06:34:16.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.869+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff558072f70 con 0x7ff55806e330 2024-09-21T06:34:16.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.871+0000 7ff555600640 1 -- 172.21.15.37:0/2567685350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff548010830 con 0x7ff55806e330 2024-09-21T06:34:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.872+0000 7ff555600640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 0x7ff53407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.872+0000 7ff555600640 1 -- 172.21.15.37:0/2567685350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff54809ba20 con 0x7ff55806e330 2024-09-21T06:34:16.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.873+0000 7ff54fe00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 0x7ff53407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.873+0000 7ff5667d8740 1 Processor -- start 2024-09-21T06:34:16.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.873+0000 7ff5667d8740 1 -- start start 2024-09-21T06:34:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.873+0000 7ff54fe00640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 0x7ff53407b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff5580b15e0 tx=0x7ff540009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.874+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b9e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.874+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b47874d0 0x55f9b4b81220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.874+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.874+0000 7ff5667d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4ab71c0 con 0x55f9b478f6b0 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4aca870 con 0x55f9b47b0390 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f9b4ab9ca0 con 0x55f9b47874d0 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57420/0 (socket says 172.21.15.37:57420) 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 -- 172.21.15.37:0/58727634 learned_addr learned my addr 172.21.15.37:0/58727634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55ce00640 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b9e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff54fe00640 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b47874d0 0x55f9b4b81220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 -- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b47874d0 msgr2=0x55f9b4b81220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b47874d0 0x55f9b4b81220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 -- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 msgr2=0x55f9b4b9e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b9e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.875+0000 7ff55d800640 1 -- 172.21.15.37:0/58727634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b48927d0 con 0x55f9b478f6b0 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.876+0000 7ff55d800640 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff550009ca0 tx=0x7ff550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab22b8b13c8eb2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.876+0000 7ff54d600640 1 -- 172.21.15.37:0/58727634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550003c70 con 0x55f9b478f6b0 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.876+0000 7ff54d600640 1 -- 172.21.15.37:0/58727634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff550040070 con 0x55f9b478f6b0 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.876+0000 7ff54d600640 1 -- 172.21.15.37:0/58727634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550031640 con 0x55f9b478f6b0 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 -- 172.21.15.37:0/58727634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 msgr2=0x55f9b4b81760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff550009ca0 tx=0x7ff550004a40 comp rx=0 tx=0).stop 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 -- 172.21.15.37:0/58727634 shutdown_connections 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b478f6b0 0x55f9b4b81760 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b47874d0 0x55f9b4b81220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 --2- 172.21.15.37:0/58727634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b9e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 -- 172.21.15.37:0/58727634 >> 172.21.15.37:0/58727634 conn(0x55f9b47a44e0 msgr2=0x55f9b4b57440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 -- 172.21.15.37:0/58727634 shutdown_connections 2024-09-21T06:34:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.877+0000 7ff5667d8740 1 -- 172.21.15.37:0/58727634 wait complete. 2024-09-21T06:34:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.878+0000 7ff5667d8740 1 -- start start 2024-09-21T06:34:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.878+0000 7ff5667d8740 1 Processor -- start 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b9b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b56a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4a60410 con 0x55f9b47874d0 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4abdab0 con 0x55f9b47b0390 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f9b482a740 con 0x55f9b478f6b0 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff5667d8740 10 client.0 _ll_register_callbacks cb 0x55f9b4a3d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57434/0 (socket says 172.21.15.37:57434) 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 -- 172.21.15.37:0/4291231983 learned_addr learned my addr 172.21.15.37:0/4291231983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55d800640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b56a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff54fe00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b9b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 -- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 msgr2=0x55f9b4b9b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b9b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 -- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 msgr2=0x55f9b4b56a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b56a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.880+0000 7ff55ce00640 1 -- 172.21.15.37:0/4291231983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b481b970 con 0x55f9b47874d0 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff55ce00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff548063440 tx=0x7ff548062e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a830a46784629 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff546000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548089070 con 0x55f9b47874d0 2024-09-21T06:34:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff54805ee00 con 0x55f9b47874d0 2024-09-21T06:34:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff550040070 con 0x55f9b47874d0 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7ff548084050 con 0x55f9b47874d0 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff5667d8740 10 client.7776 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55f9b47f96a0 con 0x55f9b47874d0 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.881+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff54806f070 con 0x55f9b47874d0 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7ff54805f000 con 0x55f9b47874d0 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 10 client.7776 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 10 client.7776 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 20 client.7776 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b476ef60 0x55f9b4bae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 4 client.7776 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b4acc7c0 con 0x55f9b476ef60 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.882+0000 7ff54fe00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b476ef60 0x55f9b4bae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.883+0000 7ff54fe00640 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b476ef60 0x55f9b4bae3c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31cf60b2046dbfd8 server_cookie=6ff5e2212fcbfba0 in_seq=0 out_seq=0 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.883+0000 7ff546000640 10 client.7776 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff546000640 1 -- 172.21.15.37:0/4291231983 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9b4acc7c0 con 0x55f9b476ef60 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff546000640 10 client.7776 handle_command_reply: tid=0 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 client.7776 shutdown 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 msgr2=0x55f9b4bab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff548063440 tx=0x7ff548062e70 comp rx=0 tx=0).stop 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 shutdown_connections 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b476ef60 0x55f9b4bae3c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47b0390 0x55f9b4b56a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b9b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff546000640 0 client.7776 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 --2- 172.21.15.37:0/4291231983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47874d0 0x55f9b4bab850 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 >> 172.21.15.37:0/4291231983 conn(0x55f9b47a44e0 msgr2=0x55f9b4b50830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 shutdown_connections 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.884+0000 7ff5667d8740 1 -- 172.21.15.37:0/4291231983 wait complete. 2024-09-21T06:34:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.885+0000 7ff5667d8740 20 client.7776 trim_cache size 0 max 16384 2024-09-21T06:34:16.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.903+0000 7ff5667d8740 1 Processor -- start 2024-09-21T06:34:16.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.903+0000 7ff5667d8740 1 -- start start 2024-09-21T06:34:16.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.904+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bbf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.904+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff5667d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5582029c0 con 0x55f9b47b0390 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff558201b90 con 0x55f9b47874d0 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff558202290 con 0x55f9b478f6b0 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57438/0 (socket says 172.21.15.37:57438) 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 -- 172.21.15.37:0/3949456706 learned_addr learned my addr 172.21.15.37:0/3949456706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff54fe00640 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55ce00640 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bbf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 -- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 msgr2=0x55f9b4b58370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b58370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 -- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 msgr2=0x55f9b4bbf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bbf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.905+0000 7ff55d800640 1 -- 172.21.15.37:0/3949456706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b4b51b30 con 0x55f9b47b0390 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.906+0000 7ff55d800640 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff550009cd0 tx=0x7ff55002a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0994363fc27e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.906+0000 7ff54cc00640 1 -- 172.21.15.37:0/3949456706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550054070 con 0x55f9b47b0390 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.906+0000 7ff54cc00640 1 -- 172.21.15.37:0/3949456706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff55004b040 con 0x55f9b47b0390 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.906+0000 7ff54cc00640 1 -- 172.21.15.37:0/3949456706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500318e0 con 0x55f9b47b0390 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 -- 172.21.15.37:0/3949456706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 msgr2=0x55f9b4b588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff550009cd0 tx=0x7ff55002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 -- 172.21.15.37:0/3949456706 shutdown_connections 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4b588b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4b58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3949456706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bbf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 -- 172.21.15.37:0/3949456706 >> 172.21.15.37:0/3949456706 conn(0x55f9b47a44e0 msgr2=0x55f9b4b6f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 -- 172.21.15.37:0/3949456706 shutdown_connections 2024-09-21T06:34:16.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.907+0000 7ff5667d8740 1 -- 172.21.15.37:0/3949456706 wait complete. 2024-09-21T06:34:16.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.908+0000 7ff5667d8740 1 -- start start 2024-09-21T06:34:16.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.908+0000 7ff5667d8740 1 Processor -- start 2024-09-21T06:34:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.909+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bb3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.909+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4baa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.909+0000 7ff5667d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff5667d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff558200c10 con 0x55f9b47b0390 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4a80ee0 con 0x55f9b47874d0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff5667d8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f9b4c1f2e0 con 0x55f9b478f6b0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff5667d8740 10 client.0 _ll_register_callbacks cb 0x7ff548009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57448/0 (socket says 172.21.15.37:57448) 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 -- 172.21.15.37:0/3887038459 learned_addr learned my addr 172.21.15.37:0/3887038459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55ce00640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bb3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff54fe00640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4baa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 -- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 msgr2=0x55f9b4baa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4baa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 -- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 msgr2=0x55f9b4bb3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bb3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.910+0000 7ff55d800640 1 -- 172.21.15.37:0/3887038459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b4b9bb60 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff55d800640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff550009bb0 tx=0x7ff550004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51663be7d8366328 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550054040 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff5500039f0 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55004b040 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7ff550064050 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 10 client.7782 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55f9b4b9ae40 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff55004b040 con 0x55f9b47b0390 2024-09-21T06:34:16.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7ff550057070 con 0x55f9b47b0390 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 10 client.7782 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 10 client.7782 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 20 client.7782 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.911+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b4bbf260 0x55f9b4b94140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.912+0000 7ff5667d8740 4 client.7782 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.912+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9b4ac8270 con 0x55f9b4bbf260 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.912+0000 7ff55ce00640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b4bbf260 0x55f9b4b94140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff55ce00640 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b4bbf260 0x55f9b4b94140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c60ce7f65a714abf server_cookie=e7561c64ef6fa3a0 in_seq=0 out_seq=0 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff546000640 10 client.7782 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff546000640 1 -- 172.21.15.37:0/3887038459 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f9b4ac8270 con 0x55f9b4bbf260 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff546000640 10 client.7782 handle_command_reply: tid=0 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 client.7782 shutdown 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 msgr2=0x55f9b4baa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff550009bb0 tx=0x7ff550004f80 comp rx=0 tx=0).stop 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 shutdown_connections 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55f9b4bbf260 0x55f9b4b94140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9b47b0390 0x55f9b4baa7e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9b478f6b0 0x55f9b4baa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 --2- 172.21.15.37:0/3887038459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9b47874d0 0x55f9b4bb3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff546000640 0 client.7782 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 >> 172.21.15.37:0/3887038459 conn(0x55f9b47a44e0 msgr2=0x55f9b4b52680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 shutdown_connections 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.913+0000 7ff5667d8740 1 -- 172.21.15.37:0/3887038459 wait complete. 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff5667d8740 20 client.7782 trim_cache size 0 max 16384 2024-09-21T06:34:16.918 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:16.919 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 msgr2=0x7ff53407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 0x7ff53407b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff5580b15e0 tx=0x7ff540009040 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 msgr2=0x7ff5580a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e790 tx=0x7ff54800ec50 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 shutdown_connections 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff534078b90 0x7ff53407b0a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55815fc50 0x7ff5581f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5580cf9a0 0x7ff5581f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 --2- 172.21.15.37:0/2567685350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff55806e330 0x7ff5580a0520 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 >> 172.21.15.37:0/2567685350 conn(0x7ff5580b1b90 msgr2=0x7ff55806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.914+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 shutdown_connections 2024-09-21T06:34:16.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:16.915+0000 7ff55f800640 1 -- 172.21.15.37:0/2567685350 wait complete. 2024-09-21T06:34:16.939 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:16.939 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:16.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T06:34:17.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 -- 172.21.15.37:0/2898981309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 msgr2=0x7f9cac074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/2898981309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac074620 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005fd0 tx=0x7f9ca002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 -- 172.21.15.37:0/2898981309 shutdown_connections 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/2898981309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac074620 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 -- 172.21.15.37:0/2898981309 >> 172.21.15.37:0/2898981309 conn(0x7f9cac0b1b90 msgr2=0x7f9cac0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 -- 172.21.15.37:0/2898981309 shutdown_connections 2024-09-21T06:34:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.166+0000 7f9cb3c00640 1 -- 172.21.15.37:0/2898981309 wait complete. 2024-09-21T06:34:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.169+0000 7f9cb3c00640 1 Processor -- start 2024-09-21T06:34:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.169+0000 7f9cb3c00640 1 -- start start 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 0x7f9cac1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cac0c5650 0x7f9cac0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac0cf790 con 0x7f9cac156a50 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac0cf610 con 0x7f9cac074b60 2024-09-21T06:34:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cb3c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac0cf910 con 0x7f9cac0c5650 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.170+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57468/0 (socket says 172.21.15.37:57468) 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 -- 172.21.15.37:0/955639872 learned_addr learned my addr 172.21.15.37:0/955639872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cb1200640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 0x7f9cac1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cac0c5650 0x7f9cac0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 -- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cac0c5650 msgr2=0x7f9cac0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cac0c5650 0x7f9cac0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 -- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 msgr2=0x7f9cac1f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 0x7f9cac1f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 -- 172.21.15.37:0/955639872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac06cff0 con 0x7f9cac156a50 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cb1200640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 0x7f9cac1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.171+0000 7f9cabe00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b3e0 tx=0x7f9c9800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.172+0000 7f9ca8c00640 1 -- 172.21.15.37:0/955639872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98015020 con 0x7f9cac156a50 2024-09-21T06:34:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.172+0000 7f9ca8c00640 1 -- 172.21.15.37:0/955639872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c98011920 con 0x7f9cac156a50 2024-09-21T06:34:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.172+0000 7f9ca8c00640 1 -- 172.21.15.37:0/955639872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98011d70 con 0x7f9cac156a50 2024-09-21T06:34:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.172+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac0c3330 con 0x7f9cac156a50 2024-09-21T06:34:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.172+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac0c3040 con 0x7f9cac156a50 2024-09-21T06:34:17.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.174+0000 7f9ca8c00640 1 -- 172.21.15.37:0/955639872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c98018020 con 0x7f9cac156a50 2024-09-21T06:34:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.176+0000 7f9ca8c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 0x7f9c8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.176+0000 7f9ca8c00640 1 -- 172.21.15.37:0/955639872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9c9809af60 con 0x7f9cac156a50 2024-09-21T06:34:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.176+0000 7f9cb1200640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 0x7f9c8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.176+0000 7f9cbadc3740 1 Processor -- start 2024-09-21T06:34:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.177+0000 7f9cbadc3740 1 -- start start 2024-09-21T06:34:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.177+0000 7f9cb1200640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 0x7f9c8c07b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0033000 tx=0x7f9ca0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300c16b0 0x55a8304b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a8303e91c0 con 0x55a8300b94d0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a8303fc870 con 0x55a8300c16b0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a8303ebca0 con 0x55a8300e2390 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57484/0 (socket says 172.21.15.37:57484) 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cabe00640 1 -- 172.21.15.37:0/3090439681 learned_addr learned my addr 172.21.15.37:0/3090439681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.178+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300c16b0 0x55a8304b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 -- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 msgr2=0x55a8304d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 -- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300c16b0 msgr2=0x55a8304b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300c16b0 0x55a8304b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 -- 172.21.15.37:0/3090439681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8301c47d0 con 0x55a8300b94d0 2024-09-21T06:34:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9c9805fce0 tx=0x7f9c98062950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ce92e9aae5985c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9c9d600640 1 -- 172.21.15.37:0/3090439681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98067550 con 0x55a8300b94d0 2024-09-21T06:34:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9c9d600640 1 -- 172.21.15.37:0/3090439681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c9806c490 con 0x55a8300b94d0 2024-09-21T06:34:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.179+0000 7f9c9d600640 1 -- 172.21.15.37:0/3090439681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98061cd0 con 0x55a8300b94d0 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3090439681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 msgr2=0x55a8304b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9c9805fce0 tx=0x7f9c98062950 comp rx=0 tx=0).stop 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3090439681 shutdown_connections 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304b3220 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300c16b0 0x55a8304b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3090439681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3090439681 >> 172.21.15.37:0/3090439681 conn(0x55a8300d64e0 msgr2=0x55a830489440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3090439681 shutdown_connections 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.180+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3090439681 wait complete. 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.181+0000 7f9cbadc3740 1 -- start start 2024-09-21T06:34:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.182+0000 7f9cbadc3740 1 Processor -- start 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300e2390 0x55a8304a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a830392410 con 0x55a8300b94d0 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a8303efab0 con 0x55a8300e2390 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a83015c740 con 0x55a8300c16b0 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cbadc3740 10 client.0 _ll_register_callbacks cb 0x55a83036f360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cb1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.183+0000 7f9cb1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57498/0 (socket says 172.21.15.37:57498) 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 -- 172.21.15.37:0/3906394820 learned_addr learned my addr 172.21.15.37:0/3906394820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300e2390 0x55a8304a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 -- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 msgr2=0x55a8304c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 -- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300e2390 msgr2=0x55a8304a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300e2390 0x55a8304a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 -- 172.21.15.37:0/3906394820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83014d970 con 0x55a8300b94d0 2024-09-21T06:34:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63397af918c75e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.184+0000 7f9c92000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca004d070 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9ca00049a0 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9806c490 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f9ca0058050 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 10 client.7797 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55a83012b6a0 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c9806c490 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f9ca005b020 con 0x55a8300b94d0 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 10 client.7797 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 10 client.7797 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 20 client.7797 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8300a0f60 0x55a8304ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 4 client.7797 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.185+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8303fe7c0 con 0x55a8300a0f60 2024-09-21T06:34:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.186+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8300a0f60 0x55a8304ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.186+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8300a0f60 0x55a8304ddd40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10338f74115025cd server_cookie=90ac52447feb353e in_seq=0 out_seq=0 2024-09-21T06:34:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.186+0000 7f9c92000640 10 client.7797 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.187+0000 7f9c92000640 1 -- 172.21.15.37:0/3906394820 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8303fe7c0 con 0x55a8300a0f60 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.187+0000 7f9c92000640 10 client.7797 handle_command_reply: tid=0 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.187+0000 7f9cbadc3740 1 client.7797 shutdown 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 msgr2=0x55a8304c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca0009260 comp rx=0 tx=0).stop 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 shutdown_connections 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8300a0f60 0x55a8304ddd40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300e2390 0x55a8304a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9c92000640 0 client.7797 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3906394820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300b94d0 0x55a8304c85f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 >> 172.21.15.37:0/3906394820 conn(0x55a8300d64e0 msgr2=0x55a8304a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 shutdown_connections 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3906394820 wait complete. 2024-09-21T06:34:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.188+0000 7f9cbadc3740 20 client.7797 trim_cache size 0 max 16384 2024-09-21T06:34:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.207+0000 7f9cbadc3740 1 Processor -- start 2024-09-21T06:34:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.207+0000 7f9cbadc3740 1 -- start start 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac2027b0 con 0x55a8300c16b0 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac201980 con 0x55a8300b94d0 2024-09-21T06:34:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac202080 con 0x55a8300e2390 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57508/0 (socket says 172.21.15.37:57508) 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cabe00640 1 -- 172.21.15.37:0/2835307341 learned_addr learned my addr 172.21.15.37:0/2835307341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.208+0000 7f9cb1200640 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 -- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 msgr2=0x55a8304aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 -- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 msgr2=0x55a8304a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 -- 172.21.15.37:0/2835307341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8304b0580 con 0x55a8300c16b0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9cabe00640 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bfd0 tx=0x7f9c98065070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb86d7fe158da39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9c9cc00640 1 -- 172.21.15.37:0/2835307341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98061cd0 con 0x55a8300c16b0 2024-09-21T06:34:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9c9cc00640 1 -- 172.21.15.37:0/2835307341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c98061e70 con 0x55a8300c16b0 2024-09-21T06:34:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.209+0000 7f9c9cc00640 1 -- 172.21.15.37:0/2835307341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9805e620 con 0x55a8300c16b0 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 -- 172.21.15.37:0/2835307341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 msgr2=0x55a8304a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bfd0 tx=0x7f9c98065070 comp rx=0 tx=0).stop 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 -- 172.21.15.37:0/2835307341 shutdown_connections 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300c16b0 0x55a8304a14d0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300e2390 0x55a8304aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/2835307341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 -- 172.21.15.37:0/2835307341 >> 172.21.15.37:0/2835307341 conn(0x55a8300d64e0 msgr2=0x55a8304ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 -- 172.21.15.37:0/2835307341 shutdown_connections 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.210+0000 7f9cbadc3740 1 -- 172.21.15.37:0/2835307341 wait complete. 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.211+0000 7f9cbadc3740 1 -- start start 2024-09-21T06:34:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.211+0000 7f9cbadc3740 1 Processor -- start 2024-09-21T06:34:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a8304d0fa0 con 0x55a8300e2390 2024-09-21T06:34:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1f9730 con 0x55a8300b94d0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a8303fe7c0 con 0x55a8300c16b0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cbadc3740 10 client.0 _ll_register_callbacks cb 0x7f9ca0004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57514/0 (socket says 172.21.15.37:57514) 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 -- 172.21.15.37:0/3450124471 learned_addr learned my addr 172.21.15.37:0/3450124471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cabe00640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 -- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 msgr2=0x55a8304a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 -- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 msgr2=0x55a8304d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.213+0000 7f9cb1c00640 1 -- 172.21.15.37:0/3450124471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a830485a20 con 0x55a8300e2390 2024-09-21T06:34:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9cb1c00640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40012e0 tx=0x7f9ca4005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c378ee7d63024020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9c92000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4017700 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9ca40178a0 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c98061e70 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f9ca4027050 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9cbadc3740 10 client.7803 fetch_fsmap learned FSMap version 96 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.214+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55a830489810 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c98061e70 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 96) ==== 3554+0+0 (secure 0 0 0) 0x7f9ca402a020 con 0x55a8300e2390 2024-09-21T06:34:17.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 10 client.7803 fetch_fsmap finished waiting for FSMap version 96 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 20 client.7803 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8304a14d0 0x55a83054cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 4 client.7803 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8303fa270 con 0x55a8304a14d0 2024-09-21T06:34:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.215+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8304a14d0 0x55a83054cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9cb1200640 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8304a14d0 0x55a83054cf60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819218063688542f server_cookie=6822f675fe3b532c in_seq=0 out_seq=0 2024-09-21T06:34:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9c92000640 10 client.7803 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9c92000640 1 -- 172.21.15.37:0/3450124471 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x55a8303fa270 con 0x55a8304a14d0 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9c92000640 10 client.7803 handle_command_reply: tid=0 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9cbadc3740 1 client.7803 shutdown 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 msgr2=0x55a8304a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.216+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40012e0 tx=0x7f9ca4005420 comp rx=0 tx=0).stop 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 shutdown_connections 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x55a8304a14d0 0x55a83054cf60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8300e2390 0x55a8304a0f90 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8300c16b0 0x55a8304a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9c92000640 0 client.7803 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 --2- 172.21.15.37:0/3450124471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8300b94d0 0x55a8304d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 >> 172.21.15.37:0/3450124471 conn(0x55a8300d64e0 msgr2=0x55a8304cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 shutdown_connections 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 1 -- 172.21.15.37:0/3450124471 wait complete. 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cbadc3740 20 client.7803 trim_cache size 0 max 16384 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:34:15.993402+0000", 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:17.224 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 msgr2=0x7f9c8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 0x7f9c8c07b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0033000 tx=0x7f9ca0004730 comp rx=0 tx=0).stop 2024-09-21T06:34:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 msgr2=0x7f9cac0ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.217+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b3e0 tx=0x7f9c9800b8a0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 shutdown_connections 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9c8c078b90 0x7f9c8c07b0a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cac156a50 0x7f9cac0ca6f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cac0c5650 0x7f9cac0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 --2- 172.21.15.37:0/955639872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cac074b60 0x7f9cac1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 >> 172.21.15.37:0/955639872 conn(0x7f9cac0b1b90 msgr2=0x7f9cac0c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 shutdown_connections 2024-09-21T06:34:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.218+0000 7f9cb3c00640 1 -- 172.21.15.37:0/955639872 wait complete. 2024-09-21T06:34:17.244 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:17.245 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:17.254 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:34:17.252+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:34:17.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 -- 172.21.15.37:0/1698810779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 msgr2=0x7fcea015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 --2- 172.21.15.37:0/1698810779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea015e5e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fce98005fd0 tx=0x7fce9802a820 comp rx=0 tx=0).stop 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 -- 172.21.15.37:0/1698810779 shutdown_connections 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 --2- 172.21.15.37:0/1698810779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea015e5e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 -- 172.21.15.37:0/1698810779 >> 172.21.15.37:0/1698810779 conn(0x7fcea00d4360 msgr2=0x7fcea00d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 -- 172.21.15.37:0/1698810779 shutdown_connections 2024-09-21T06:34:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.470+0000 7fcea5800640 1 -- 172.21.15.37:0/1698810779 wait complete. 2024-09-21T06:34:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.472+0000 7fcea5800640 1 Processor -- start 2024-09-21T06:34:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.473+0000 7fcea5800640 1 -- start start 2024-09-21T06:34:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea00ceff0 0x7fcea0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea0156a70 0x7fcea01f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcea00b8f80 con 0x7fcea00d2520 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcea00b8e00 con 0x7fcea00ceff0 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fcea5800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcea00b9100 con 0x7fcea0156a70 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fce97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fce97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57532/0 (socket says 172.21.15.37:57532) 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fce97400640 1 -- 172.21.15.37:0/3705382399 learned_addr learned my addr 172.21.15.37:0/3705382399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.474+0000 7fce9fe00640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea00ceff0 0x7fcea0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fcea4e00640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea0156a70 0x7fcea01f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 -- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea0156a70 msgr2=0x7fcea01f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea0156a70 0x7fcea01f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 -- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea00ceff0 msgr2=0x7fcea0155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea00ceff0 0x7fcea0155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 -- 172.21.15.37:0/3705382399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea0156310 con 0x7fcea00d2520 2024-09-21T06:34:17.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce97400640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c370 tx=0x7fce8800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88017070 con 0x7fcea00d2520 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce880035f0 con 0x7fcea00d2520 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.475+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea00cdb50 con 0x7fcea00d2520 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.476+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88003a40 con 0x7fcea00d2520 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.476+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea00cddf0 con 0x7fcea00d2520 2024-09-21T06:34:17.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.477+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea015e5e0 con 0x7fcea00d2520 2024-09-21T06:34:17.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.478+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce8801a070 con 0x7fcea00d2520 2024-09-21T06:34:17.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.479+0000 7fce9cc00640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 0x7fce7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:17.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.479+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fce8809a0c0 con 0x7fcea00d2520 2024-09-21T06:34:17.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.480+0000 7fce9fe00640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 0x7fce7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:17.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.480+0000 7fce9fe00640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 0x7fce7c07b0a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fce90000f60 tx=0x7fce90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:17.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.484+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce88061da0 con 0x7fcea00d2520 2024-09-21T06:34:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.721+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcea00b55e0 con 0x7fcea00d2520 2024-09-21T06:34:17.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.722+0000 7fce9cc00640 1 -- 172.21.15.37:0/3705382399 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5273 (secure 0 0 0) 0x7fce880667d0 con 0x7fcea00d2520 2024-09-21T06:34:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:17.724 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":97,"btime":"2024-09-21T06:34:17:254110+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:17.131226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:17.724 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 97 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 msgr2=0x7fce7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 0x7fce7c07b0a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fce90000f60 tx=0x7fce90009040 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 msgr2=0x7fcea01f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c370 tx=0x7fce8800c830 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 shutdown_connections 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fce7c078b90 0x7fce7c07b0a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcea00d2520 0x7fcea01f89c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea0156a70 0x7fcea01f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 --2- 172.21.15.37:0/3705382399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea00ceff0 0x7fcea0155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 >> 172.21.15.37:0/3705382399 conn(0x7fcea00d4360 msgr2=0x7fcea015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 shutdown_connections 2024-09-21T06:34:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:17.724+0000 7fcea5800640 1 -- 172.21.15.37:0/3705382399 wait complete. 2024-09-21T06:34:17.750 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:17.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 -- 172.21.15.37:0/2975193501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0074d90 msgr2=0x7fc9c006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 --2- 172.21.15.37:0/2975193501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0074d90 0x7fc9c006f710 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000a940 tx=0x7fc9b002b8f0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 -- 172.21.15.37:0/2975193501 shutdown_connections 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 --2- 172.21.15.37:0/2975193501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0074d90 0x7fc9c006f710 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 -- 172.21.15.37:0/2975193501 >> 172.21.15.37:0/2975193501 conn(0x7fc9c00cca40 msgr2=0x7fc9c00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 -- 172.21.15.37:0/2975193501 shutdown_connections 2024-09-21T06:34:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.013+0000 7fc9c6200640 1 -- 172.21.15.37:0/2975193501 wait complete. 2024-09-21T06:34:18.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.016+0000 7fc9c6200640 1 Processor -- start 2024-09-21T06:34:18.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.016+0000 7fc9c6200640 1 -- start start 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c0074d90 0x7fc9c00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 0x7fc9c00736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0071000 con 0x7fc9c0070030 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0070e80 con 0x7fc9c0074d90 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9c6200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c00ce2a0 con 0x7fc9c01f9250 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57538/0 (socket says 172.21.15.37:57538) 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.017+0000 7fc9bf400640 1 -- 172.21.15.37:0/1345262483 learned_addr learned my addr 172.21.15.37:0/1345262483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bea00640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c0074d90 0x7fc9c00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:18.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bfe00640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 0x7fc9c00736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 -- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 msgr2=0x7fc9c00736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 0x7fc9c00736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 -- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c0074d90 msgr2=0x7fc9c00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c0074d90 0x7fc9c00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 -- 172.21.15.37:0/1345262483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c00c99c0 con 0x7fc9c0070030 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bfe00640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 0x7fc9c00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9bf400640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000aa60 tx=0x7fc9b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.018+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0005220 con 0x7fc9c0070030 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.019+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9b002e070 con 0x7fc9c0070030 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.019+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c00b47d0 con 0x7fc9c0070030 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.019+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00047a0 con 0x7fc9c0070030 2024-09-21T06:34:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.019+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c00b4d90 con 0x7fc9c0070030 2024-09-21T06:34:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.020+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c006f710 con 0x7fc9c0070030 2024-09-21T06:34:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.021+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9b0004940 con 0x7fc9c0070030 2024-09-21T06:34:18.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.022+0000 7fc9b7400640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:18.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.022+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc9b00bc810 con 0x7fc9c0070030 2024-09-21T06:34:18.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.022+0000 7fc9bea00640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:18.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.023+0000 7fc9bea00640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 0x7fc9a007b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0072c50 tx=0x7fc9ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.026+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc9b0084a20 con 0x7fc9c0070030 2024-09-21T06:34:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.305+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc9c00b5f40 con 0x7fc9c0070030 2024-09-21T06:34:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.305+0000 7fc9b7400640 1 -- 172.21.15.37:0/1345262483 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1479 (secure 0 0 0) 0x7fc9b0084320 con 0x7fc9c0070030 2024-09-21T06:34:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:18.308 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":406,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":62,"data_bytes":668443,"bytes_used":265850880,"bytes_avail":772828262400,"bytes_total":773094113280,"write_bytes_sec":4777,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":97,"btime":"2024-09-21T06:34:17:254110+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-21T06:33:33.181284+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.307+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 msgr2=0x7fc9a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 0x7fc9a007b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0072c50 tx=0x7fc9ac009040 comp rx=0 tx=0).stop 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 msgr2=0x7fc9c00b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000aa60 tx=0x7fc9b0004270 comp rx=0 tx=0).stop 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 shutdown_connections 2024-09-21T06:34:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c01f9250 0x7fc9c00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c0074d90 0x7fc9c00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 --2- 172.21.15.37:0/1345262483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9c0070030 0x7fc9c00b5870 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 >> 172.21.15.37:0/1345262483 conn(0x7fc9c00cca40 msgr2=0x7fc9c00b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 shutdown_connections 2024-09-21T06:34:18.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:18.308+0000 7fc9c6200640 1 -- 172.21.15.37:0/1345262483 wait complete. 2024-09-21T06:34:18.335 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-21T06:34:19.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 -- 172.21.15.37:0/4289798276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c003f90 msgr2=0x7f1a1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 --2- 172.21.15.37:0/4289798276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c003f90 0x7f1a1c004800 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009b50 tx=0x7f1a1c005830 comp rx=0 tx=0).stop 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a37e00640 1 -- 172.21.15.37:0/4289798276 reap_dead start 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 -- 172.21.15.37:0/4289798276 shutdown_connections 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 -- 172.21.15.37:0/4289798276 >> 172.21.15.37:0/4289798276 conn(0x7f1a380cca40 msgr2=0x7f1a380cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 -- 172.21.15.37:0/4289798276 shutdown_connections 2024-09-21T06:34:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.572+0000 7f1a3e600640 1 -- 172.21.15.37:0/4289798276 wait complete. 2024-09-21T06:34:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.575+0000 7f1a3e600640 1 Processor -- start 2024-09-21T06:34:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.575+0000 7f1a3e600640 1 -- start start 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a1c003f90 0x7f1a380b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a3815ae70 0x7f1a380b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3806df70 con 0x7f1a380b2e10 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3806ddf0 con 0x7f1a3815ae70 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a3e600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3806e0f0 con 0x7f1a1c003f90 2024-09-21T06:34:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57556/0 (socket says 172.21.15.37:57556) 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.576+0000 7f1a37e00640 1 -- 172.21.15.37:0/1783776288 learned_addr learned my addr 172.21.15.37:0/1783776288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37400640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a1c003f90 0x7f1a380b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a2fe00640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a3815ae70 0x7f1a380b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 -- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a1c003f90 msgr2=0x7f1a380b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a1c003f90 0x7f1a380b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 -- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a3815ae70 msgr2=0x7f1a380b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a3815ae70 0x7f1a380b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 -- 172.21.15.37:0/1783776288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380b3760 con 0x7f1a380b2e10 2024-09-21T06:34:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a37e00640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1a300016f0 tx=0x7f1a30006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30011900 con 0x7f1a380b2e10 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.577+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a30017070 con 0x7f1a380b2e10 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.578+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a380b5220 con 0x7f1a380b2e10 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.578+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30011cc0 con 0x7f1a380b2e10 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.578+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a380cdff0 con 0x7f1a380b2e10 2024-09-21T06:34:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.579+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3806f790 con 0x7f1a380b2e10 2024-09-21T06:34:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.580+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a3001c460 con 0x7f1a380b2e10 2024-09-21T06:34:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.581+0000 7f1a2f400640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 0x7f1a1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.581+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f1a3009b890 con 0x7f1a380b2e10 2024-09-21T06:34:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.582+0000 7f1a37400640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 0x7f1a1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:19.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.582+0000 7f1a37400640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 0x7f1a1407b0a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a3815a730 tx=0x7f1a28001e70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:19.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.585+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a380b3760 con 0x7f1a380b2e10 2024-09-21T06:34:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.820+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a3815b4a0 con 0x7f1a380b2e10 2024-09-21T06:34:19.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.821+0000 7f1a2f400640 1 -- 172.21.15.37:0/1783776288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5274 (secure 0 0 0) 0x7f1a3806e740 con 0x7f1a380b2e10 2024-09-21T06:34:19.823 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:19.823 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"btime":"2024-09-21T06:34:19:264979+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:19.087701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7096,7389,7392]},"id":6}]} 2024-09-21T06:34:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 98 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.823+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 msgr2=0x7f1a1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.823+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 0x7f1a1407b0a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a3815a730 tx=0x7f1a28001e70 comp rx=0 tx=0).stop 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.823+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 msgr2=0x7f1a380b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.823+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1a300016f0 tx=0x7f1a30006430 comp rx=0 tx=0).stop 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 shutdown_connections 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1a14078b90 0x7f1a1407b0a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a380b2e10 0x7f1a380b3220 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a3815ae70 0x7f1a380b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 --2- 172.21.15.37:0/1783776288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a1c003f90 0x7f1a380b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 >> 172.21.15.37:0/1783776288 conn(0x7f1a380cca40 msgr2=0x7f1a3815a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 shutdown_connections 2024-09-21T06:34:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:19.824+0000 7f1a3e600640 1 -- 172.21.15.37:0/1783776288 wait complete. 2024-09-21T06:34:19.852 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:19.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.076+0000 7f742b200640 1 -- 172.21.15.37:0/4294618180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c0b9e40 msgr2=0x7f742c1622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.076+0000 7f742b200640 1 --2- 172.21.15.37:0/4294618180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c0b9e40 0x7f742c1622b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f742c15c800 tx=0x7f741c02a580 comp rx=0 tx=0).stop 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 -- 172.21.15.37:0/4294618180 shutdown_connections 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 --2- 172.21.15.37:0/4294618180 >> v2:172.21.15.150:3300/0 conn(0x7f742c06d7f0 0x7f742c154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 --2- 172.21.15.37:0/4294618180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c1627f0 0x7f742c06d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 --2- 172.21.15.37:0/4294618180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c0b9e40 0x7f742c1622b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 -- 172.21.15.37:0/4294618180 >> 172.21.15.37:0/4294618180 conn(0x7f742c0d1930 msgr2=0x7f742c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 -- 172.21.15.37:0/4294618180 shutdown_connections 2024-09-21T06:34:20.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.077+0000 7f742b200640 1 -- 172.21.15.37:0/4294618180 wait complete. 2024-09-21T06:34:20.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.079+0000 7f742b200640 1 Processor -- start 2024-09-21T06:34:20.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.079+0000 7f742b200640 1 -- start start 2024-09-21T06:34:20.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.080+0000 7f742b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c06d7f0 0x7f742c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.080+0000 7f742b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f742c0b9e40 0x7f742c1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f742c155770 con 0x7f742c1627f0 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1555f0 con 0x7f742c0b9e40 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f742c1558f0 con 0x7f742c06d7f0 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57574/0 (socket says 172.21.15.37:57574) 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 -- 172.21.15.37:0/626826125 learned_addr learned my addr 172.21.15.37:0/626826125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f7429e00640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c06d7f0 0x7f742c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f7423e00640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f742c0b9e40 0x7f742c1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 -- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c06d7f0 msgr2=0x7f742c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c06d7f0 0x7f742c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 -- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f742c0b9e40 msgr2=0x7f742c1f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f742c0b9e40 0x7f742c1f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.081+0000 7f742a800640 1 -- 172.21.15.37:0/626826125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1f6950 con 0x7f742c1627f0 2024-09-21T06:34:20.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f742a800640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9e0 tx=0x7f742400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424019070 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f742400cc30 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240146a0 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c15ed00 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.082+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1585c0 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.083+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c1622b0 con 0x7f742c1627f0 2024-09-21T06:34:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.084+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f742401c070 con 0x7f742c1627f0 2024-09-21T06:34:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.085+0000 7f7422000640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 0x7f740807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.085+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f742409abf0 con 0x7f742c1627f0 2024-09-21T06:34:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.086+0000 7f7429e00640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 0x7f740807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.087+0000 7f7429e00640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 0x7f740807b0a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741c033000 tx=0x7f741c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.090+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f742c1f6950 con 0x7f742c1627f0 2024-09-21T06:34:20.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.362+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f742c160d50 con 0x7f742c1627f0 2024-09-21T06:34:20.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.363+0000 7f7422000640 1 -- 172.21.15.37:0/626826125 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2014 (secure 0 0 0) 0x7f742400ce20 con 0x7f742c1627f0 2024-09-21T06:34:20.365 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:20.365 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":408,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":265928704,"bytes_avail":772828184576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9810,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-09-21T06:34:19.193583+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:20.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 msgr2=0x7f740807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 0x7f740807b0a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741c033000 tx=0x7f741c004730 comp rx=0 tx=0).stop 2024-09-21T06:34:20.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 msgr2=0x7f742c0c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9e0 tx=0x7f742400bea0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 shutdown_connections 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7408078b90 0x7f740807b0a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742c1627f0 0x7f742c0c8600 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f742c0b9e40 0x7f742c1f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 --2- 172.21.15.37:0/626826125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f742c06d7f0 0x7f742c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 >> 172.21.15.37:0/626826125 conn(0x7f742c0d1930 msgr2=0x7f742c0b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 shutdown_connections 2024-09-21T06:34:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.365+0000 7f742b200640 1 -- 172.21.15.37:0/626826125 wait complete. 2024-09-21T06:34:20.393 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:34:20.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:20.394 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub resume 2024-09-21T06:34:20.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:34:20.532 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:34:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 -- 172.21.15.37:0/4012722556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00af5b0 msgr2=0x7f95f015ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 --2- 172.21.15.37:0/4012722556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00af5b0 0x7f95f015ead0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009be0 tx=0x7f95e802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 -- 172.21.15.37:0/4012722556 shutdown_connections 2024-09-21T06:34:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 --2- 172.21.15.37:0/4012722556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00af5b0 0x7f95f015ead0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 -- 172.21.15.37:0/4012722556 >> 172.21.15.37:0/4012722556 conn(0x7f95f00b1b90 msgr2=0x7f95f00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 -- 172.21.15.37:0/4012722556 shutdown_connections 2024-09-21T06:34:20.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.623+0000 7f95f6800640 1 -- 172.21.15.37:0/4012722556 wait complete. 2024-09-21T06:34:20.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.625+0000 7f95f6800640 1 Processor -- start 2024-09-21T06:34:20.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.625+0000 7f95f6800640 1 -- start start 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.626+0000 7f95f6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f00af5b0 0x7f95f01f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.626+0000 7f95f6800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00cc530 0x7f95f01e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.626+0000 7f95f6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95f6800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00cefb0 con 0x7f95f015df80 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95f6800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00cee30 con 0x7f95f00af5b0 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95f6800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f95f00cf130 con 0x7f95f00cc530 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57582/0 (socket says 172.21.15.37:57582) 2024-09-21T06:34:20.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 -- 172.21.15.37:0/1157152879 learned_addr learned my addr 172.21.15.37:0/1157152879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95e6a00640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00cc530 0x7f95f01e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95ef400640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f00af5b0 0x7f95f01f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 -- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00cc530 msgr2=0x7f95f01e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00cc530 0x7f95f01e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 -- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f00af5b0 msgr2=0x7f95f01f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f00af5b0 0x7f95f01f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.627+0000 7f95efe00640 1 -- 172.21.15.37:0/1157152879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f0161b50 con 0x7f95f015df80 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95efe00640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009d00 tx=0x7f95e8003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95e7e00640 1 -- 172.21.15.37:0/1157152879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8042020 con 0x7f95f015df80 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95e7e00640 1 -- 172.21.15.37:0/1157152879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95e8033b00 con 0x7f95f015df80 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01f88a0 con 0x7f95f015df80 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95e7e00640 1 -- 172.21.15.37:0/1157152879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8037040 con 0x7f95f015df80 2024-09-21T06:34:20.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.628+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01f8e60 con 0x7f95f015df80 2024-09-21T06:34:20.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.630+0000 7f95e7e00640 1 -- 172.21.15.37:0/1157152879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95e8045020 con 0x7f95f015df80 2024-09-21T06:34:20.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.631+0000 7f95e7e00640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 0x7f95cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.632+0000 7f95e7e00640 1 -- 172.21.15.37:0/1157152879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f95e80bb960 con 0x7f95f015df80 2024-09-21T06:34:20.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.632+0000 7f95ef400640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 0x7f95cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.632+0000 7f95fd811740 1 Processor -- start 2024-09-21T06:34:20.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.633+0000 7f95fd811740 1 -- start start 2024-09-21T06:34:20.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.633+0000 7f95ef400640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 0x7f95cc07b0d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f95e0001120 tx=0x7f95e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bb0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027be6b0 0x561502bb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561502ae61c0 con 0x5615027df390 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561502af9870 con 0x5615027be6b0 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561502ae8ca0 con 0x5615027b64d0 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57590/0 (socket says 172.21.15.37:57590) 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95ef400640 1 -- 172.21.15.37:0/3110675889 learned_addr learned my addr 172.21.15.37:0/3110675889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95e6a00640 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bb0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.634+0000 7f95efe00640 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027be6b0 0x561502bb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 -- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 msgr2=0x561502bb0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bb0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 -- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027be6b0 msgr2=0x561502bb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027be6b0 0x561502bb0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 -- 172.21.15.37:0/3110675889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615028c17d0 con 0x5615027df390 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95ef400640 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95e000f920 tx=0x7f95e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c513ceb61a9d5ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95dfe00640 1 -- 172.21.15.37:0/3110675889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0025070 con 0x5615027df390 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95dfe00640 1 -- 172.21.15.37:0/3110675889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95e001b050 con 0x5615027df390 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.635+0000 7f95dfe00640 1 -- 172.21.15.37:0/3110675889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0014e40 con 0x5615027df390 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 -- 172.21.15.37:0/3110675889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 msgr2=0x561502bcd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95e000f920 tx=0x7f95e0012000 comp rx=0 tx=0).stop 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 -- 172.21.15.37:0/3110675889 shutdown_connections 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027be6b0 0x561502bb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bb0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 --2- 172.21.15.37:0/3110675889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027df390 0x561502bcd550 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 -- 172.21.15.37:0/3110675889 >> 172.21.15.37:0/3110675889 conn(0x5615027d34e0 msgr2=0x561502b86440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 -- 172.21.15.37:0/3110675889 shutdown_connections 2024-09-21T06:34:20.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.636+0000 7f95fd811740 1 -- 172.21.15.37:0/3110675889 wait complete. 2024-09-21T06:34:20.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.637+0000 7f95fd811740 1 -- start start 2024-09-21T06:34:20.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.638+0000 7f95fd811740 1 Processor -- start 2024-09-21T06:34:20.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027df390 0x561502bc6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561502a8f410 con 0x5615027be6b0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561502aecab0 con 0x5615027df390 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561502859740 con 0x5615027b64d0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95fd811740 10 client.0 _ll_register_callbacks cb 0x561502a6c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95e6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95e6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57598/0 (socket says 172.21.15.37:57598) 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.639+0000 7f95e6a00640 1 -- 172.21.15.37:0/2759307215 learned_addr learned my addr 172.21.15.37:0/2759307215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95ef400640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95efe00640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027df390 0x561502bc6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 -- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 msgr2=0x561502bc4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bc4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 -- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027df390 msgr2=0x561502bc6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027df390 0x561502bc6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 -- 172.21.15.37:0/2759307215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56150284a970 con 0x5615027be6b0 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f95d80124f0 tx=0x7f95d8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178e31d7929c089b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d801a070 con 0x5615027be6b0 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f95d8014670 con 0x5615027be6b0 2024-09-21T06:34:20.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.640+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e001b050 con 0x5615027be6b0 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f95d8025050 con 0x5615027be6b0 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 10 client.7845 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x5615028286a0 con 0x5615027be6b0 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95e001b050 con 0x5615027be6b0 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f95d8028020 con 0x5615027be6b0 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 10 client.7845 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 10 client.7845 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:20.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 20 client.7845 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56150279df60 0x561502b8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 4 client.7845 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.641+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561502afb7c0 con 0x56150279df60 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.642+0000 7f95ef400640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56150279df60 0x561502b8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.642+0000 7f95ef400640 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56150279df60 0x561502b8f7b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69195df25c69f3a6 server_cookie=720f09982a434b2d in_seq=0 out_seq=0 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.642+0000 7f95dcc00640 10 client.7845 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95dcc00640 1 -- 172.21.15.37:0/2759307215 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561502afb7c0 con 0x56150279df60 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95dcc00640 10 client.7845 handle_command_reply: tid=0 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 client.7845 shutdown 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 msgr2=0x561502bc6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f95d80124f0 tx=0x7f95d8001e70 comp rx=0 tx=0).stop 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 shutdown_connections 2024-09-21T06:34:20.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x56150279df60 0x561502b8f7b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bc6550 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027df390 0x561502bc6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 --2- 172.21.15.37:0/2759307215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027b64d0 0x561502bc4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95dcc00640 0 client.7845 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.643+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 >> 172.21.15.37:0/2759307215 conn(0x5615027d34e0 msgr2=0x561502bb05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.644+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 shutdown_connections 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.644+0000 7f95fd811740 1 -- 172.21.15.37:0/2759307215 wait complete. 2024-09-21T06:34:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.644+0000 7f95fd811740 20 client.7845 trim_cache size 0 max 16384 2024-09-21T06:34:20.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.662+0000 7f95fd811740 1 Processor -- start 2024-09-21T06:34:20.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.662+0000 7f95fd811740 1 -- start start 2024-09-21T06:34:20.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502bcad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bdcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0202700 con 0x5615027be6b0 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f95f02018a0 con 0x5615027b64d0 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.663+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f95f0201fd0 con 0x5615027df390 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57602/0 (socket says 172.21.15.37:57602) 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 -- 172.21.15.37:0/2058017024 learned_addr learned my addr 172.21.15.37:0/2058017024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95efe00640 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bdcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95ef400640 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502bcad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 -- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 msgr2=0x561502bdcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bdcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 -- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 msgr2=0x561502bcad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502bcad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.664+0000 7f95e6a00640 1 -- 172.21.15.37:0/2058017024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561502bc5390 con 0x5615027be6b0 2024-09-21T06:34:20.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f95d80013c0 tx=0x7f95d8001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc13e91dce2228f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95df400640 1 -- 172.21.15.37:0/2058017024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d801a040 con 0x5615027be6b0 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95df400640 1 -- 172.21.15.37:0/2058017024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95d8014bf0 con 0x5615027be6b0 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95df400640 1 -- 172.21.15.37:0/2058017024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d80156a0 con 0x5615027be6b0 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 -- 172.21.15.37:0/2058017024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 msgr2=0x561502bcb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f95d80013c0 tx=0x7f95d8001890 comp rx=0 tx=0).stop 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 -- 172.21.15.37:0/2058017024 shutdown_connections 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615027be6b0 0x561502bcb2a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bdcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 --2- 172.21.15.37:0/2058017024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502bcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.665+0000 7f95fd811740 1 -- 172.21.15.37:0/2058017024 >> 172.21.15.37:0/2058017024 conn(0x5615027d34e0 msgr2=0x561502bad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.666+0000 7f95fd811740 1 -- 172.21.15.37:0/2058017024 shutdown_connections 2024-09-21T06:34:20.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.666+0000 7f95fd811740 1 -- 172.21.15.37:0/2058017024 wait complete. 2024-09-21T06:34:20.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.667+0000 7f95fd811740 1 -- start start 2024-09-21T06:34:20.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.667+0000 7f95fd811740 1 Processor -- start 2024-09-21T06:34:20.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502c4a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bc3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01f6060 con 0x561502b8f880 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561502af9510 con 0x5615027b64d0 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561502c4b480 con 0x5615027df390 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95fd811740 10 client.0 _ll_register_callbacks cb 0x7f95d8001da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57616/0 (socket says 172.21.15.37:57616) 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.668+0000 7f95efe00640 1 -- 172.21.15.37:0/2635156871 learned_addr learned my addr 172.21.15.37:0/2635156871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95ef400640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502c4a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95e6a00640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bc3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 -- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 msgr2=0x561502bc3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bc3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 -- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 msgr2=0x561502c4a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502c4a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 -- 172.21.15.37:0/2635156871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561502bee830 con 0x561502b8f880 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95efe00640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f95e80b9c60 tx=0x7f95e8082de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1742b901de1fa6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e808a040 con 0x561502b8f880 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f95e8082810 con 0x561502b8f880 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.669+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d8014bf0 con 0x561502b8f880 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f95e8087fe0 con 0x561502b8f880 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 10 client.7851 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x561502bc4070 con 0x561502b8f880 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95d8014bf0 con 0x561502b8f880 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f95e808a1e0 con 0x561502b8f880 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 10 client.7851 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 10 client.7851 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 20 client.7851 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x561502b85eb0 0x561502bcab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 4 client.7851 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x561502af7270 con 0x561502b85eb0 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.670+0000 7f95ef400640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x561502b85eb0 0x561502bcab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95ef400640 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x561502b85eb0 0x561502bcab50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed74a33b5358753 server_cookie=2cd364af21a158dd in_seq=0 out_seq=0 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95dcc00640 10 client.7851 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95dcc00640 1 -- 172.21.15.37:0/2635156871 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561502af7270 con 0x561502b85eb0 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95dcc00640 10 client.7851 handle_command_reply: tid=0 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95fd811740 1 client.7851 shutdown 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 msgr2=0x561502c44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f95e80b9c60 tx=0x7f95e8082de0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 shutdown_connections 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.671+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x561502b85eb0 0x561502bcab50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561502b8f880 0x561502c44760 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5615027df390 0x561502bc3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95dcc00640 0 client.7851 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 --2- 172.21.15.37:0/2635156871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615027b64d0 0x561502c4a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 >> 172.21.15.37:0/2635156871 conn(0x5615027d34e0 msgr2=0x561502bad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 shutdown_connections 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 1 -- 172.21.15.37:0/2635156871 wait complete. 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95fd811740 20 client.7851 trim_cache size 0 max 16384 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:34:20.677 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 msgr2=0x7f95cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 0x7f95cc07b0d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f95e0001120 tx=0x7f95e0009040 comp rx=0 tx=0).stop 2024-09-21T06:34:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 msgr2=0x7f95f01e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009d00 tx=0x7f95e8003e70 comp rx=0 tx=0).stop 2024-09-21T06:34:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 shutdown_connections 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f95cc078bc0 0x7f95cc07b0d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f015df80 0x7f95f01e7740 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95f00cc530 0x7f95f01e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.672+0000 7f95f6800640 1 --2- 172.21.15.37:0/1157152879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f00af5b0 0x7f95f01f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.673+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 >> 172.21.15.37:0/1157152879 conn(0x7f95f00b1b90 msgr2=0x7f95f00caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.673+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 shutdown_connections 2024-09-21T06:34:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.673+0000 7f95f6800640 1 -- 172.21.15.37:0/1157152879 wait complete. 2024-09-21T06:34:20.696 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:20.696 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 -- 172.21.15.37:0/4087141521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 msgr2=0x7f11b0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 --2- 172.21.15.37:0/4087141521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0074620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009830 tx=0x7f11a002a510 comp rx=0 tx=0).stop 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 -- 172.21.15.37:0/4087141521 shutdown_connections 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 --2- 172.21.15.37:0/4087141521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 0x7f11b006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 --2- 172.21.15.37:0/4087141521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0074620 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 -- 172.21.15.37:0/4087141521 >> 172.21.15.37:0/4087141521 conn(0x7f11b00b1b90 msgr2=0x7f11b00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 -- 172.21.15.37:0/4087141521 shutdown_connections 2024-09-21T06:34:20.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.962+0000 7f11b6c00640 1 -- 172.21.15.37:0/4087141521 wait complete. 2024-09-21T06:34:20.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.964+0000 7f11b6c00640 1 Processor -- start 2024-09-21T06:34:20.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.964+0000 7f11b6c00640 1 -- start start 2024-09-21T06:34:20.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 0x7f11b01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11b00ce500 con 0x7f11b006e330 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11b00ce380 con 0x7f11b0156a50 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b6c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f11b00ce680 con 0x7f11b0074b60 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57628/0 (socket says 172.21.15.37:57628) 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11afe00640 1 -- 172.21.15.37:0/3837683399 learned_addr learned my addr 172.21.15.37:0/3837683399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11af400640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 0x7f11b01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.966+0000 7f11b4c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 -- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 msgr2=0x7f11b01571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 0x7f11b01571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 -- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 msgr2=0x7f11b0157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 -- 172.21.15.37:0/3837683399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b015a510 con 0x7f11b006e330 2024-09-21T06:34:20.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11b4c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:20.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11afe00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009950 tx=0x7f11a0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11a7e00640 1 -- 172.21.15.37:0/3837683399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0005c90 con 0x7f11b006e330 2024-09-21T06:34:20.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11a7e00640 1 -- 172.21.15.37:0/3837683399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11a0031e50 con 0x7f11b006e330 2024-09-21T06:34:20.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b0159470 con 0x7f11b006e330 2024-09-21T06:34:20.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11a7e00640 1 -- 172.21.15.37:0/3837683399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0005260 con 0x7f11b006e330 2024-09-21T06:34:20.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.967+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b0159a30 con 0x7f11b006e330 2024-09-21T06:34:20.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.970+0000 7f11a7e00640 1 -- 172.21.15.37:0/3837683399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11a0005400 con 0x7f11b006e330 2024-09-21T06:34:20.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.971+0000 7f11a7e00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 0x7f119007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.971+0000 7f11a7e00640 1 -- 172.21.15.37:0/3837683399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f11a00bbff0 con 0x7f11b006e330 2024-09-21T06:34:20.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.971+0000 7f11af400640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 0x7f119007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.972+0000 7f11af400640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 0x7f119007b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f11b00b20a0 tx=0x7f119c009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.972+0000 7f11bdcbb740 1 Processor -- start 2024-09-21T06:34:20.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.973+0000 7f11bdcbb740 1 -- start start 2024-09-21T06:34:20.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee98550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee7b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556abedb11c0 con 0x556abea814d0 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556abedc4870 con 0x556abea896b0 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556abedb3ca0 con 0x556abeaaa390 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57636/0 (socket says 172.21.15.37:57636) 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11af400640 1 -- 172.21.15.37:0/3973489831 learned_addr learned my addr 172.21.15.37:0/3973489831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11b4c00640 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee7b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.975+0000 7f11afe00640 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee98550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 -- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 msgr2=0x556abee98550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee98550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 -- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 msgr2=0x556abee7b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee7b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 -- 172.21.15.37:0/3973489831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abeb8c7d0 con 0x556abea814d0 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f11af400640 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f119c011e40 tx=0x7f119c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce378cc20a3d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f119be00640 1 -- 172.21.15.37:0/3973489831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c00ecb0 con 0x556abea814d0 2024-09-21T06:34:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f119be00640 1 -- 172.21.15.37:0/3973489831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f119c00ee50 con 0x556abea814d0 2024-09-21T06:34:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.976+0000 7f119be00640 1 -- 172.21.15.37:0/3973489831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c026690 con 0x556abea814d0 2024-09-21T06:34:20.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.977+0000 7f11bdcbb740 1 -- 172.21.15.37:0/3973489831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 msgr2=0x556abee7b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.977+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f119c011e40 tx=0x7f119c014000 comp rx=0 tx=0).stop 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 -- 172.21.15.37:0/3973489831 shutdown_connections 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abee7b220 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee7b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/3973489831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee98550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 -- 172.21.15.37:0/3973489831 >> 172.21.15.37:0/3973489831 conn(0x556abea9e4e0 msgr2=0x556abee51440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 -- 172.21.15.37:0/3973489831 shutdown_connections 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.978+0000 7f11bdcbb740 1 -- 172.21.15.37:0/3973489831 wait complete. 2024-09-21T06:34:20.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.979+0000 7f11bdcbb740 1 -- start start 2024-09-21T06:34:20.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.979+0000 7f11bdcbb740 1 Processor -- start 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556abed5a410 con 0x556abea814d0 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556abedb7ab0 con 0x556abea896b0 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.980+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556abeb24740 con 0x556abeaaa390 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11bdcbb740 10 client.0 _ll_register_callbacks cb 0x556abed37360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57652/0 (socket says 172.21.15.37:57652) 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 -- 172.21.15.37:0/1556124984 learned_addr learned my addr 172.21.15.37:0/1556124984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:20.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11af400640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11b4c00640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 -- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 msgr2=0x556abee5d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee5d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 -- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 msgr2=0x556abee5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 -- 172.21.15.37:0/1556124984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abeb15970 con 0x556abea814d0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11af400640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f11afe00640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005820 tx=0x7f11a0084150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9a8212bf3cd6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.981+0000 7f1198c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0048040 con 0x556abea814d0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f11a0088150 con 0x556abea814d0 2024-09-21T06:34:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c00ee50 con 0x556abea814d0 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f11a0084230 con 0x556abea814d0 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 10 client.7860 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x556abeaf36a0 con 0x556abea814d0 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f119c00ee50 con 0x556abea814d0 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f11a0082d90 con 0x556abea814d0 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 10 client.7860 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 10 client.7860 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 20 client.7860 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abea68f60 0x556abee50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 4 client.7860 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.982+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556abedc67c0 con 0x556abea68f60 2024-09-21T06:34:20.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.983+0000 7f11af400640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abea68f60 0x556abee50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.983+0000 7f11af400640 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abea68f60 0x556abee50c90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93f5fdc7a5fad57b server_cookie=9d06947fc1e6d99d in_seq=0 out_seq=0 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.983+0000 7f1198c00640 10 client.7860 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f1198c00640 1 -- 172.21.15.37:0/1556124984 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556abedc67c0 con 0x556abea68f60 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f1198c00640 10 client.7860 handle_command_reply: tid=0 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 client.7860 shutdown 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 msgr2=0x556abeea9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005820 tx=0x7f11a0084150 comp rx=0 tx=0).stop 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 shutdown_connections 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abea68f60 0x556abee50c90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abeaaa390 0x556abee5d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea896b0 0x556abee5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f1198c00640 0 client.7860 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1556124984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea814d0 0x556abeea9140 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 >> 172.21.15.37:0/1556124984 conn(0x556abea9e4e0 msgr2=0x556abee995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 shutdown_connections 2024-09-21T06:34:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.984+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1556124984 wait complete. 2024-09-21T06:34:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:20.985+0000 7f11bdcbb740 20 client.7860 trim_cache size 0 max 16384 2024-09-21T06:34:21.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.004+0000 7f11bdcbb740 1 Processor -- start 2024-09-21T06:34:21.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.004+0000 7f11bdcbb740 1 -- start start 2024-09-21T06:34:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea814d0 0x556abee52310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea896b0 0x556abee52850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11b02006d0 con 0x556abeaaa390 2024-09-21T06:34:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01ff870 con 0x556abea814d0 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.005+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f11b01fffa0 con 0x556abea896b0 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57662/0 (socket says 172.21.15.37:57662) 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 -- 172.21.15.37:0/1335356168 learned_addr learned my addr 172.21.15.37:0/1335356168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11afe00640 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea814d0 0x556abee52310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11af400640 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea896b0 0x556abee52850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 -- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea896b0 msgr2=0x556abee52850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea896b0 0x556abee52850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 -- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea814d0 msgr2=0x556abee52310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea814d0 0x556abee52310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 -- 172.21.15.37:0/1335356168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abee90060 con 0x556abeaaa390 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.006+0000 7f11b4c00640 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f11a8003c90 tx=0x7f11a8005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ee0a9ede3f778a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.007+0000 7f119b400640 1 -- 172.21.15.37:0/1335356168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a801a020 con 0x556abeaaa390 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.007+0000 7f119b400640 1 -- 172.21.15.37:0/1335356168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11a8001c80 con 0x556abeaaa390 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.007+0000 7f119b400640 1 -- 172.21.15.37:0/1335356168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8002090 con 0x556abeaaa390 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.007+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1335356168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 msgr2=0x556abeeb8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.007+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f11a8003c90 tx=0x7f11a8005420 comp rx=0 tx=0).stop 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1335356168 shutdown_connections 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abeaaa390 0x556abeeb8e70 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea896b0 0x556abee52850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1335356168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abea814d0 0x556abee52310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1335356168 >> 172.21.15.37:0/1335356168 conn(0x556abea9e4e0 msgr2=0x556abee8e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1335356168 shutdown_connections 2024-09-21T06:34:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.008+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1335356168 wait complete. 2024-09-21T06:34:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.009+0000 7f11bdcbb740 1 -- start start 2024-09-21T06:34:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.009+0000 7f11bdcbb740 1 Processor -- start 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea814d0 0x556abef0f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abeaaa390 0x556abeeb8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556abee90ef0 con 0x556abea896b0 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11b00b4d40 con 0x556abeaaa390 2024-09-21T06:34:21.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556abef19590 con 0x556abea814d0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.010+0000 7f11bdcbb740 10 client.0 _ll_register_callbacks cb 0x7f11a00801d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57670/0 (socket says 172.21.15.37:57670) 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 -- 172.21.15.37:0/1693802625 learned_addr learned my addr 172.21.15.37:0/1693802625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11afe00640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea814d0 0x556abef0f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11b4c00640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abeaaa390 0x556abeeb8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 -- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea814d0 msgr2=0x556abef0f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea814d0 0x556abef0f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 -- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abeaaa390 msgr2=0x556abeeb8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abeaaa390 0x556abeeb8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 -- 172.21.15.37:0/1693802625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abee4dfa0 con 0x556abea896b0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f11af400640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f119c011e10 tx=0x7f119c0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612d1344a7e54b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.011+0000 7f1198c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:21.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0098a0 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f119c00edd0 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a8001c80 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f119c043050 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 10 client.7866 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x556abeeb84c0 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11a8001c80 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f119c046020 con 0x556abea896b0 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 10 client.7866 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 10 client.7866 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 20 client.7866 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abee5cf30 0x556abeeb9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 4 client.7866 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:21.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.012+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 --> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556abedc2270 con 0x556abee5cf30 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.013+0000 7f11afe00640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abee5cf30 0x556abeeb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.013+0000 7f11afe00640 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abee5cf30 0x556abeeb9180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cd8d8d5cda595c server_cookie=3703004cf00638fc in_seq=0 out_seq=0 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.013+0000 7f1198c00640 10 client.7866 ms_handle_connect on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f1198c00640 1 -- 172.21.15.37:0/1693802625 <== mds.1 v2:172.21.15.37:6832/1748999037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556abedc2270 con 0x556abee5cf30 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f1198c00640 10 client.7866 handle_command_reply: tid=0 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 client.7866 shutdown 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 msgr2=0x556abee4fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f119c011e10 tx=0x7f119c0094a0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 shutdown_connections 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:6832/1748999037,v1:172.21.15.37:6835/1748999037] conn(0x556abee5cf30 0x556abeeb9180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556abea896b0 0x556abee4fe20 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556abeaaa390 0x556abeeb8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 --2- 172.21.15.37:0/1693802625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556abea814d0 0x556abef0f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f1198c00640 0 client.7866 ms_handle_reset on v2:172.21.15.37:6832/1748999037 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 >> 172.21.15.37:0/1693802625 conn(0x556abea9e4e0 msgr2=0x556abee52310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 shutdown_connections 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 1 -- 172.21.15.37:0/1693802625 wait complete. 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.014+0000 7f11bdcbb740 20 client.7866 trim_cache size 0 max 16384 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:21.019 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:21.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:21.020 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:21.020 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.020 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.020 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 msgr2=0x7f119007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 0x7f119007b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f11b00b20a0 tx=0x7f119c009260 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 msgr2=0x7f11b00c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009950 tx=0x7f11a0033c20 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 shutdown_connections 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1190078b90 0x7f119007b0a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b0156a50 0x7f11b0157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b0074b60 0x7f11b01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 --2- 172.21.15.37:0/3837683399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11b006e330 0x7f11b00c92c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 >> 172.21.15.37:0/3837683399 conn(0x7f11b00b1b90 msgr2=0x7f11b00b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 shutdown_connections 2024-09-21T06:34:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.015+0000 7f11b6c00640 1 -- 172.21.15.37:0/3837683399 wait complete. 2024-09-21T06:34:21.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:21.040 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:21.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 -- 172.21.15.37:0/3166561814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c160a00 msgr2=0x7f641c06cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 --2- 172.21.15.37:0/3166561814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c160a00 0x7f641c06cf90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f6414005fd0 tx=0x7f641402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 -- 172.21.15.37:0/3166561814 shutdown_connections 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 --2- 172.21.15.37:0/3166561814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c160a00 0x7f641c06cf90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 --2- 172.21.15.37:0/3166561814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c0ccef0 0x7f641c0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 -- 172.21.15.37:0/3166561814 >> 172.21.15.37:0/3166561814 conn(0x7f641c06f920 msgr2=0x7f641c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 -- 172.21.15.37:0/3166561814 shutdown_connections 2024-09-21T06:34:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.267+0000 7f641b200640 1 -- 172.21.15.37:0/3166561814 wait complete. 2024-09-21T06:34:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.270+0000 7f641b200640 1 Processor -- start 2024-09-21T06:34:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.270+0000 7f641b200640 1 -- start start 2024-09-21T06:34:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c06e2c0 0x7f641c144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f641c160a00 0x7f641c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f641c06d650 con 0x7f641c144840 2024-09-21T06:34:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f641c06d4d0 con 0x7f641c160a00 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f641c06d7d0 con 0x7f641c06e2c0 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.271+0000 7f641a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57692/0 (socket says 172.21.15.37:57692) 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 -- 172.21.15.37:0/3020006980 learned_addr learned my addr 172.21.15.37:0/3020006980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f6419400640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f641c160a00 0x7f641c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f6419e00640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c06e2c0 0x7f641c144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 -- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c06e2c0 msgr2=0x7f641c144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c06e2c0 0x7f641c144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 -- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f641c160a00 msgr2=0x7f641c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f641c160a00 0x7f641c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 -- 172.21.15.37:0/3020006980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c074be0 con 0x7f641c144840 2024-09-21T06:34:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.272+0000 7f641a800640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6414009cd0 tx=0x7f64140039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.273+0000 7f6412000640 1 -- 172.21.15.37:0/3020006980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414031990 con 0x7f641c144840 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.273+0000 7f6412000640 1 -- 172.21.15.37:0/3020006980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6414031c20 con 0x7f641c144840 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.273+0000 7f6412000640 1 -- 172.21.15.37:0/3020006980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140330d0 con 0x7f641c144840 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.273+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c1f82c0 con 0x7f641c144840 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.273+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c1f8880 con 0x7f641c144840 2024-09-21T06:34:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.275+0000 7f6412000640 1 -- 172.21.15.37:0/3020006980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6414033270 con 0x7f641c144840 2024-09-21T06:34:21.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.276+0000 7f6412000640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 0x7f63fc07b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.276+0000 7f6412000640 1 -- 172.21.15.37:0/3020006980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f64140bc330 con 0x7f641c144840 2024-09-21T06:34:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.277+0000 7f6419e00640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 0x7f63fc07b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.277+0000 7f6427c34740 1 Processor -- start 2024-09-21T06:34:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.277+0000 7f6427c34740 1 -- start start 2024-09-21T06:34:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.277+0000 7f6419e00640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 0x7f63fc07b0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f640c001720 tx=0x7f640c007e90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.278+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cf2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6427c34740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6427c34740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de57c0b1c0 con 0x55de578db4d0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de57c1e870 con 0x55de57904390 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55de57c0dca0 con 0x55de578e36b0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cf2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57702/0 (socket says 172.21.15.37:57702) 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f641a800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 -- 172.21.15.37:0/4084335919 learned_addr learned my addr 172.21.15.37:0/4084335919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 -- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 msgr2=0x55de57cd5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 -- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 msgr2=0x55de57cf2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cf2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419400640 1 -- 172.21.15.37:0/4084335919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de579e67d0 con 0x55de578db4d0 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.279+0000 7f6419e00640 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cf2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.280+0000 7f6419400640 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f640800bc00 tx=0x7f64080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857dd6447b0751c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.282+0000 7f6406600640 1 -- 172.21.15.37:0/4084335919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408011440 con 0x55de578db4d0 2024-09-21T06:34:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.282+0000 7f6406600640 1 -- 172.21.15.37:0/4084335919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64080115e0 con 0x55de578db4d0 2024-09-21T06:34:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.282+0000 7f6406600640 1 -- 172.21.15.37:0/4084335919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408011a30 con 0x55de578db4d0 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.282+0000 7f6427c34740 1 -- 172.21.15.37:0/4084335919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 msgr2=0x55de57cd5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.282+0000 7f6427c34740 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f640800bc00 tx=0x7f64080107a0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 -- 172.21.15.37:0/4084335919 shutdown_connections 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57cd5400 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 --2- 172.21.15.37:0/4084335919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cf2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 -- 172.21.15.37:0/4084335919 >> 172.21.15.37:0/4084335919 conn(0x55de578f84e0 msgr2=0x55de57cab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 -- 172.21.15.37:0/4084335919 shutdown_connections 2024-09-21T06:34:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.283+0000 7f6427c34740 1 -- 172.21.15.37:0/4084335919 wait complete. 2024-09-21T06:34:21.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.284+0000 7f6427c34740 1 -- start start 2024-09-21T06:34:21.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.284+0000 7f6427c34740 1 Processor -- start 2024-09-21T06:34:21.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cd1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de57bb4410 con 0x55de578db4d0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de57c11ab0 con 0x55de57904390 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55de5797e740 con 0x55de578e36b0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6427c34740 10 client.0 _ll_register_callbacks cb 0x55de57b91360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.286+0000 7f6419e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57716/0 (socket says 172.21.15.37:57716) 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 -- 172.21.15.37:0/3687408931 learned_addr learned my addr 172.21.15.37:0/3687408931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419400640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f641a800640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cd1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 -- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 msgr2=0x55de57cd0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 -- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 msgr2=0x55de57cd1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cd1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 -- 172.21.15.37:0/3687408931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5796f970 con 0x55de578db4d0 2024-09-21T06:34:21.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f6419e00640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f640c014700 tx=0x7f640c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fa6ec3d912a4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.287+0000 7f63faa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.288+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c025070 con 0x55de578db4d0 2024-09-21T06:34:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.288+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64080115e0 con 0x55de578db4d0 2024-09-21T06:34:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.288+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f640c015850 con 0x55de578db4d0 2024-09-21T06:34:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f640c01b040 con 0x55de578db4d0 2024-09-21T06:34:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 10 client.7881 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55de5794d6a0 con 0x55de578db4d0 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64080115e0 con 0x55de578db4d0 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f640c015a50 con 0x55de578db4d0 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 10 client.7881 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 10 client.7881 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 20 client.7881 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de578c2f60 0x55de57cce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 4 client.7881 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.289+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de57c207c0 con 0x55de578c2f60 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.290+0000 7f6419400640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de578c2f60 0x55de57cce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.290+0000 7f6419400640 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de578c2f60 0x55de57cce8b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5b3652952a348a9 server_cookie=6203af1537aa07da in_seq=0 out_seq=0 2024-09-21T06:34:21.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.290+0000 7f63faa00640 10 client.7881 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:21.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.291+0000 7f63faa00640 1 -- 172.21.15.37:0/3687408931 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de57c207c0 con 0x55de578c2f60 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.291+0000 7f63faa00640 10 client.7881 handle_command_reply: tid=0 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.291+0000 7f6427c34740 1 client.7881 shutdown 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 msgr2=0x55de57ce9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f640c014700 tx=0x7f640c000ed0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 shutdown_connections 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de578c2f60 0x55de57cce8b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57cd1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57cd0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 --2- 172.21.15.37:0/3687408931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ce9270 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f63faa00640 0 client.7881 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 >> 172.21.15.37:0/3687408931 conn(0x55de578f84e0 msgr2=0x55de57ca4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 shutdown_connections 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 1 -- 172.21.15.37:0/3687408931 wait complete. 2024-09-21T06:34:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.292+0000 7f6427c34740 20 client.7881 trim_cache size 0 max 16384 2024-09-21T06:34:21.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.311+0000 7f6427c34740 1 Processor -- start 2024-09-21T06:34:21.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.311+0000 7f6427c34740 1 -- start start 2024-09-21T06:34:21.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57d12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57ccf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f641c2026c0 con 0x55de578db4d0 2024-09-21T06:34:21.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f641c201860 con 0x55de57904390 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.312+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f641c201f90 con 0x55de578e36b0 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57730/0 (socket says 172.21.15.37:57730) 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 -- 172.21.15.37:0/228239036 learned_addr learned my addr 172.21.15.37:0/228239036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419400640 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57d12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f641a800640 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57ccf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 -- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 msgr2=0x55de57d12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57d12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 -- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 msgr2=0x55de57ccf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57ccf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 -- 172.21.15.37:0/228239036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de57ceb960 con 0x55de578db4d0 2024-09-21T06:34:21.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.313+0000 7f6419e00640 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f640c000ff0 tx=0x7f640c015480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68849d3392c87ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6405c00640 1 -- 172.21.15.37:0/228239036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c025040 con 0x55de578db4d0 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6405c00640 1 -- 172.21.15.37:0/228239036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f640c01c040 con 0x55de578db4d0 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6405c00640 1 -- 172.21.15.37:0/228239036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c006890 con 0x55de578db4d0 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 -- 172.21.15.37:0/228239036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 msgr2=0x55de57ca6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f640c000ff0 tx=0x7f640c015480 comp rx=0 tx=0).stop 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 -- 172.21.15.37:0/228239036 shutdown_connections 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de57904390 0x55de57ccf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de578e36b0 0x55de57d12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 --2- 172.21.15.37:0/228239036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578db4d0 0x55de57ca6be0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.314+0000 7f6427c34740 1 -- 172.21.15.37:0/228239036 >> 172.21.15.37:0/228239036 conn(0x55de578f84e0 msgr2=0x55de57cc3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.315+0000 7f6427c34740 1 -- 172.21.15.37:0/228239036 shutdown_connections 2024-09-21T06:34:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.315+0000 7f6427c34740 1 -- 172.21.15.37:0/228239036 wait complete. 2024-09-21T06:34:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.315+0000 7f6427c34740 1 -- start start 2024-09-21T06:34:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.316+0000 7f6427c34740 1 Processor -- start 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de578db4d0 0x55de57d73390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de57904390 0x55de57d6ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f641c200900 con 0x55de578e36b0 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de57cff820 con 0x55de578db4d0 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55de57d6f950 con 0x55de57904390 2024-09-21T06:34:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6427c34740 10 client.0 _ll_register_callbacks cb 0x7f640c001d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6419400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6419400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57740/0 (socket says 172.21.15.37:57740) 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.317+0000 7f6419400640 1 -- 172.21.15.37:0/3293844549 learned_addr learned my addr 172.21.15.37:0/3293844549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f641a800640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de57904390 0x55de57d6ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419e00640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de578db4d0 0x55de57d73390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 -- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de57904390 msgr2=0x55de57d6ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de57904390 0x55de57d6ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 -- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de578db4d0 msgr2=0x55de57d73390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de578db4d0 0x55de57d73390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 -- 172.21.15.37:0/3293844549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de57d119f0 con 0x55de578e36b0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f6419400640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f640800b320 tx=0x7f640800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0b7192a26dcc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f63faa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640800b600 con 0x55de578e36b0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.318+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f6408004040 con 0x55de578e36b0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c01c040 con 0x55de578e36b0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f6408039050 con 0x55de578e36b0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 10 client.7887 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55de57cceb60 con 0x55de578e36b0 2024-09-21T06:34:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f640c01c040 con 0x55de578e36b0 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f640803c020 con 0x55de578e36b0 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 10 client.7887 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 10 client.7887 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 20 client.7887 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de57ccc730 0x55de57cccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 4 client.7887 mds_command: new command op to 7096 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.319+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 --> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de57c1c270 con 0x55de57ccc730 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.320+0000 7f6419e00640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de57ccc730 0x55de57cccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.320+0000 7f6419e00640 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de57ccc730 0x55de57cccb40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a16970259d0fc0 server_cookie=1dc03d60f8fac135 in_seq=0 out_seq=0 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.320+0000 7f63faa00640 10 client.7887 ms_handle_connect on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f63faa00640 1 -- 172.21.15.37:0/3293844549 <== mds.2 v2:172.21.15.150:6835/3722358982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55de57c1c270 con 0x55de57ccc730 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f63faa00640 10 client.7887 handle_command_reply: tid=0 2024-09-21T06:34:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 client.7887 shutdown 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 msgr2=0x55de57d11ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f640800b320 tx=0x7f640800be60 comp rx=0 tx=0).stop 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 shutdown_connections 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:6835/3722358982,v1:172.21.15.150:6837/3722358982] conn(0x55de57ccc730 0x55de57cccb40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de578e36b0 0x55de57d11ed0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de57904390 0x55de57d6ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 --2- 172.21.15.37:0/3293844549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de578db4d0 0x55de57d73390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f63faa00640 0 client.7887 ms_handle_reset on v2:172.21.15.150:6835/3722358982 2024-09-21T06:34:21.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 >> 172.21.15.37:0/3293844549 conn(0x55de578f84e0 msgr2=0x55de57d12800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 shutdown_connections 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 1 -- 172.21.15.37:0/3293844549 wait complete. 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.321+0000 7f6427c34740 20 client.7887 trim_cache size 0 max 16384 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.328 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 msgr2=0x7f63fc07b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 0x7f63fc07b0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f640c001720 tx=0x7f640c007e90 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 msgr2=0x7f641c15ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6414009cd0 tx=0x7f64140039f0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 shutdown_connections 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63fc078bb0 0x7f63fc07b0c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f641c144840 0x7f641c15ade0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f641c160a00 0x7f641c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 --2- 172.21.15.37:0/3020006980 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f641c06e2c0 0x7f641c144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 >> 172.21.15.37:0/3020006980 conn(0x7f641c06f920 msgr2=0x7f641c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 shutdown_connections 2024-09-21T06:34:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.322+0000 7f641b200640 1 -- 172.21.15.37:0/3020006980 wait complete. 2024-09-21T06:34:21.346 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:21.346 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:34:21.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T06:34:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 -- 172.21.15.37:0/367481701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 msgr2=0x7f77180b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 --2- 172.21.15.37:0/367481701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77180b4f90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f77180cd790 tx=0x7f770802a540 comp rx=0 tx=0).stop 2024-09-21T06:34:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 -- 172.21.15.37:0/367481701 shutdown_connections 2024-09-21T06:34:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 --2- 172.21.15.37:0/367481701 >> v2:172.21.15.150:3300/0 conn(0x7f77180ccfc0 0x7f77180cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 --2- 172.21.15.37:0/367481701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77180b4f90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 -- 172.21.15.37:0/367481701 >> 172.21.15.37:0/367481701 conn(0x7f771806f920 msgr2=0x7f77180b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.574+0000 7f771d400640 1 -- 172.21.15.37:0/367481701 shutdown_connections 2024-09-21T06:34:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.575+0000 7f771d400640 1 -- 172.21.15.37:0/367481701 wait complete. 2024-09-21T06:34:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.577+0000 7f771d400640 1 Processor -- start 2024-09-21T06:34:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.577+0000 7f771d400640 1 -- start start 2024-09-21T06:34:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718156c60 0x7f77180b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718159540 0x7f77180737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f771806d560 con 0x7f771806e370 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f771806d3e0 con 0x7f7718159540 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f771d400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f771806d6e0 con 0x7f7718156c60 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.578+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57750/0 (socket says 172.21.15.37:57750) 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 -- 172.21.15.37:0/2820982657 learned_addr learned my addr 172.21.15.37:0/2820982657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f771ca00640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718159540 0x7f77180737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718156c60 0x7f77180b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 -- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718156c60 msgr2=0x7f77180b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718156c60 0x7f77180b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 -- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718159540 msgr2=0x7f77180737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718159540 0x7f77180737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 -- 172.21.15.37:0/2820982657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181f6400 con 0x7f771806e370 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.579+0000 7f7717e00640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7708009920 tx=0x7f7708033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.580+0000 7f770fe00640 1 -- 172.21.15.37:0/2820982657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708031d60 con 0x7f771806e370 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.580+0000 7f770fe00640 1 -- 172.21.15.37:0/2820982657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7708005020 con 0x7f771806e370 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.580+0000 7f770fe00640 1 -- 172.21.15.37:0/2820982657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708005470 con 0x7f771806e370 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.580+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77181584a0 con 0x7f771806e370 2024-09-21T06:34:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.580+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7718158a60 con 0x7f771806e370 2024-09-21T06:34:21.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.582+0000 7f770fe00640 1 -- 172.21.15.37:0/2820982657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77080047a0 con 0x7f771806e370 2024-09-21T06:34:21.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.584+0000 7f770fe00640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 0x7f76f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.584+0000 7f770fe00640 1 -- 172.21.15.37:0/2820982657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f77080bbf80 con 0x7f771806e370 2024-09-21T06:34:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.585+0000 7f7725a67740 1 Processor -- start 2024-09-21T06:34:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.585+0000 7f7717400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 0x7f76f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.585+0000 7f7725a67740 1 -- start start 2024-09-21T06:34:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.585+0000 7f7717400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 0x7f76f807b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f771815bb10 tx=0x7f7704007e90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585c6c781c0 con 0x5585c69484d0 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585c6c8b870 con 0x5585c69506b0 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5585c6c7aca0 con 0x5585c6971390 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7717400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7717400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57762/0 (socket says 172.21.15.37:57762) 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.586+0000 7f7717400640 1 -- 172.21.15.37:0/944538764 learned_addr learned my addr 172.21.15.37:0/944538764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717e00640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f771ca00640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 -- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 msgr2=0x5585c6d5f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 -- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 msgr2=0x5585c6d42940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d42940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 -- 172.21.15.37:0/944538764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c6a537d0 con 0x5585c69484d0 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f771ca00640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7717400640 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f770400f9f0 tx=0x7f7704012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ac155874acd713 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7703e00640 1 -- 172.21.15.37:0/944538764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770401b050 con 0x5585c69484d0 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7703e00640 1 -- 172.21.15.37:0/944538764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7704025070 con 0x5585c69484d0 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.587+0000 7f7703e00640 1 -- 172.21.15.37:0/944538764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704020440 con 0x5585c69484d0 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 -- 172.21.15.37:0/944538764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 msgr2=0x5585c6d42400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f770400f9f0 tx=0x7f7704012000 comp rx=0 tx=0).stop 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 -- 172.21.15.37:0/944538764 shutdown_connections 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d42400 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 --2- 172.21.15.37:0/944538764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 -- 172.21.15.37:0/944538764 >> 172.21.15.37:0/944538764 conn(0x5585c69654e0 msgr2=0x5585c6d18620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 -- 172.21.15.37:0/944538764 shutdown_connections 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.588+0000 7f7725a67740 1 -- 172.21.15.37:0/944538764 wait complete. 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.589+0000 7f7725a67740 1 -- start start 2024-09-21T06:34:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.590+0000 7f7725a67740 1 Processor -- start 2024-09-21T06:34:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c69506b0 0x5585c6d599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c6971390 0x5585c6d59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585c6c21410 con 0x5585c69484d0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585c6c7eab0 con 0x5585c6971390 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5585c69eb740 con 0x5585c69506b0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7725a67740 10 client.0 _ll_register_callbacks cb 0x5585c6bfe360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.591+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57776/0 (socket says 172.21.15.37:57776) 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 -- 172.21.15.37:0/1211796071 learned_addr learned my addr 172.21.15.37:0/1211796071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f771ca00640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c6971390 0x5585c6d59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717400640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c69506b0 0x5585c6d599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 -- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c69506b0 msgr2=0x5585c6d599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c69506b0 0x5585c6d599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 -- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c6971390 msgr2=0x5585c6d59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c6971390 0x5585c6d59f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 -- 172.21.15.37:0/1211796071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c69dc970 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7717e00640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f77080ba2b0 tx=0x7f770807fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5e6eb4cff4cab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.592+0000 7f7700c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708048070 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f770806f170 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7704025070 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f7708083770 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 10 client.7902 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x5585c69ba6a0 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7704025070 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f770808d0b0 con 0x5585c69484d0 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 10 client.7902 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 10 client.7902 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 20 client.7902 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c692ff60 0x5585c6d6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 4 client.7902 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.593+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c6c8d7c0 con 0x5585c692ff60 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.594+0000 7f7717400640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c692ff60 0x5585c6d6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.594+0000 7f7717400640 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c692ff60 0x5585c6d6fa30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5b7ef0306988c6 server_cookie=2285777546b60d87 in_seq=0 out_seq=0 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.594+0000 7f7700c00640 10 client.7902 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.595+0000 7f7700c00640 1 -- 172.21.15.37:0/1211796071 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5585c6c8d7c0 con 0x5585c692ff60 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.595+0000 7f7700c00640 10 client.7902 handle_command_reply: tid=0 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.595+0000 7f7725a67740 1 client.7902 shutdown 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 msgr2=0x5585c6d3f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f77080ba2b0 tx=0x7f770807fc10 comp rx=0 tx=0).stop 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 shutdown_connections 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c692ff60 0x5585c6d6fa30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c6971390 0x5585c6d59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c69506b0 0x5585c6d599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 --2- 172.21.15.37:0/1211796071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d3f200 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 >> 172.21.15.37:0/1211796071 conn(0x5585c69654e0 msgr2=0x5585c6d60790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7700c00640 0 client.7902 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 shutdown_connections 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 1 -- 172.21.15.37:0/1211796071 wait complete. 2024-09-21T06:34:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.596+0000 7f7725a67740 20 client.7902 trim_cache size 0 max 16384 2024-09-21T06:34:21.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.614+0000 7f7725a67740 1 Processor -- start 2024-09-21T06:34:21.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.615+0000 7f7725a67740 1 -- start start 2024-09-21T06:34:21.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d80630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77182027a0 con 0x5585c69484d0 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7718201940 con 0x5585c69506b0 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7718202070 con 0x5585c6971390 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7717e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57778/0 (socket says 172.21.15.37:57778) 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7717e00640 1 -- 172.21.15.37:0/1947821451 learned_addr learned my addr 172.21.15.37:0/1947821451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f771ca00640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.616+0000 7f7717400640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d80630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 -- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 msgr2=0x5585c6d5c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 -- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 msgr2=0x5585c6d80630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d80630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 -- 172.21.15.37:0/1947821451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c6d5d8a0 con 0x5585c69484d0 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f771ca00640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717400640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d80630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7717e00640 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7708083bf0 tx=0x7f7708083c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8199e11938b5db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7703400640 1 -- 172.21.15.37:0/1947821451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708048040 con 0x5585c69484d0 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7703400640 1 -- 172.21.15.37:0/1947821451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7708088690 con 0x5585c69484d0 2024-09-21T06:34:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.617+0000 7f7703400640 1 -- 172.21.15.37:0/1947821451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080802b0 con 0x5585c69484d0 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 -- 172.21.15.37:0/1947821451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 msgr2=0x5585c6d56b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7708083bf0 tx=0x7f7708083c80 comp rx=0 tx=0).stop 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 -- 172.21.15.37:0/1947821451 shutdown_connections 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d5c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69506b0 0x5585c6d80630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 --2- 172.21.15.37:0/1947821451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69484d0 0x5585c6d56b70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 -- 172.21.15.37:0/1947821451 >> 172.21.15.37:0/1947821451 conn(0x5585c69654e0 msgr2=0x5585c6d12380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 -- 172.21.15.37:0/1947821451 shutdown_connections 2024-09-21T06:34:21.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.618+0000 7f7725a67740 1 -- 172.21.15.37:0/1947821451 wait complete. 2024-09-21T06:34:21.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.619+0000 7f7725a67740 1 -- start start 2024-09-21T06:34:21.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.620+0000 7f7725a67740 1 Processor -- start 2024-09-21T06:34:21.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 0x5585c6d5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77181f9630 con 0x5585c69506b0 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585c6c7ae20 con 0x5585c69484d0 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5585c6d39fa0 con 0x5585c6971390 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7725a67740 10 client.0 _ll_register_callbacks cb 0x7f7708009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7717400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7717400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57790/0 (socket says 172.21.15.37:57790) 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7717400640 1 -- 172.21.15.37:0/1036757038 learned_addr learned my addr 172.21.15.37:0/1036757038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f771ca00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.621+0000 7f7717e00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 0x5585c6d5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 -- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 msgr2=0x5585c6d314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 -- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 msgr2=0x5585c6d5eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 0x5585c6d5eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 -- 172.21.15.37:0/1036757038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c6d16480 con 0x5585c69506b0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f771ca00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717e00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 0x5585c6d5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7717400640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7704021000 tx=0x7f77040013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa12ad0537f24e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7700c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770401b040 con 0x5585c69506b0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f7704020440 con 0x5585c69506b0 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7708088690 con 0x5585c69506b0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.622+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f7704042050 con 0x5585c69506b0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 10 client.7908 fetch_fsmap learned FSMap version 99 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x5585c6d30be0 con 0x5585c69506b0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7708088690 con 0x5585c69506b0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 99) ==== 3558+0+0 (secure 0 0 0) 0x7f7704045020 con 0x5585c69506b0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 10 client.7908 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 10 client.7908 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 20 client.7908 populate_metadata read hostname 'smithi037' 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c6d5b100 0x5585c6d31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 4 client.7908 mds_command: new command op to 7392 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 --> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585c6c89270 con 0x5585c6d5b100 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.623+0000 7f7717e00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c6d5b100 0x5585c6d31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.624+0000 7f7717e00640 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c6d5b100 0x5585c6d31a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391a991d9e0b5a8c server_cookie=f82653f0d80c6757 in_seq=0 out_seq=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.624+0000 7f7700c00640 10 client.7908 ms_handle_connect on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.624+0000 7f7700c00640 1 -- 172.21.15.37:0/1036757038 <== mds.0 v2:172.21.15.150:6834/2979710501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x5585c6c89270 con 0x5585c6d5b100 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.624+0000 7f7700c00640 10 client.7908 handle_command_reply: tid=0 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.624+0000 7f7725a67740 1 client.7908 shutdown 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 msgr2=0x5585c6d5f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7704021000 tx=0x7f77040013c0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 shutdown_connections 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:6834/2979710501,v1:172.21.15.150:6836/2979710501] conn(0x5585c6d5b100 0x5585c6d31a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585c69506b0 0x5585c6d5f400 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5585c6971390 0x5585c6d314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 --2- 172.21.15.37:0/1036757038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585c69484d0 0x5585c6d5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7700c00640 0 client.7908 ms_handle_reset on v2:172.21.15.150:6834/2979710501 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 >> 172.21.15.37:0/1036757038 conn(0x5585c69654e0 msgr2=0x5585c6d56940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 shutdown_connections 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 1 -- 172.21.15.37:0/1036757038 wait complete. 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.625+0000 7f7725a67740 20 client.7908 trim_cache size 0 max 16384 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:21.634 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2925eb7c-09a8-49df-85fc-0703b500c34a", 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "2925eb7c-09a8-49df-85fc-0703b500c34a": { 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:34:15.993402+0000", 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:21.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 msgr2=0x7f76f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 0x7f76f807b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f771815bb10 tx=0x7f7704007e90 comp rx=0 tx=0).stop 2024-09-21T06:34:21.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 msgr2=0x7f77181f5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7708009920 tx=0x7f7708033c90 comp rx=0 tx=0).stop 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 shutdown_connections 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f76f8078b90 0x7f76f807b0a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7718159540 0x7f77180737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7718156c60 0x7f77180b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 --2- 172.21.15.37:0/2820982657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f771806e370 0x7f77181f5ec0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 >> 172.21.15.37:0/2820982657 conn(0x7f771806f920 msgr2=0x7f77180b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 shutdown_connections 2024-09-21T06:34:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.626+0000 7f771d400640 1 -- 172.21.15.37:0/2820982657 wait complete. 2024-09-21T06:34:21.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:34:21.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.878+0000 7f942ee00640 1 -- 172.21.15.37:0/1007535049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280d0b00 msgr2=0x7f9428145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.878+0000 7f942ee00640 1 --2- 172.21.15.37:0/1007535049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280d0b00 0x7f9428145040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f941c005fd0 tx=0x7f941c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.879+0000 7f942ee00640 1 -- 172.21.15.37:0/1007535049 shutdown_connections 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.879+0000 7f942ee00640 1 --2- 172.21.15.37:0/1007535049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280d0b00 0x7f9428145040 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.879+0000 7f942ee00640 1 -- 172.21.15.37:0/1007535049 >> 172.21.15.37:0/1007535049 conn(0x7f94280d1020 msgr2=0x7f94280d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.879+0000 7f942ee00640 1 -- 172.21.15.37:0/1007535049 shutdown_connections 2024-09-21T06:34:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.879+0000 7f942ee00640 1 -- 172.21.15.37:0/1007535049 wait complete. 2024-09-21T06:34:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.881+0000 7f942ee00640 1 Processor -- start 2024-09-21T06:34:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.881+0000 7f942ee00640 1 -- start start 2024-09-21T06:34:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.882+0000 7f942ee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 0x7f94281f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.882+0000 7f942ee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 0x7f94280b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942ee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94280cdf50 con 0x7f94280cc240 2024-09-21T06:34:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942ee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9428072240 con 0x7f94280d0b00 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942ee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f94280ce0d0 con 0x7f9428160990 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57800/0 (socket says 172.21.15.37:57800) 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 -- 172.21.15.37:0/2170622897 learned_addr learned my addr 172.21.15.37:0/2170622897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942da00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 0x7f94281f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942d000640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 0x7f94280b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 -- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 msgr2=0x7f94280b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 0x7f94280b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 -- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 msgr2=0x7f94281f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 0x7f94281f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942e400640 1 -- 172.21.15.37:0/2170622897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280ccb90 con 0x7f94280cc240 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942da00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 0x7f94281f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.883+0000 7f942d000640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 0x7f94280b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f942e400640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f942000c700 tx=0x7f942000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420003b70 con 0x7f94280cc240 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9420003d30 con 0x7f94280cc240 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94280b6070 con 0x7f94280cc240 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420005330 con 0x7f94280cc240 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.884+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94280ae600 con 0x7f94280cc240 2024-09-21T06:34:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.885+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94280ae090 con 0x7f94280cc240 2024-09-21T06:34:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.886+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94200054d0 con 0x7f94280cc240 2024-09-21T06:34:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.887+0000 7f9425600640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 0x7f940807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.887+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f942009ac80 con 0x7f94280cc240 2024-09-21T06:34:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.888+0000 7f942da00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 0x7f940807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.889+0000 7f942da00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 0x7f940807b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f941c033000 tx=0x7f941c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:21.892+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94280ccb90 con 0x7f94280cc240 2024-09-21T06:34:22.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.131+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9428141140 con 0x7f94280cc240 2024-09-21T06:34:22.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.132+0000 7f9425600640 1 -- 172.21.15.37:0/2170622897 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f9420062f20 con 0x7f94280cc240 2024-09-21T06:34:22.134 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:22.135 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 msgr2=0x7f940807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 0x7f940807b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f941c033000 tx=0x7f941c004730 comp rx=0 tx=0).stop 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 msgr2=0x7f94280cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f942000c700 tx=0x7f942000cbc0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 shutdown_connections 2024-09-21T06:34:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9408078bc0 0x7f940807b0d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94280cc240 0x7f94280cc650 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9428160990 0x7f94280b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 --2- 172.21.15.37:0/2170622897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94280d0b00 0x7f94281f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 >> 172.21.15.37:0/2170622897 conn(0x7f94280d1020 msgr2=0x7f94280b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 shutdown_connections 2024-09-21T06:34:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.134+0000 7f942ee00640 1 -- 172.21.15.37:0/2170622897 wait complete. 2024-09-21T06:34:22.158 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:22.159 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.383+0000 7f74ada00640 1 -- 172.21.15.37:0/485049712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8160980 msgr2=0x7f74a8145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.383+0000 7f74ada00640 1 --2- 172.21.15.37:0/485049712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8160980 0x7f74a8145040 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7498009860 tx=0x7f749802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.384+0000 7f74ada00640 1 -- 172.21.15.37:0/485049712 shutdown_connections 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.384+0000 7f74ada00640 1 --2- 172.21.15.37:0/485049712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8160980 0x7f74a8145040 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.384+0000 7f74ada00640 1 -- 172.21.15.37:0/485049712 >> 172.21.15.37:0/485049712 conn(0x7f74a80d1020 msgr2=0x7f74a80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.384+0000 7f74ada00640 1 -- 172.21.15.37:0/485049712 shutdown_connections 2024-09-21T06:34:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.384+0000 7f74ada00640 1 -- 172.21.15.37:0/485049712 wait complete. 2024-09-21T06:34:22.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.386+0000 7f74ada00640 1 Processor -- start 2024-09-21T06:34:22.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.386+0000 7f74ada00640 1 -- start start 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74a8160980 0x7f74a815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 0x7f74a815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74a806dba0 con 0x7f74a8145580 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74a806da20 con 0x7f74a81f8eb0 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ada00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f74a806dd20 con 0x7f74a8160980 2024-09-21T06:34:22.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57830/0 (socket says 172.21.15.37:57830) 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 -- 172.21.15.37:0/297086046 learned_addr learned my addr 172.21.15.37:0/297086046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ad000640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 0x7f74a815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7400640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74a8160980 0x7f74a815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 -- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74a8160980 msgr2=0x7f74a815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74a8160980 0x7f74a815b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 -- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 msgr2=0x7f74a815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 0x7f74a815fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74a7e00640 1 -- 172.21.15.37:0/297086046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a81f9630 con 0x7f74a8145580 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.388+0000 7f74ad000640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 0x7f74a815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f74a7e00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7498009980 tx=0x7f7498033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498031d60 con 0x7f74a8145580 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7498005020 con 0x7f74a8145580 2024-09-21T06:34:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498005450 con 0x7f74a8145580 2024-09-21T06:34:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a81f55d0 con 0x7f74a8145580 2024-09-21T06:34:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.389+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81f5b90 con 0x7f74a8145580 2024-09-21T06:34:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.390+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a8145040 con 0x7f74a8145580 2024-09-21T06:34:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.391+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74980047a0 con 0x7f74a8145580 2024-09-21T06:34:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.392+0000 7f749fe00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 0x7f748807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.393+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f74980bc250 con 0x7f74a8145580 2024-09-21T06:34:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.393+0000 7f74a7400640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 0x7f748807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.394+0000 7f74a7400640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 0x7f748807b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b2920 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.397+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f749803f030 con 0x7f74a8145580 2024-09-21T06:34:22.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.666+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f74a8145340 con 0x7f74a8145580 2024-09-21T06:34:22.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.667+0000 7f749fe00640 1 -- 172.21.15.37:0/297086046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2014 (secure 0 0 0) 0x7f7498084460 con 0x7f74a8145580 2024-09-21T06:34:22.669 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:22.669 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":265928704,"bytes_avail":772828184576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-09-21T06:34:19.193583+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 msgr2=0x7f748807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 0x7f748807b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b2920 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 msgr2=0x7f74a81f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7498009980 tx=0x7f7498033d60 comp rx=0 tx=0).stop 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 shutdown_connections 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7488078b90 0x7f748807b0a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74a81f8eb0 0x7f74a815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74a8160980 0x7f74a815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 --2- 172.21.15.37:0/297086046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a8145580 0x7f74a81f8970 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:22.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 >> 172.21.15.37:0/297086046 conn(0x7f74a80d1020 msgr2=0x7f74a80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:22.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 shutdown_connections 2024-09-21T06:34:22.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:22.670+0000 7f74ada00640 1 -- 172.21.15.37:0/297086046 wait complete. 2024-09-21T06:34:22.697 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:34:23.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 -- 172.21.15.37:0/3046145427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff806d7f0 msgr2=0x7f6ff8154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:23.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 --2- 172.21.15.37:0/3046145427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff806d7f0 0x7f6ff8154ea0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8162210 tx=0x7f6ff002aac0 comp rx=0 tx=0).stop 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 -- 172.21.15.37:0/3046145427 shutdown_connections 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 --2- 172.21.15.37:0/3046145427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff806d7f0 0x7f6ff8154ea0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 --2- 172.21.15.37:0/3046145427 >> v2:172.21.15.150:3300/0 conn(0x7f6ff81629c0 0x7f6ff806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 -- 172.21.15.37:0/3046145427 >> 172.21.15.37:0/3046145427 conn(0x7f6ff80d1930 msgr2=0x7f6ff80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 -- 172.21.15.37:0/3046145427 shutdown_connections 2024-09-21T06:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.934+0000 7f6ffee00640 1 -- 172.21.15.37:0/3046145427 wait complete. 2024-09-21T06:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.936+0000 7f6ffee00640 1 Processor -- start 2024-09-21T06:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.936+0000 7f6ffee00640 1 -- start start 2024-09-21T06:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ff806d7f0 0x7f6ff81e8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 0x7f6ff814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8155820 con 0x7f6ff80b9e40 2024-09-21T06:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81556a0 con 0x7f6ff81f5c30 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff81559a0 con 0x7f6ff806d7f0 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57086/0 (socket says 172.21.15.37:57086) 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffd000640 1 -- 172.21.15.37:0/1142514584 learned_addr learned my addr 172.21.15.37:0/1142514584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffe400640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 0x7f6ff814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffda00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ff806d7f0 0x7f6ff81e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffd000640 1 -- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ff806d7f0 msgr2=0x7f6ff81e8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.938+0000 7f6ffd000640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ff806d7f0 0x7f6ff81e8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffd000640 1 -- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 msgr2=0x7f6ff814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffd000640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 0x7f6ff814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffd000640 1 -- 172.21.15.37:0/1142514584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff80b8210 con 0x7f6ff80b9e40 2024-09-21T06:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffe400640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 0x7f6ff814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffd000640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400d740 tx=0x7f6fe400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4017070 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe4009d20 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff80b84f0 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40106c0 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.939+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff80b8ab0 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.940+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff81f6110 con 0x7f6ff80b9e40 2024-09-21T06:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.942+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fe401a070 con 0x7f6ff80b9e40 2024-09-21T06:34:23.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.942+0000 7f6ff5600640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.943+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f6fe409b0a0 con 0x7f6ff80b9e40 2024-09-21T06:34:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.943+0000 7f6ffda00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:23.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.944+0000 7f6ffda00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0015e0 tx=0x7f6fec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:23.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:23.947+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fe4063340 con 0x7f6ff80b9e40 2024-09-21T06:34:24.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.183+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ff8154ea0 con 0x7f6ff80b9e40 2024-09-21T06:34:24.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.184+0000 7f6ff5600640 1 -- 172.21.15.37:0/1142514584 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f6fe4062c40 con 0x7f6ff80b9e40 2024-09-21T06:34:24.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:24.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:24.187 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 msgr2=0x7f6fd807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0015e0 tx=0x7f6fec009040 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 msgr2=0x7f6ff80b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400d740 tx=0x7f6fe400dc00 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 shutdown_connections 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff80b9e40 0x7f6ff80b92b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ff81f5c30 0x7f6ff814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 --2- 172.21.15.37:0/1142514584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ff806d7f0 0x7f6ff81e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 >> 172.21.15.37:0/1142514584 conn(0x7f6ff80d1930 msgr2=0x7f6ff8160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 shutdown_connections 2024-09-21T06:34:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.186+0000 7f6ffee00640 1 -- 172.21.15.37:0/1142514584 wait complete. 2024-09-21T06:34:24.211 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:24.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 -- 172.21.15.37:0/3373474443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 msgr2=0x7f30680ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 --2- 172.21.15.37:0/3373474443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680ce960 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3060005fd0 tx=0x7f306002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 -- 172.21.15.37:0/3373474443 shutdown_connections 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 --2- 172.21.15.37:0/3373474443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680ce960 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 --2- 172.21.15.37:0/3373474443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f306806e500 0x7f306815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 -- 172.21.15.37:0/3373474443 >> 172.21.15.37:0/3373474443 conn(0x7f3068073400 msgr2=0x7f3068073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 -- 172.21.15.37:0/3373474443 shutdown_connections 2024-09-21T06:34:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.435+0000 7f3067e00640 1 -- 172.21.15.37:0/3373474443 wait complete. 2024-09-21T06:34:24.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.438+0000 7f3067e00640 1 Processor -- start 2024-09-21T06:34:24.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.438+0000 7f3067e00640 1 -- start start 2024-09-21T06:34:24.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 0x7f30681535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:24.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:24.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680ae8b0 0x7f30680aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30680cf3a0 con 0x7f30680afb80 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f30680cf220 con 0x7f30680ae8b0 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.439+0000 7f3067e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f30680cf520 con 0x7f3068073f20 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57100/0 (socket says 172.21.15.37:57100) 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 -- 172.21.15.37:0/2379555145 learned_addr learned my addr 172.21.15.37:0/2379555145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066a00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 0x7f30681535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3067400640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680ae8b0 0x7f30680aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 -- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 msgr2=0x7f30681535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 0x7f30681535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 -- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680ae8b0 msgr2=0x7f30680aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680ae8b0 0x7f30680aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 -- 172.21.15.37:0/2379555145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30680af200 con 0x7f30680afb80 2024-09-21T06:34:24.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066a00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 0x7f30681535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:24.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.440+0000 7f3066000640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f305400c9a0 tx=0x7f305400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:24.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.441+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054007bb0 con 0x7f30680afb80 2024-09-21T06:34:24.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.441+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3054007e40 con 0x7f30680afb80 2024-09-21T06:34:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.441+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305400f6b0 con 0x7f30680afb80 2024-09-21T06:34:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.441+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30680cafc0 con 0x7f30680afb80 2024-09-21T06:34:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.441+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30680cb580 con 0x7f30680afb80 2024-09-21T06:34:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.442+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306815c3c0 con 0x7f30680afb80 2024-09-21T06:34:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.443+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f305400f850 con 0x7f30680afb80 2024-09-21T06:34:24.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.444+0000 7f305ea00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 0x7f304807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:24.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.444+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f305409b5b0 con 0x7f30680afb80 2024-09-21T06:34:24.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.445+0000 7f3066a00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 0x7f304807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:24.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.445+0000 7f3066a00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 0x7f304807b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3058000f60 tx=0x7f3058009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:24.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.449+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f306815c3c0 con 0x7f30680afb80 2024-09-21T06:34:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.718+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3068073d00 con 0x7f30680afb80 2024-09-21T06:34:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.719+0000 7f305ea00640 1 -- 172.21.15.37:0/2379555145 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2014 (secure 0 0 0) 0x7f3068073d00 con 0x7f30680afb80 2024-09-21T06:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":412,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":265932800,"bytes_avail":772828180480,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7336,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-09-21T06:34:19.193583+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 msgr2=0x7f304807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 0x7f304807b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3058000f60 tx=0x7f3058009040 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 msgr2=0x7f30680cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f305400c9a0 tx=0x7f305400ce60 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 shutdown_connections 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3048078b90 0x7f304807b0a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30680afb80 0x7f30680cc5e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680ae8b0 0x7f30680aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 --2- 172.21.15.37:0/2379555145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3068073f20 0x7f30681535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 >> 172.21.15.37:0/2379555145 conn(0x7f3068073400 msgr2=0x7f306806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 shutdown_connections 2024-09-21T06:34:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:24.721+0000 7f3067e00640 1 -- 172.21.15.37:0/2379555145 wait complete. 2024-09-21T06:34:24.746 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:34:25.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.983+0000 7f58f7800640 1 -- 172.21.15.37:0/2393483472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f00ce860 msgr2=0x7f58f015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.983+0000 7f58f7800640 1 --2- 172.21.15.37:0/2393483472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f00ce860 0x7f58f015f5a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005fd0 tx=0x7f58e802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 -- 172.21.15.37:0/2393483472 shutdown_connections 2024-09-21T06:34:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 --2- 172.21.15.37:0/2393483472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00cf930 0x7f58f00ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 --2- 172.21.15.37:0/2393483472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f00ce860 0x7f58f015f5a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 -- 172.21.15.37:0/2393483472 >> 172.21.15.37:0/2393483472 conn(0x7f58f00cca40 msgr2=0x7f58f00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 -- 172.21.15.37:0/2393483472 shutdown_connections 2024-09-21T06:34:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.984+0000 7f58f7800640 1 -- 172.21.15.37:0/2393483472 wait complete. 2024-09-21T06:34:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.986+0000 7f58f7800640 1 Processor -- start 2024-09-21T06:34:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.986+0000 7f58f7800640 1 -- start start 2024-09-21T06:34:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00ce860 0x7f58f00a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f015b9f0 0x7f58f00c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58f006df70 con 0x7f58f00cf930 2024-09-21T06:34:25.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58f006ddf0 con 0x7f58f015b9f0 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.987+0000 7f58f7800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f58f006e0f0 con 0x7f58f00ce860 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57106/0 (socket says 172.21.15.37:57106) 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 -- 172.21.15.37:0/3211378957 learned_addr learned my addr 172.21.15.37:0/3211378957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58f5800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f015b9f0 0x7f58f00c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58f4e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00ce860 0x7f58f00a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 -- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00ce860 msgr2=0x7f58f00a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00ce860 0x7f58f00a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 -- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f015b9f0 msgr2=0x7f58f00c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f015b9f0 0x7f58f00c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.988+0000 7f58e7e00640 1 -- 172.21.15.37:0/3211378957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f0155c40 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58e7e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e9f0 tx=0x7f58d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d800cd80 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d8004030 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f0155230 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8004460 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.989+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01f77f0 con 0x7f58f00cf930 2024-09-21T06:34:25.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.990+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f006e610 con 0x7f58f00cf930 2024-09-21T06:34:25.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.992+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58d80035f0 con 0x7f58f00cf930 2024-09-21T06:34:25.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.992+0000 7f58ed600640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 0x7f58cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.993+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f58d809b7d0 con 0x7f58f00cf930 2024-09-21T06:34:25.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.993+0000 7f58f4e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 0x7f58cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.994+0000 7f58f4e00640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 0x7f58cc07b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f58e0007fd0 tx=0x7f58e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:25.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:25.997+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58d80633a0 con 0x7f58f00cf930 2024-09-21T06:34:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.236+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58f00b15c0 con 0x7f58f00cf930 2024-09-21T06:34:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.236+0000 7f58ed600640 1 -- 172.21.15.37:0/3211378957 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f58d8067dd0 con 0x7f58f00cf930 2024-09-21T06:34:26.239 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:26.239 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 msgr2=0x7f58cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 0x7f58cc07b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f58e0007fd0 tx=0x7f58e0009040 comp rx=0 tx=0).stop 2024-09-21T06:34:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 msgr2=0x7f58f01f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e9f0 tx=0x7f58d800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 shutdown_connections 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f58cc078bc0 0x7f58cc07b0d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f00cf930 0x7f58f01f2d90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58f015b9f0 0x7f58f00c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 --2- 172.21.15.37:0/3211378957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58f00ce860 0x7f58f00a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 >> 172.21.15.37:0/3211378957 conn(0x7f58f00cca40 msgr2=0x7f58f00ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 shutdown_connections 2024-09-21T06:34:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.239+0000 7f58f7800640 1 -- 172.21.15.37:0/3211378957 wait complete. 2024-09-21T06:34:26.269 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:26.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 -- 172.21.15.37:0/949387799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 msgr2=0x7fabd80ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 --2- 172.21.15.37:0/949387799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd80ccce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009830 tx=0x7fabc002a5c0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 -- 172.21.15.37:0/949387799 shutdown_connections 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 --2- 172.21.15.37:0/949387799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd80ccce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 -- 172.21.15.37:0/949387799 >> 172.21.15.37:0/949387799 conn(0x7fabd806f940 msgr2=0x7fabd80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 -- 172.21.15.37:0/949387799 shutdown_connections 2024-09-21T06:34:26.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.494+0000 7fabdd200640 1 -- 172.21.15.37:0/949387799 wait complete. 2024-09-21T06:34:26.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.497+0000 7fabdd200640 1 Processor -- start 2024-09-21T06:34:26.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.497+0000 7fabdd200640 1 -- start start 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabd80b4990 0x7fabd81eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd81eab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fabd806d6f0 con 0x7fabd815cb20 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabd806d570 con 0x7fabd80cc8d0 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabdd200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fabd806d870 con 0x7fabd80b4990 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabd7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabd7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57114/0 (socket says 172.21.15.37:57114) 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.498+0000 7fabd7e00640 1 -- 172.21.15.37:0/4189698826 learned_addr learned my addr 172.21.15.37:0/4189698826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabcea00640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd81eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:26.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7400640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabd80b4990 0x7fabd81eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 -- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabd80b4990 msgr2=0x7fabd81eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabd80b4990 0x7fabd81eb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 -- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 msgr2=0x7fabd81eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd81eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 -- 172.21.15.37:0/4189698826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd81e9ad0 con 0x7fabd815cb20 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabd7e00640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c700 tx=0x7fabd000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.499+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0003b70 con 0x7fabd815cb20 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.500+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd0003d10 con 0x7fabd815cb20 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.500+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd81e9db0 con 0x7fabd815cb20 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.500+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd000aab0 con 0x7fabd815cb20 2024-09-21T06:34:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.500+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd81ea370 con 0x7fabd815cb20 2024-09-21T06:34:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.501+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd806ab30 con 0x7fabd815cb20 2024-09-21T06:34:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.502+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fabd000ac50 con 0x7fabd815cb20 2024-09-21T06:34:26.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.503+0000 7fabcfe00640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 0x7fabb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:26.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.504+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fabd009adc0 con 0x7fabd815cb20 2024-09-21T06:34:26.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.504+0000 7fabd7400640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 0x7fabb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:26.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.505+0000 7fabd7400640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 0x7fabb407b0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fabc80013f0 tx=0x7fabc8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.508+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fabd806ab30 con 0x7fabd815cb20 2024-09-21T06:34:26.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.781+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fabd806dd20 con 0x7fabd815cb20 2024-09-21T06:34:26.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.782+0000 7fabcfe00640 1 -- 172.21.15.37:0/4189698826 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7fabd0062fd0 con 0x7fabd815cb20 2024-09-21T06:34:26.784 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:26.784 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":414,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":265932800,"bytes_avail":772828180480,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-09-21T06:34:25.195448+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 msgr2=0x7fabb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 0x7fabb407b0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fabc80013f0 tx=0x7fabc8009040 comp rx=0 tx=0).stop 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 msgr2=0x7fabd8133260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c700 tx=0x7fabd000cbc0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 shutdown_connections 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fabb4078b90 0x7fabb407b0a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabd815cb20 0x7fabd8133260 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd80cc8d0 0x7fabd81eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 --2- 172.21.15.37:0/4189698826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabd80b4990 0x7fabd81eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:26.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 >> 172.21.15.37:0/4189698826 conn(0x7fabd806f940 msgr2=0x7fabd80cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:26.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 shutdown_connections 2024-09-21T06:34:26.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:26.784+0000 7fabdd200640 1 -- 172.21.15.37:0/4189698826 wait complete. 2024-09-21T06:34:26.808 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:34:27.026 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:34:27.024+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:34:27.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:28.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 -- 172.21.15.37:0/1452595142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac06e500 msgr2=0x7f03ac0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 --2- 172.21.15.37:0/1452595142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac06e500 0x7f03ac0cc020 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f03ac076360 tx=0x7f03a402aac0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 -- 172.21.15.37:0/1452595142 shutdown_connections 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 --2- 172.21.15.37:0/1452595142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac06e500 0x7f03ac0cc020 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 --2- 172.21.15.37:0/1452595142 >> v2:172.21.15.150:3300/0 conn(0x7f03ac074cb0 0x7f03ac073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 -- 172.21.15.37:0/1452595142 >> 172.21.15.37:0/1452595142 conn(0x7f03ac0cca40 msgr2=0x7f03ac0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 -- 172.21.15.37:0/1452595142 shutdown_connections 2024-09-21T06:34:28.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.045+0000 7f03b4000640 1 -- 172.21.15.37:0/1452595142 wait complete. 2024-09-21T06:34:28.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.048+0000 7f03b4000640 1 Processor -- start 2024-09-21T06:34:28.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.048+0000 7f03b4000640 1 -- start start 2024-09-21T06:34:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03ac0cbc10 0x7f03ac09cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03ac09d2d0 0x7f03ac1e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac06f310 con 0x7f03ac074cb0 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac06f190 con 0x7f03ac09d2d0 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b4000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac06f490 con 0x7f03ac0cbc10 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.049+0000 7f03b1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57124/0 (socket says 172.21.15.37:57124) 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 -- 172.21.15.37:0/3819599033 learned_addr learned my addr 172.21.15.37:0/3819599033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b2000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03ac09d2d0 0x7f03ac1e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b0c00640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03ac0cbc10 0x7f03ac09cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 -- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03ac0cbc10 msgr2=0x7f03ac09cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03ac0cbc10 0x7f03ac09cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 -- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03ac09d2d0 msgr2=0x7f03ac1e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03ac09d2d0 0x7f03ac1e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 -- 172.21.15.37:0/3819599033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac14aaf0 con 0x7f03ac074cb0 2024-09-21T06:34:28.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.050+0000 7f03b1600640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009fd0 tx=0x7f03a000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.051+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000ccc0 con 0x7f03ac074cb0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.051+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03a000ee70 con 0x7f03ac074cb0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.051+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac0c7ed0 con 0x7f03ac074cb0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.051+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000ce60 con 0x7f03ac074cb0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.051+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1e7c20 con 0x7f03ac074cb0 2024-09-21T06:34:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.052+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac06f940 con 0x7f03ac074cb0 2024-09-21T06:34:28.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.053+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03a0004280 con 0x7f03ac074cb0 2024-09-21T06:34:28.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.055+0000 7f03a9600640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.055+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f03a009b8b0 con 0x7f03ac074cb0 2024-09-21T06:34:28.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.055+0000 7f03b0c00640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.056+0000 7f03b0c00640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 0x7f038c07b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f03ac151200 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.060+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03a0063ac0 con 0x7f03ac074cb0 2024-09-21T06:34:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.299+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03ac15e690 con 0x7f03ac074cb0 2024-09-21T06:34:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.300+0000 7f03a9600640 1 -- 172.21.15.37:0/3819599033 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f03a00633c0 con 0x7f03ac074cb0 2024-09-21T06:34:28.302 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:28.302 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 msgr2=0x7f038c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 0x7f038c07b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f03ac151200 tx=0x7f0398009040 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 msgr2=0x7f03ac1e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009fd0 tx=0x7f03a000ec20 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 shutdown_connections 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03ac09d2d0 0x7f03ac1e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03ac0cbc10 0x7f03ac09cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 --2- 172.21.15.37:0/3819599033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03ac074cb0 0x7f03ac1e4d60 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 >> 172.21.15.37:0/3819599033 conn(0x7f03ac0cca40 msgr2=0x7f03ac06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 shutdown_connections 2024-09-21T06:34:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.302+0000 7f03b4000640 1 -- 172.21.15.37:0/3819599033 wait complete. 2024-09-21T06:34:28.331 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:28.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 -- 172.21.15.37:0/1482846171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3828004800 msgr2=0x7f3828004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 --2- 172.21.15.37:0/1482846171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3828004800 0x7f3828004c10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f382c0016c0 tx=0x7f3828005830 comp rx=0 tx=0).stop 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 -- 172.21.15.37:0/1482846171 shutdown_connections 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 --2- 172.21.15.37:0/1482846171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3828004800 0x7f3828004c10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 -- 172.21.15.37:0/1482846171 >> 172.21.15.37:0/1482846171 conn(0x7f38440cca40 msgr2=0x7f38440cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 -- 172.21.15.37:0/1482846171 shutdown_connections 2024-09-21T06:34:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.556+0000 7f384aa00640 1 -- 172.21.15.37:0/1482846171 wait complete. 2024-09-21T06:34:28.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.559+0000 7f384aa00640 1 Processor -- start 2024-09-21T06:34:28.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.559+0000 7f384aa00640 1 -- start start 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3828004800 0x7f38440c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38440ca000 0x7f38441f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f384415e7c0 con 0x7f38440ce530 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38440cb960 con 0x7f38440ca000 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f384aa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f384415e940 con 0x7f3828004800 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f3848a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f3848a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57140/0 (socket says 172.21.15.37:57140) 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.560+0000 7f3848a00640 1 -- 172.21.15.37:0/2728365957 learned_addr learned my addr 172.21.15.37:0/2728365957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3843e00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3828004800 0x7f38440c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f383b400640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38440ca000 0x7f38441f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 -- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3828004800 msgr2=0x7f38440c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3828004800 0x7f38440c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 -- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38440ca000 msgr2=0x7f38441f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38440ca000 0x7f38441f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 -- 172.21.15.37:0/2728365957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38440cee80 con 0x7f38440ce530 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.561+0000 7f3848a00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3844072d20 tx=0x7f383c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.562+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c003970 con 0x7f38440ce530 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.562+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f383c015070 con 0x7f38440ce530 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.562+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c004030 con 0x7f38440ce530 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.562+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441f8940 con 0x7f38440ce530 2024-09-21T06:34:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.562+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441f8f00 con 0x7f38440ce530 2024-09-21T06:34:28.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.563+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384406ddf0 con 0x7f38440ce530 2024-09-21T06:34:28.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.564+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f383c0041d0 con 0x7f38440ce530 2024-09-21T06:34:28.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.565+0000 7f3840c00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 0x7f382007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.565+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f383c09b9f0 con 0x7f38440ce530 2024-09-21T06:34:28.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.565+0000 7f3843e00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 0x7f382007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.566+0000 7f3843e00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 0x7f382007b0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3834006c10 tx=0x7f38340067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.569+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38440cee80 con 0x7f38440ce530 2024-09-21T06:34:28.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.839+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f38440b5f40 con 0x7f38440ce530 2024-09-21T06:34:28.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.839+0000 7f3840c00640 1 -- 172.21.15.37:0/2728365957 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7f383c063640 con 0x7f38440ce530 2024-09-21T06:34:28.841 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:28.842 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":416,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":265932800,"bytes_avail":772828180480,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-09-21T06:34:25.195448+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 msgr2=0x7f382007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 0x7f382007b0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3834006c10 tx=0x7f38340067a0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 msgr2=0x7f38440ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3844072d20 tx=0x7f383c010f40 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 shutdown_connections 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3820078bc0 0x7f382007b0d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38440ce530 0x7f38440ce940 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38440ca000 0x7f38441f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 --2- 172.21.15.37:0/2728365957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3828004800 0x7f38440c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.841+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 >> 172.21.15.37:0/2728365957 conn(0x7f38440cca40 msgr2=0x7f38441620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.842+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 shutdown_connections 2024-09-21T06:34:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:28.842+0000 7f384aa00640 1 -- 172.21.15.37:0/2728365957 wait complete. 2024-09-21T06:34:28.866 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:34:29.867 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:30.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccce000640 1 -- 172.21.15.37:0/3177736396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80047a0 con 0x7fccd00cbc10 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 -- 172.21.15.37:0/3177736396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd00cbc10 msgr2=0x7fccd00cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 --2- 172.21.15.37:0/3177736396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd00cbc10 0x7fccd00cc020 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fccd0076390 tx=0x7fccb802a560 comp rx=0 tx=0).stop 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 -- 172.21.15.37:0/3177736396 shutdown_connections 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 --2- 172.21.15.37:0/3177736396 >> v2:172.21.15.150:3300/0 conn(0x7fccd006ddf0 0x7fccd006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 --2- 172.21.15.37:0/3177736396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd00cbc10 0x7fccd00cc020 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 --2- 172.21.15.37:0/3177736396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd0074cb0 0x7fccd0073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 -- 172.21.15.37:0/3177736396 >> 172.21.15.37:0/3177736396 conn(0x7fccd00cca40 msgr2=0x7fccd00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 -- 172.21.15.37:0/3177736396 shutdown_connections 2024-09-21T06:34:30.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.105+0000 7fccd5e00640 1 -- 172.21.15.37:0/3177736396 wait complete. 2024-09-21T06:34:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.108+0000 7fccd5e00640 1 Processor -- start 2024-09-21T06:34:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.108+0000 7fccd5e00640 1 -- start start 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 0x7fccd0154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd00cbc10 0x7fccd00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccd006f170 con 0x7fccd006ddf0 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd006eff0 con 0x7fccd0074cb0 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fccd5e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fccd006f2f0 con 0x7fccd00cbc10 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fcccea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.109+0000 7fcccea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57158/0 (socket says 172.21.15.37:57158) 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 -- 172.21.15.37:0/2707501314 learned_addr learned my addr 172.21.15.37:0/2707501314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fccc7e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 0x7fccd0154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccf400640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd00cbc10 0x7fccd00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 -- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd00cbc10 msgr2=0x7fccd00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd00cbc10 0x7fccd00b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 -- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 msgr2=0x7fccd0154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 0x7fccd0154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 -- 172.21.15.37:0/2707501314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd0155500 con 0x7fccd006ddf0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fccc7e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 0x7fccd0154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.110+0000 7fcccea00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b410 tx=0x7fccc000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.111+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0015020 con 0x7fccd006ddf0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.111+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccc0011920 con 0x7fccd006ddf0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.111+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0011d70 con 0x7fccd006ddf0 2024-09-21T06:34:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.111+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd00b1550 con 0x7fccd006ddf0 2024-09-21T06:34:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.111+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd00b17f0 con 0x7fccd006ddf0 2024-09-21T06:34:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.112+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd0051040 con 0x7fccd006ddf0 2024-09-21T06:34:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.113+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fccc0018020 con 0x7fccd006ddf0 2024-09-21T06:34:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.114+0000 7fccc7400640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 0x7fccac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.114+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fccc009ae70 con 0x7fccd006ddf0 2024-09-21T06:34:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.114+0000 7fccc7e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 0x7fccac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.115+0000 7fccc7e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 0x7fccac07b0a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fccb8033000 tx=0x7fccb8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.118+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fccc001f020 con 0x7fccd006ddf0 2024-09-21T06:34:30.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.357+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd015b2a0 con 0x7fccd006ddf0 2024-09-21T06:34:30.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.357+0000 7fccc7400640 1 -- 172.21.15.37:0/2707501314 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7fccd0155500 con 0x7fccd006ddf0 2024-09-21T06:34:30.360 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:30.360 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:30.360 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:30.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 msgr2=0x7fccac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 0x7fccac07b0a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fccb8033000 tx=0x7fccb8004730 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 msgr2=0x7fccd00cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b410 tx=0x7fccc000b8d0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 shutdown_connections 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fccac078b90 0x7fccac07b0a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd00cbc10 0x7fccd00b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0074cb0 0x7fccd0154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 --2- 172.21.15.37:0/2707501314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccd006ddf0 0x7fccd00cb440 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 >> 172.21.15.37:0/2707501314 conn(0x7fccd00cca40 msgr2=0x7fccd0070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 shutdown_connections 2024-09-21T06:34:30.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.360+0000 7fccd5e00640 1 -- 172.21.15.37:0/2707501314 wait complete. 2024-09-21T06:34:30.385 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:30.385 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 -- 172.21.15.37:0/1804029274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f718406ddf0 msgr2=0x7f7184154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 --2- 172.21.15.37:0/1804029274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f718406ddf0 0x7f7184154f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f717c009be0 tx=0x7f717c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 -- 172.21.15.37:0/1804029274 shutdown_connections 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 --2- 172.21.15.37:0/1804029274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f718406ddf0 0x7f7184154f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 -- 172.21.15.37:0/1804029274 >> 172.21.15.37:0/1804029274 conn(0x7f71840d4360 msgr2=0x7f71840d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.610+0000 7f7183200640 1 -- 172.21.15.37:0/1804029274 shutdown_connections 2024-09-21T06:34:30.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.611+0000 7f7183200640 1 -- 172.21.15.37:0/1804029274 wait complete. 2024-09-21T06:34:30.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.613+0000 7f7183200640 1 Processor -- start 2024-09-21T06:34:30.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.613+0000 7f7183200640 1 -- start start 2024-09-21T06:34:30.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 0x7f71841f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71841622a0 0x7f71841f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7184155670 con 0x7f71840cede0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7184163130 con 0x7f718406ddf0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7183200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f71841557f0 con 0x7f71841622a0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.614+0000 7f7179400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57166/0 (socket says 172.21.15.37:57166) 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 -- 172.21.15.37:0/2784019165 learned_addr learned my addr 172.21.15.37:0/2784019165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7182800640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71841622a0 0x7f71841f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7181e00640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 0x7f71841f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 -- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71841622a0 msgr2=0x7f71841f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71841622a0 0x7f71841f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 -- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 msgr2=0x7f71841f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 0x7f71841f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 -- 172.21.15.37:0/2784019165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718415ece0 con 0x7f71840cede0 2024-09-21T06:34:30.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7181e00640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 0x7f71841f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:30.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.615+0000 7f7179400640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f716c00c4f0 tx=0x7f716c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:30.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.616+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c016020 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.616+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f716c012920 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.616+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718415efc0 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.616+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c012d10 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.616+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7184157040 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.617+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7184154f20 con 0x7f71840cede0 2024-09-21T06:34:30.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.618+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f716c019020 con 0x7f71840cede0 2024-09-21T06:34:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.619+0000 7f717aa00640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 0x7f716007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.620+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f716c09b100 con 0x7f71840cede0 2024-09-21T06:34:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.620+0000 7f7181e00640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 0x7f716007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.621+0000 7f7181e00640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 0x7f716007b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7174001120 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.624+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f716c020020 con 0x7f71840cede0 2024-09-21T06:34:30.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.899+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7184076180 con 0x7f71840cede0 2024-09-21T06:34:30.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.899+0000 7f717aa00640 1 -- 172.21.15.37:0/2784019165 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1979 (secure 0 0 0) 0x7f716c063310 con 0x7f71840cede0 2024-09-21T06:34:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:30.902 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":418,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-09-21T06:34:29.196546+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 msgr2=0x7f716007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 0x7f716007b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7174001120 tx=0x7f7174009040 comp rx=0 tx=0).stop 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 msgr2=0x7f71841f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f716c00c4f0 tx=0x7f716c00c9b0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 shutdown_connections 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7160078b90 0x7f716007b0a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71840cede0 0x7f71841f8cb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71841622a0 0x7f71841f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 --2- 172.21.15.37:0/2784019165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f718406ddf0 0x7f71841f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:30.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 >> 172.21.15.37:0/2784019165 conn(0x7f71840d4360 msgr2=0x7f718415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:30.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 shutdown_connections 2024-09-21T06:34:30.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:30.902+0000 7f7183200640 1 -- 172.21.15.37:0/2784019165 wait complete. 2024-09-21T06:34:30.928 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:34:31.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 -- 172.21.15.37:0/119061545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18005350 msgr2=0x7efd1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 --2- 172.21.15.37:0/119061545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18005350 0x7efd1800b9c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7efd28005fd0 tx=0x7efd18005830 comp rx=0 tx=0).stop 2024-09-21T06:34:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 -- 172.21.15.37:0/119061545 shutdown_connections 2024-09-21T06:34:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 --2- 172.21.15.37:0/119061545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18005350 0x7efd1800b9c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 -- 172.21.15.37:0/119061545 >> 172.21.15.37:0/119061545 conn(0x7efd3006f940 msgr2=0x7efd300b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 -- 172.21.15.37:0/119061545 shutdown_connections 2024-09-21T06:34:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.170+0000 7efd36e00640 1 -- 172.21.15.37:0/119061545 wait complete. 2024-09-21T06:34:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.172+0000 7efd36e00640 1 Processor -- start 2024-09-21T06:34:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.172+0000 7efd36e00640 1 -- start start 2024-09-21T06:34:32.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd18005350 0x7efd3013b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3013c560 0x7efd30140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd30160dd0 con 0x7efd30141100 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd30160c50 con 0x7efd18005350 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7efd30160f50 con 0x7efd3013c560 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57174/0 (socket says 172.21.15.37:57174) 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 -- 172.21.15.37:0/2727981872 learned_addr learned my addr 172.21.15.37:0/2727981872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd35000640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3013c560 0x7efd30140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd35a00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd18005350 0x7efd3013b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 -- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3013c560 msgr2=0x7efd30140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3013c560 0x7efd30140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 -- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd18005350 msgr2=0x7efd3013b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd18005350 0x7efd3013b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.174+0000 7efd36400640 1 -- 172.21.15.37:0/2727981872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3013fe00 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd36400640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e20 tx=0x7efd28004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28009200 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd28041070 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28009ad0 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd301400e0 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.175+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301406a0 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.176+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd300ccbc0 con 0x7efd30141100 2024-09-21T06:34:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.177+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd28046460 con 0x7efd30141100 2024-09-21T06:34:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.179+0000 7efd2d600640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 0x7efd1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.179+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7efd280bbff0 con 0x7efd30141100 2024-09-21T06:34:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.179+0000 7efd35a00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 0x7efd1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.180+0000 7efd35a00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 0x7efd1007b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efd24006600 tx=0x7efd24006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:32.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.184+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd28084220 con 0x7efd30141100 2024-09-21T06:34:32.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.417+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd300cc0e0 con 0x7efd30141100 2024-09-21T06:34:32.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.417+0000 7efd2d600640 1 -- 172.21.15.37:0/2727981872 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7efd28083b20 con 0x7efd30141100 2024-09-21T06:34:32.420 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:32.420 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:32.420 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:32.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 msgr2=0x7efd1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 0x7efd1007b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efd24006600 tx=0x7efd24006b70 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 msgr2=0x7efd301f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e20 tx=0x7efd28004250 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 shutdown_connections 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7efd10078b90 0x7efd1007b0a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd30141100 0x7efd301f2df0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3013c560 0x7efd30140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 --2- 172.21.15.37:0/2727981872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd18005350 0x7efd3013b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 >> 172.21.15.37:0/2727981872 conn(0x7efd3006f940 msgr2=0x7efd300b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 shutdown_connections 2024-09-21T06:34:32.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.420+0000 7efd36e00640 1 -- 172.21.15.37:0/2727981872 wait complete. 2024-09-21T06:34:32.445 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:32.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 -- 172.21.15.37:0/1422052819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b78004800 msgr2=0x7f1b78004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 --2- 172.21.15.37:0/1422052819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b78004800 0x7f1b78004c10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1b94072bb0 tx=0x7f1b78005830 comp rx=0 tx=0).stop 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b93e00640 1 -- 172.21.15.37:0/1422052819 reap_dead start 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 -- 172.21.15.37:0/1422052819 shutdown_connections 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 -- 172.21.15.37:0/1422052819 >> 172.21.15.37:0/1422052819 conn(0x7f1b9406f940 msgr2=0x7f1b940b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 -- 172.21.15.37:0/1422052819 shutdown_connections 2024-09-21T06:34:32.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.670+0000 7f1b99200640 1 -- 172.21.15.37:0/1422052819 wait complete. 2024-09-21T06:34:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.672+0000 7f1b99200640 1 Processor -- start 2024-09-21T06:34:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.673+0000 7f1b99200640 1 -- start start 2024-09-21T06:34:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 0x7f1b941555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b941f4110 0x7f1b941f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940b4810 con 0x7f1b941f6390 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940b4690 con 0x7f1b78004800 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b99200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1b940b4990 con 0x7f1b941f4110 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b8be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b8be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57190/0 (socket says 172.21.15.37:57190) 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b8be00640 1 -- 172.21.15.37:0/3513742992 learned_addr learned my addr 172.21.15.37:0/3513742992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b93400640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 0x7f1b941555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.674+0000 7f1b93e00640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b941f4110 0x7f1b941f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 -- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b941f4110 msgr2=0x7f1b941f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b941f4110 0x7f1b941f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 -- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 msgr2=0x7f1b941555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 0x7f1b941555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 -- 172.21.15.37:0/3513742992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b94153f20 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b93400640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 0x7f1b941555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8be00640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009c10 tx=0x7f1b7c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c038050 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7c039040 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b94154200 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c004270 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.675+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941547c0 con 0x7f1b941f6390 2024-09-21T06:34:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.677+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b94153c90 con 0x7f1b941f6390 2024-09-21T06:34:32.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.678+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b7c033970 con 0x7f1b941f6390 2024-09-21T06:34:32.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.679+0000 7f1b8b400640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 0x7f1b7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:32.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.680+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f1b7c0bbe00 con 0x7f1b941f6390 2024-09-21T06:34:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.680+0000 7f1b93400640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 0x7f1b7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.681+0000 7f1b93400640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 0x7f1b7007b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009c60 tx=0x7f1b84005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.684+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b7c0840a0 con 0x7f1b941f6390 2024-09-21T06:34:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.954+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1b940b4f20 con 0x7f1b941f6390 2024-09-21T06:34:32.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.955+0000 7f1b8b400640 1 -- 172.21.15.37:0/3513742992 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1959 (secure 0 0 0) 0x7f1b7c0839a0 con 0x7f1b941f6390 2024-09-21T06:34:32.957 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:32.957 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":420,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-09-21T06:34:29.196546+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.957+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 msgr2=0x7f1b7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.957+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 0x7f1b7007b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009c60 tx=0x7f1b84005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.957+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 msgr2=0x7f1b94155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.957+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009c10 tx=0x7f1b7c033670 comp rx=0 tx=0).stop 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 shutdown_connections 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1b70078bc0 0x7f1b7007b0d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b941f6390 0x7f1b94155b00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b941f4110 0x7f1b941f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 --2- 172.21.15.37:0/3513742992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b78004800 0x7f1b941555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 >> 172.21.15.37:0/3513742992 conn(0x7f1b9406f940 msgr2=0x7f1b9415edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 shutdown_connections 2024-09-21T06:34:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:32.958+0000 7f1b99200640 1 -- 172.21.15.37:0/3513742992 wait complete. 2024-09-21T06:34:32.982 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:34:33.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 -- 172.21.15.37:0/1215412763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 msgr2=0x7fd8800b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 --2- 172.21.15.37:0/1215412763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800b4da0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd8800afb10 tx=0x7fd87802aac0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 -- 172.21.15.37:0/1215412763 shutdown_connections 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 --2- 172.21.15.37:0/1215412763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800b4da0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 --2- 172.21.15.37:0/1215412763 >> v2:172.21.15.150:3300/0 conn(0x7fd8800cc8d0 0x7fd8800ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 -- 172.21.15.37:0/1215412763 >> 172.21.15.37:0/1215412763 conn(0x7fd88006f940 msgr2=0x7fd8800b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.220+0000 7fd885000640 1 -- 172.21.15.37:0/1215412763 shutdown_connections 2024-09-21T06:34:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.221+0000 7fd885000640 1 -- 172.21.15.37:0/1215412763 wait complete. 2024-09-21T06:34:34.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.223+0000 7fd885000640 1 Processor -- start 2024-09-21T06:34:34.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.223+0000 7fd885000640 1 -- start start 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 0x7fd8801454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 0x7fd8800c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd88006d600 con 0x7fd8800b4990 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd88006d480 con 0x7fd88015cb20 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.224+0000 7fd885000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd88006d780 con 0x7fd8800cc8d0 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53270/0 (socket says 172.21.15.37:53270) 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 -- 172.21.15.37:0/4044643625 learned_addr learned my addr 172.21.15.37:0/4044643625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87fe00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 0x7fd8800c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87ea00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 0x7fd8801454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 -- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 msgr2=0x7fd8801454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 0x7fd8801454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 -- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 msgr2=0x7fd8800c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 0x7fd8800c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 -- 172.21.15.37:0/4044643625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8800c5c00 con 0x7fd8800b4990 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87ea00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 0x7fd8801454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87fe00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 0x7fd8800c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.225+0000 7fd87f400640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c3d0 tx=0x7fd87000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.226+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870017070 con 0x7fd8800b4990 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.226+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8700053c0 con 0x7fd8800b4990 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.226+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870003850 con 0x7fd8800b4990 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.226+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8800c8c50 con 0x7fd8800b4990 2024-09-21T06:34:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.226+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801ea440 con 0x7fd8800b4990 2024-09-21T06:34:34.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.227+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd880074320 con 0x7fd8800b4990 2024-09-21T06:34:34.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.228+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd87001a070 con 0x7fd8800b4990 2024-09-21T06:34:34.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.229+0000 7fd877400640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 0x7fd86007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.229+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd87009a350 con 0x7fd8800b4990 2024-09-21T06:34:34.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.230+0000 7fd87ea00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 0x7fd86007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.231+0000 7fd87ea00640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 0x7fd86007b0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005fd0 tx=0x7fd86c005ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:34.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.234+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd880074320 con 0x7fd8800b4990 2024-09-21T06:34:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.468+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8800ce2f0 con 0x7fd8800b4990 2024-09-21T06:34:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.468+0000 7fd877400640 1 -- 172.21.15.37:0/4044643625 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7fd8800ce2f0 con 0x7fd8800b4990 2024-09-21T06:34:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:34.471 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 msgr2=0x7fd86007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 0x7fd86007b0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005fd0 tx=0x7fd86c005ea0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 msgr2=0x7fd8800c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c3d0 tx=0x7fd87000c890 comp rx=0 tx=0).stop 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 shutdown_connections 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd860078bc0 0x7fd86007b0d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd88015cb20 0x7fd8800c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8800cc8d0 0x7fd8801454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 --2- 172.21.15.37:0/4044643625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8800b4990 0x7fd8800c6420 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 >> 172.21.15.37:0/4044643625 conn(0x7fd88006f940 msgr2=0x7fd880074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 shutdown_connections 2024-09-21T06:34:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.471+0000 7fd885000640 1 -- 172.21.15.37:0/4044643625 wait complete. 2024-09-21T06:34:34.496 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:34.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 -- 172.21.15.37:0/345623667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 msgr2=0x7f7ea400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 --2- 172.21.15.37:0/345623667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ea400b9c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001d90 tx=0x7f7ea4005830 comp rx=0 tx=0).stop 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 -- 172.21.15.37:0/345623667 shutdown_connections 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 --2- 172.21.15.37:0/345623667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ea400b9c0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 -- 172.21.15.37:0/345623667 >> 172.21.15.37:0/345623667 conn(0x7f7ec00cca40 msgr2=0x7f7ec00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 -- 172.21.15.37:0/345623667 shutdown_connections 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.722+0000 7f7ec5600640 1 -- 172.21.15.37:0/345623667 wait complete. 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.724+0000 7f7ec5600640 1 Processor -- start 2024-09-21T06:34:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.724+0000 7f7ec5600640 1 -- start start 2024-09-21T06:34:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 0x7f7ec00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ec01f6af0 0x7f7ec01f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0070e80 con 0x7f7ea4005250 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0071000 con 0x7f7ec01f6af0 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ec5600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec006ddf0 con 0x7f7ec00b51c0 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53292/0 (socket says 172.21.15.37:53292) 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 -- 172.21.15.37:0/942519169 learned_addr learned my addr 172.21.15.37:0/942519169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7eb7e00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 0x7f7ec00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebf400640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ec01f6af0 0x7f7ec01f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 -- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 msgr2=0x7f7ec00b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 0x7f7ec00b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 -- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ec01f6af0 msgr2=0x7f7ec01f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ec01f6af0 0x7f7ec01f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7ebea00640 1 -- 172.21.15.37:0/942519169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec01f5a30 con 0x7f7ea4005250 2024-09-21T06:34:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.726+0000 7f7eb7e00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 0x7f7ec00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7ebea00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e660 tx=0x7f7eb000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0015020 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eb000cb30 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0011460 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec01f5d10 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.727+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec01f62d0 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.728+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec006f710 con 0x7f7ea4005250 2024-09-21T06:34:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.729+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7eb0018020 con 0x7f7ea4005250 2024-09-21T06:34:34.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.730+0000 7f7eb6a00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 0x7f7e9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:34.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.730+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f7eb009b1b0 con 0x7f7ea4005250 2024-09-21T06:34:34.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.731+0000 7f7eb7e00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 0x7f7e9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:34.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.732+0000 7f7eb7e00640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 0x7f7e9c07b0d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00c76e0 tx=0x7f7ea8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:34.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:34.735+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eb001f030 con 0x7f7ea4005250 2024-09-21T06:34:35.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.007+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7ec00b5f40 con 0x7f7ea4005250 2024-09-21T06:34:35.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.008+0000 7f7eb6a00640 1 -- 172.21.15.37:0/942519169 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1913 (secure 0 0 0) 0x7f7eb0063450 con 0x7f7ea4005250 2024-09-21T06:34:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:35.010 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":423,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":265945088,"bytes_avail":772828168192,"bytes_total":773094113280,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-09-21T06:34:33.197917+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-21T06:34:35.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 msgr2=0x7f7e9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 0x7f7e9c07b0d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00c76e0 tx=0x7f7ea8005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 msgr2=0x7f7ec00c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e660 tx=0x7f7eb000eb20 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 shutdown_connections 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7e9c078bc0 0x7f7e9c07b0d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ec01f6af0 0x7f7ec01f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ec00b51c0 0x7f7ec00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 --2- 172.21.15.37:0/942519169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea4005250 0x7f7ec00c9d00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 >> 172.21.15.37:0/942519169 conn(0x7f7ec00cca40 msgr2=0x7f7ec00a0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 shutdown_connections 2024-09-21T06:34:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:35.010+0000 7f7ec5600640 1 -- 172.21.15.37:0/942519169 wait complete. 2024-09-21T06:34:35.036 INFO:tasks.ceph.ceph_manager.ceph:{'d': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:34:36.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 -- 172.21.15.37:0/3809210452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef840cefa0 msgr2=0x7fef840ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 --2- 172.21.15.37:0/3809210452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef840cefa0 0x7fef840ce320 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009830 tx=0x7fef6c02a510 comp rx=0 tx=0).stop 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 -- 172.21.15.37:0/3809210452 shutdown_connections 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 --2- 172.21.15.37:0/3809210452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef840cefa0 0x7fef840ce320 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 --2- 172.21.15.37:0/3809210452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef8415b9f0 0x7fef840cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 -- 172.21.15.37:0/3809210452 >> 172.21.15.37:0/3809210452 conn(0x7fef840cca40 msgr2=0x7fef840cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 -- 172.21.15.37:0/3809210452 shutdown_connections 2024-09-21T06:34:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.273+0000 7fef8b200640 1 -- 172.21.15.37:0/3809210452 wait complete. 2024-09-21T06:34:36.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.275+0000 7fef8b200640 1 Processor -- start 2024-09-21T06:34:36.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.276+0000 7fef8b200640 1 -- start start 2024-09-21T06:34:36.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef840ce860 0x7fef841584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef840cefa0 0x7fef84155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef8406df70 con 0x7fef8415b9f0 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef8406ddf0 con 0x7fef840ce860 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef8b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fef8406e0f0 con 0x7fef840cefa0 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef89200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef89200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53308/0 (socket says 172.21.15.37:53308) 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.277+0000 7fef89200640 1 -- 172.21.15.37:0/1819574879 learned_addr learned my addr 172.21.15.37:0/1819574879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef83e00640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef840ce860 0x7fef841584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef7b400640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef840cefa0 0x7fef84155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 -- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef840cefa0 msgr2=0x7fef84155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef840cefa0 0x7fef84155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 -- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef840ce860 msgr2=0x7fef841584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef840ce860 0x7fef841584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 -- 172.21.15.37:0/1819574879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef841f3f80 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef89200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00caa0 tx=0x7fef7c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c016020 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef7c00ac30 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.278+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef84152a80 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.279+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00ae60 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.279+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef84153000 con 0x7fef8415b9f0 2024-09-21T06:34:36.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.280+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8406e5c0 con 0x7fef8415b9f0 2024-09-21T06:34:36.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.281+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef7c019020 con 0x7fef8415b9f0 2024-09-21T06:34:36.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.282+0000 7fef80c00640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 0x7fef6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.282+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fef7c09b180 con 0x7fef8415b9f0 2024-09-21T06:34:36.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.282+0000 7fef83e00640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 0x7fef6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.283+0000 7fef83e00640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 0x7fef6007b0a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fef74009ac0 tx=0x7fef74005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.286+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef7c020030 con 0x7fef8415b9f0 2024-09-21T06:34:36.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.521+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef840696c0 con 0x7fef8415b9f0 2024-09-21T06:34:36.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.522+0000 7fef80c00640 1 -- 172.21.15.37:0/1819574879 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7fef7c063420 con 0x7fef8415b9f0 2024-09-21T06:34:36.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:36.524 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:36.524 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 msgr2=0x7fef6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 0x7fef6007b0a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fef74009ac0 tx=0x7fef74005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 msgr2=0x7fef84155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00caa0 tx=0x7fef7c00cf60 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 shutdown_connections 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef60078b90 0x7fef6007b0a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef8415b9f0 0x7fef84155580 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef840cefa0 0x7fef84155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 --2- 172.21.15.37:0/1819574879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef840ce860 0x7fef841584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 >> 172.21.15.37:0/1819574879 conn(0x7fef840cca40 msgr2=0x7fef8406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.524+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 shutdown_connections 2024-09-21T06:34:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.525+0000 7fef8b200640 1 -- 172.21.15.37:0/1819574879 wait complete. 2024-09-21T06:34:36.552 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'a', 'b'] 2024-09-21T06:34:36.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.777+0000 7fc709800640 1 -- 172.21.15.37:0/53696616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8004800 msgr2=0x7fc6e8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.777+0000 7fc709800640 1 --2- 172.21.15.37:0/53696616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8004800 0x7fc6e8004c10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000c470 tx=0x7fc6e8005830 comp rx=0 tx=0).stop 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.778+0000 7fc709800640 1 -- 172.21.15.37:0/53696616 shutdown_connections 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.778+0000 7fc709800640 1 --2- 172.21.15.37:0/53696616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8004800 0x7fc6e8004c10 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.778+0000 7fc709800640 1 -- 172.21.15.37:0/53696616 >> 172.21.15.37:0/53696616 conn(0x7fc70406f940 msgr2=0x7fc7040b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.778+0000 7fc709800640 1 -- 172.21.15.37:0/53696616 shutdown_connections 2024-09-21T06:34:36.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.778+0000 7fc709800640 1 -- 172.21.15.37:0/53696616 wait complete. 2024-09-21T06:34:36.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.780+0000 7fc709800640 1 Processor -- start 2024-09-21T06:34:36.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.780+0000 7fc709800640 1 -- start start 2024-09-21T06:34:36.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e8004800 0x7fc7041fa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7040aece0 0x7fc7040a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc70415c530 con 0x7fc7040ab580 2024-09-21T06:34:36.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc70415c3b0 con 0x7fc6e8004800 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc709800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc70415c6b0 con 0x7fc7040aece0 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc703400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc703400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53322/0 (socket says 172.21.15.37:53322) 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc703400640 1 -- 172.21.15.37:0/2695067222 learned_addr learned my addr 172.21.15.37:0/2695067222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc703e00640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e8004800 0x7fc7041fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.782+0000 7fc708e00640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7040aece0 0x7fc7040a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 -- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7040aece0 msgr2=0x7fc7040a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7040aece0 0x7fc7040a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 -- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e8004800 msgr2=0x7fc7041fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e8004800 0x7fc7041fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 -- 172.21.15.37:0/2695067222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7041fac70 con 0x7fc7040ab580 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc703400640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc6f002b2d0 tx=0x7fc6f00060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0042070 con 0x7fc7040ab580 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6f00035f0 con 0x7fc7040ab580 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0039070 con 0x7fc7040ab580 2024-09-21T06:34:36.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7040abb20 con 0x7fc7040ab580 2024-09-21T06:34:36.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.783+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7040afd20 con 0x7fc7040ab580 2024-09-21T06:34:36.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.784+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7040b3570 con 0x7fc7040ab580 2024-09-21T06:34:36.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.786+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6f0003a40 con 0x7fc7040ab580 2024-09-21T06:34:36.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.787+0000 7fc6fbe00640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 0x7fc6e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:36.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.787+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc6f00ba8a0 con 0x7fc7040ab580 2024-09-21T06:34:36.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.787+0000 7fc703e00640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 0x7fc6e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:36.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.788+0000 7fc703e00640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 0x7fc6e407b0d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc70406b880 tx=0x7fc6f4006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:36.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:36.791+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7041fac70 con 0x7fc7040ab580 2024-09-21T06:34:37.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.067+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc70415cc70 con 0x7fc7040ab580 2024-09-21T06:34:37.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.067+0000 7fc6fbe00640 1 -- 172.21.15.37:0/2695067222 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1865 (secure 0 0 0) 0x7fc70406e840 con 0x7fc7040ab580 2024-09-21T06:34:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:37.070 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":425,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":265945088,"bytes_avail":772828168192,"bytes_total":773094113280,"write_bytes_sec":597,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7096},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7389},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7392}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-09-21T06:34:35.198506+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:34:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 msgr2=0x7fc6e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 0x7fc6e407b0d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc70406b880 tx=0x7fc6f4006b70 comp rx=0 tx=0).stop 2024-09-21T06:34:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 msgr2=0x7fc7040b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc6f002b2d0 tx=0x7fc6f00060e0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 shutdown_connections 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc6e4078bc0 0x7fc6e407b0d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7040ab580 0x7fc7040b0520 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7040aece0 0x7fc7040a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 --2- 172.21.15.37:0/2695067222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6e8004800 0x7fc7041fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 >> 172.21.15.37:0/2695067222 conn(0x7fc70406f940 msgr2=0x7fc7040d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 shutdown_connections 2024-09-21T06:34:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.070+0000 7fc709800640 1 -- 172.21.15.37:0/2695067222 wait complete. 2024-09-21T06:34:37.094 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-21T06:34:37.095 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:34:37.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:34:37.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:34:37.278 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:34:37.278 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:34:37.278 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:34:37.278 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:37.314 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:34:37.314 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:37.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.654+0000 7f0e3ac00640 1 -- 172.21.15.37:0/2174523286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e18004800 msgr2=0x7f0e18004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.654+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/2174523286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e18004800 0x7f0e18004c10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c2e0 tx=0x7f0e18005830 comp rx=0 tx=0).stop 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.655+0000 7f0e3ac00640 1 -- 172.21.15.37:0/2174523286 shutdown_connections 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.655+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/2174523286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e18004800 0x7f0e18004c10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.655+0000 7f0e3ac00640 1 -- 172.21.15.37:0/2174523286 >> 172.21.15.37:0/2174523286 conn(0x7f0e340b1b90 msgr2=0x7f0e340b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.655+0000 7f0e3ac00640 1 -- 172.21.15.37:0/2174523286 shutdown_connections 2024-09-21T06:34:37.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.655+0000 7f0e3ac00640 1 -- 172.21.15.37:0/2174523286 wait complete. 2024-09-21T06:34:37.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.657+0000 7f0e3ac00640 1 Processor -- start 2024-09-21T06:34:37.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.657+0000 7f0e3ac00640 1 -- start start 2024-09-21T06:34:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.658+0000 7f0e3ac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e18004800 0x7f0e340bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.658+0000 7f0e3ac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e340c6280 0x7f0e340bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.658+0000 7f0e3ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e3ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3406e590 con 0x7f0e340c0170 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e3ac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3406e410 con 0x7f0e340c6280 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e3ac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3406e710 con 0x7f0e18004800 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53346/0 (socket says 172.21.15.37:53346) 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 -- 172.21.15.37:0/3350413530 learned_addr learned my addr 172.21.15.37:0/3350413530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e33e00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e18004800 0x7f0e340bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e33400640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e340c6280 0x7f0e340bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 -- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e18004800 msgr2=0x7f0e340bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e18004800 0x7f0e340bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 -- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e340c6280 msgr2=0x7f0e340bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e340c6280 0x7f0e340bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.659+0000 7f0e38c00640 1 -- 172.21.15.37:0/3350413530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341574c0 con 0x7f0e340c0170 2024-09-21T06:34:37.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e38c00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005fd0 tx=0x7f0e2c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:37.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c017070 con 0x7f0e340c0170 2024-09-21T06:34:37.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e2c015070 con 0x7f0e340c0170 2024-09-21T06:34:37.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3415f870 con 0x7f0e340c0170 2024-09-21T06:34:37.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c01c7f0 con 0x7f0e340c0170 2024-09-21T06:34:37.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.660+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3415fe30 con 0x7f0e340c0170 2024-09-21T06:34:37.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.661+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34074910 con 0x7f0e340c0170 2024-09-21T06:34:37.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.662+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e2c01c990 con 0x7f0e340c0170 2024-09-21T06:34:37.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.663+0000 7f0e2be00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 0x7f0e1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.663+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0e2c09bba0 con 0x7f0e340c0170 2024-09-21T06:34:37.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.664+0000 7f0e33e00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 0x7f0e1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.665+0000 7f0e33e00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 0x7f0e1407b0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e34052860 tx=0x7f0e24006ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:37.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.668+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e2c063ed0 con 0x7f0e340c0170 2024-09-21T06:34:37.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.875+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e3415a4b0 con 0x7f0e340c0170 2024-09-21T06:34:37.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.875+0000 7f0e2be00640 1 -- 172.21.15.37:0/3350413530 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f0e2c0637d0 con 0x7f0e340c0170 2024-09-21T06:34:37.877 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 msgr2=0x7f0e1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 0x7f0e1407b0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e34052860 tx=0x7f0e24006ef0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 msgr2=0x7f0e34158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005fd0 tx=0x7f0e2c010f40 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 shutdown_connections 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0e14078bc0 0x7f0e1407b0d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e340c0170 0x7f0e34158e80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e340c6280 0x7f0e340bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 --2- 172.21.15.37:0/3350413530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e18004800 0x7f0e340bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.877+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 >> 172.21.15.37:0/3350413530 conn(0x7f0e340b1b90 msgr2=0x7f0e340ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:37.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.878+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 shutdown_connections 2024-09-21T06:34:37.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:37.878+0000 7f0e3ac00640 1 -- 172.21.15.37:0/3350413530 wait complete. 2024-09-21T06:34:37.905 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:34:37.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:37.905 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:34:37.969 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:34:37.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:37.969 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:34:38.005 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:34:38.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:34:38.005 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:38.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:34:38.045 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:34:38.046 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:34:38.046 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:38.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:34:38.081 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:34:38.082 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:34:38.082 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:38.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:34:38.124 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 -- 172.21.15.37:0/606404950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 msgr2=0x7f239815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 --2- 172.21.15.37:0/606404950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f239815cf20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2390005fd0 tx=0x7f239002a820 comp rx=0 tx=0).stop 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 -- 172.21.15.37:0/606404950 shutdown_connections 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 --2- 172.21.15.37:0/606404950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f239815cf20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 --2- 172.21.15.37:0/606404950 >> v2:172.21.15.150:3300/0 conn(0x7f23980ccf20 0x7f23980b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 -- 172.21.15.37:0/606404950 >> 172.21.15.37:0/606404950 conn(0x7f239806f940 msgr2=0x7f23980b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 -- 172.21.15.37:0/606404950 shutdown_connections 2024-09-21T06:34:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.351+0000 7f239e600640 1 -- 172.21.15.37:0/606404950 wait complete. 2024-09-21T06:34:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.353+0000 7f239e600640 1 Processor -- start 2024-09-21T06:34:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.353+0000 7f239e600640 1 -- start start 2024-09-21T06:34:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f23981f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2398157200 0x7f2398156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f239806e5f0 con 0x7f23980cddc0 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f239815d460 con 0x7f2398157200 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239e600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2398142b30 con 0x7f23980b4ed0 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f2397e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f2397e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53352/0 (socket says 172.21.15.37:53352) 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f2397e00640 1 -- 172.21.15.37:0/770081301 learned_addr learned my addr 172.21.15.37:0/770081301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239dc00640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2398157200 0x7f2398156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.355+0000 7f239d200640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f23981f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 -- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 msgr2=0x7f23981f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f23981f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 -- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2398157200 msgr2=0x7f2398156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2398157200 0x7f2398156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 -- 172.21.15.37:0/770081301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23981f2c50 con 0x7f23980cddc0 2024-09-21T06:34:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2397e00640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f238400ea20 tx=0x7f238400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400cd50 con 0x7f23980cddc0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2384004020 con 0x7f23980cddc0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384004470 con 0x7f23980cddc0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.356+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2398154f00 con 0x7f23980cddc0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.357+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981554c0 con 0x7f23980cddc0 2024-09-21T06:34:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.358+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239815d950 con 0x7f23980cddc0 2024-09-21T06:34:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.359+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23840035f0 con 0x7f23980cddc0 2024-09-21T06:34:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.360+0000 7f2394c00640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 0x7f237807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.360+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f238409b730 con 0x7f23980cddc0 2024-09-21T06:34:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.360+0000 7f239d200640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 0x7f237807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.361+0000 7f239d200640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 0x7f237807b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f238c001720 tx=0x7f238c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.364+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f23840632f0 con 0x7f23980cddc0 2024-09-21T06:34:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.600+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23981576c0 con 0x7f23980cddc0 2024-09-21T06:34:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.601+0000 7f2394c00640 1 -- 172.21.15.37:0/770081301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f2384067d20 con 0x7f23980cddc0 2024-09-21T06:34:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:38.603 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 msgr2=0x7f237807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 0x7f237807b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f238c001720 tx=0x7f238c009040 comp rx=0 tx=0).stop 2024-09-21T06:34:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 msgr2=0x7f2398155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f238400ea20 tx=0x7f238400eee0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 shutdown_connections 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2378078bc0 0x7f237807b0d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23980cddc0 0x7f2398155cc0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2398157200 0x7f2398156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 --2- 172.21.15.37:0/770081301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23980b4ed0 0x7f23981f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 >> 172.21.15.37:0/770081301 conn(0x7f239806f940 msgr2=0x7f239815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 shutdown_connections 2024-09-21T06:34:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.603+0000 7f239e600640 1 -- 172.21.15.37:0/770081301 wait complete. 2024-09-21T06:34:38.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 -- 172.21.15.37:0/3168371819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc0cbc10 msgr2=0x7fc5fc0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 --2- 172.21.15.37:0/3168371819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc0cbc10 0x7fc5fc0cc020 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc076360 tx=0x7fc5e802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 -- 172.21.15.37:0/3168371819 shutdown_connections 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 --2- 172.21.15.37:0/3168371819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc0cbc10 0x7fc5fc0cc020 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 --2- 172.21.15.37:0/3168371819 >> v2:172.21.15.150:3300/0 conn(0x7fc5fc074cb0 0x7fc5fc073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.852+0000 7fc604200640 1 -- 172.21.15.37:0/3168371819 >> 172.21.15.37:0/3168371819 conn(0x7fc5fc0cca40 msgr2=0x7fc5fc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.853+0000 7fc604200640 1 -- 172.21.15.37:0/3168371819 shutdown_connections 2024-09-21T06:34:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.853+0000 7fc604200640 1 -- 172.21.15.37:0/3168371819 wait complete. 2024-09-21T06:34:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.855+0000 7fc604200640 1 Processor -- start 2024-09-21T06:34:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.855+0000 7fc604200640 1 -- start start 2024-09-21T06:34:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.856+0000 7fc604200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5fc06ddf0 0x7fc5fc157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.856+0000 7fc604200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc604200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5fc0cbc10 0x7fc5fc0c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc604200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc06f360 con 0x7fc5fc074cb0 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc604200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc06f1e0 con 0x7fc5fc06ddf0 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc604200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc06f4e0 con 0x7fc5fc0cbc10 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53362/0 (socket says 172.21.15.37:53362) 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 -- 172.21.15.37:0/3999350140 learned_addr learned my addr 172.21.15.37:0/3999350140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc601800640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5fc06ddf0 0x7fc5fc157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc602200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5fc0cbc10 0x7fc5fc0c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 -- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5fc0cbc10 msgr2=0x7fc5fc0c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5fc0cbc10 0x7fc5fc0c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 -- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5fc06ddf0 msgr2=0x7fc5fc157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5fc06ddf0 0x7fc5fc157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.857+0000 7fc600e00640 1 -- 172.21.15.37:0/3999350140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc1f63b0 con 0x7fc5fc074cb0 2024-09-21T06:34:38.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc600e00640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8034040 tx=0x7fc5e8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e803e030 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5e8003e00 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc0cf7c0 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8035040 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.858+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc156060 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.859+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc0cf530 con 0x7fc5fc074cb0 2024-09-21T06:34:38.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.861+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc5e8041030 con 0x7fc5fc074cb0 2024-09-21T06:34:38.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.862+0000 7fc5f9600640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 0x7fc5dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:38.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.862+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc5e80bb790 con 0x7fc5fc074cb0 2024-09-21T06:34:38.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.863+0000 7fc601800640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 0x7fc5dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:38.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.864+0000 7fc601800640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 0x7fc5dc07b0a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0005fd0 tx=0x7fc5f0005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:38.867+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5e8083a30 con 0x7fc5fc074cb0 2024-09-21T06:34:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.103+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5fc0731a0 con 0x7fc5fc074cb0 2024-09-21T06:34:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.104+0000 7fc5f9600640 1 -- 172.21.15.37:0/3999350140 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7fc5fc0731a0 con 0x7fc5fc074cb0 2024-09-21T06:34:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:39.107 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.106+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 msgr2=0x7fc5dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 0x7fc5dc07b0a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0005fd0 tx=0x7fc5f0005850 comp rx=0 tx=0).stop 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 msgr2=0x7fc5fc0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8034040 tx=0x7fc5e8003910 comp rx=0 tx=0).stop 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 shutdown_connections 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5dc078b90 0x7fc5dc07b0a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc074cb0 0x7fc5fc0ce880 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5fc0cbc10 0x7fc5fc0c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 --2- 172.21.15.37:0/3999350140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5fc06ddf0 0x7fc5fc157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 >> 172.21.15.37:0/3999350140 conn(0x7fc5fc0cca40 msgr2=0x7fc5fc15dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 shutdown_connections 2024-09-21T06:34:39.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.107+0000 7fc604200640 1 -- 172.21.15.37:0/3999350140 wait complete. 2024-09-21T06:34:39.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.356+0000 7ffa9ea00640 1 -- 172.21.15.37:0/278535914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98070030 msgr2=0x7ffa9806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.356+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/278535914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98070030 0x7ffa9806ddf0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005fd0 tx=0x7ffa8402a5c0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 -- 172.21.15.37:0/278535914 shutdown_connections 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/278535914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98070030 0x7ffa9806ddf0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/278535914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa98074d90 0x7ffa9806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 -- 172.21.15.37:0/278535914 >> 172.21.15.37:0/278535914 conn(0x7ffa980cca40 msgr2=0x7ffa980cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:39.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 -- 172.21.15.37:0/278535914 shutdown_connections 2024-09-21T06:34:39.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.357+0000 7ffa9ea00640 1 -- 172.21.15.37:0/278535914 wait complete. 2024-09-21T06:34:39.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.359+0000 7ffa9ea00640 1 Processor -- start 2024-09-21T06:34:39.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.359+0000 7ffa9ea00640 1 -- start start 2024-09-21T06:34:39.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa9806e330 0x7ffa980c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa98070030 0x7ffa980c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9815ad50 con 0x7ffa98074d90 2024-09-21T06:34:39.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9815abd0 con 0x7ffa98070030 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.360+0000 7ffa9ea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ffa98070e80 con 0x7ffa9806e330 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa97e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa9806e330 0x7ffa980c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53386/0 (socket says 172.21.15.37:53386) 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 -- 172.21.15.37:0/1565944052 learned_addr learned my addr 172.21.15.37:0/1565944052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa97400640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa98070030 0x7ffa980c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 -- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa9806e330 msgr2=0x7ffa980c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa9806e330 0x7ffa980c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 -- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa98070030 msgr2=0x7ffa980c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa98070030 0x7ffa980c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 -- 172.21.15.37:0/1565944052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa980c3b90 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.361+0000 7ffa9ca00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c6b0 tx=0x7ffa9000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.362+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90003b70 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.362+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa90003d10 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.362+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90005330 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.362+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa980c67d0 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.362+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa980c6d90 con 0x7ffa98074d90 2024-09-21T06:34:39.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.363+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9806ddf0 con 0x7ffa98074d90 2024-09-21T06:34:39.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.364+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa900054d0 con 0x7ffa98074d90 2024-09-21T06:34:39.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.365+0000 7ffa8fe00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 0x7ffa7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.366+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ffa9009ab80 con 0x7ffa98074d90 2024-09-21T06:34:39.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.366+0000 7ffa97e00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 0x7ffa7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.367+0000 7ffa97e00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 0x7ffa7807b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009b30 tx=0x7ffa88005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:39.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.370+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa900627e0 con 0x7ffa98074d90 2024-09-21T06:34:39.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.568+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa980b5f40 con 0x7ffa98074d90 2024-09-21T06:34:39.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.570+0000 7ffa8fe00640 1 -- 172.21.15.37:0/1565944052 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17005 (secure 0 0 0) 0x7ffa90067210 con 0x7ffa98074d90 2024-09-21T06:34:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:39.572 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:33:48.044956+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:33:44.503000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T06:33:45.519092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 msgr2=0x7ffa7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 0x7ffa7807b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009b30 tx=0x7ffa88005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 msgr2=0x7ffa981f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c6b0 tx=0x7ffa9000cb70 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 shutdown_connections 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ffa78078b90 0x7ffa7807b0a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa98074d90 0x7ffa981f3b80 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa98070030 0x7ffa980c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 --2- 172.21.15.37:0/1565944052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa9806e330 0x7ffa980c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.572+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 >> 172.21.15.37:0/1565944052 conn(0x7ffa980cca40 msgr2=0x7ffa980a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.573+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 shutdown_connections 2024-09-21T06:34:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.573+0000 7ffa9ea00640 1 -- 172.21.15.37:0/1565944052 wait complete. 2024-09-21T06:34:39.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:34:39.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:39.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 -- 172.21.15.37:0/2574399122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c004800 msgr2=0x7f788c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 --2- 172.21.15.37:0/2574399122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c004800 0x7f788c004c10 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7890006fb0 tx=0x7f788c005830 comp rx=0 tx=0).stop 2024-09-21T06:34:39.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 -- 172.21.15.37:0/2574399122 shutdown_connections 2024-09-21T06:34:39.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 --2- 172.21.15.37:0/2574399122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c004800 0x7f788c004c10 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 -- 172.21.15.37:0/2574399122 >> 172.21.15.37:0/2574399122 conn(0x7f78a8073400 msgr2=0x7f78a8073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:39.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 -- 172.21.15.37:0/2574399122 shutdown_connections 2024-09-21T06:34:39.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.834+0000 7f78ad600640 1 -- 172.21.15.37:0/2574399122 wait complete. 2024-09-21T06:34:39.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.836+0000 7f78ad600640 1 Processor -- start 2024-09-21T06:34:39.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.836+0000 7f78ad600640 1 -- start start 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 0x7f78a815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 0x7f78a8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8074070 con 0x7f78a8160ca0 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78a806e680 con 0x7f788c004800 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78ad600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f78a80741f0 con 0x7f78a8160590 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53412/0 (socket says 172.21.15.37:53412) 2024-09-21T06:34:39.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 -- 172.21.15.37:0/3133062990 learned_addr learned my addr 172.21.15.37:0/3133062990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:39.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f789fe00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 0x7f78a8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78a7e00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 0x7f78a815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 -- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 msgr2=0x7f78a8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 0x7f78a8157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 -- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 msgr2=0x7f78a815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 0x7f78a815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78acc00640 1 -- 172.21.15.37:0/3133062990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a81584b0 con 0x7f78a8160ca0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f789fe00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 0x7f78a8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.838+0000 7f78a7e00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 0x7f78a815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78acc00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f78a0005fd0 tx=0x7f78a0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003970 con 0x7f78a8160ca0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78a0015070 con 0x7f78a8160ca0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004030 con 0x7f78a8160ca0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a815f7d0 con 0x7f78a8160ca0 2024-09-21T06:34:39.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.839+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a815fd90 con 0x7f78a8160ca0 2024-09-21T06:34:39.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.840+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a81611f0 con 0x7f78a8160ca0 2024-09-21T06:34:39.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.842+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78a00041d0 con 0x7f78a8160ca0 2024-09-21T06:34:39.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.843+0000 7f78a4c00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 0x7f788407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:39.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.843+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6391+0+0 (secure 0 0 0) 0x7f78a009ba40 con 0x7f78a8160ca0 2024-09-21T06:34:39.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.843+0000 7f78a7e00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 0x7f788407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:39.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.844+0000 7f78a7e00640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 0x7f788407b100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78980064d0 tx=0x7f7898006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:39.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:39.847+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78a0063bc0 con 0x7f78a8160ca0 2024-09-21T06:34:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.083+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78a80cbd80 con 0x7f78a8160ca0 2024-09-21T06:34:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.084+0000 7f78a4c00640 1 -- 172.21.15.37:0/3133062990 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7f78a00634c0 con 0x7f78a8160ca0 2024-09-21T06:34:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:40.086 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 msgr2=0x7f788407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 0x7f788407b100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78980064d0 tx=0x7f7898006b10 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 msgr2=0x7f78a80b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f78a0005fd0 tx=0x7f78a0010f40 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 shutdown_connections 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7884078bf0 0x7f788407b100 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78a8160ca0 0x7f78a80b4b50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a8160590 0x7f78a8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 --2- 172.21.15.37:0/3133062990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f788c004800 0x7f78a815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.086+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 >> 172.21.15.37:0/3133062990 conn(0x7f78a8073400 msgr2=0x7f78a815c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.087+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 shutdown_connections 2024-09-21T06:34:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.087+0000 7f78ad600640 1 -- 172.21.15.37:0/3133062990 wait complete. 2024-09-21T06:34:40.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:40.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 -- 172.21.15.37:0/3574918147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf08162f00 msgr2=0x7fdf080d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 --2- 172.21.15.37:0/3574918147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf08162f00 0x7fdf080d24d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdf080b7460 tx=0x7fdef802a560 comp rx=0 tx=0).stop 2024-09-21T06:34:40.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 -- 172.21.15.37:0/3574918147 shutdown_connections 2024-09-21T06:34:40.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 --2- 172.21.15.37:0/3574918147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf080b2e40 0x7fdf080b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 --2- 172.21.15.37:0/3574918147 >> v2:172.21.15.150:3300/0 conn(0x7fdf08162280 0x7fdf08162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 --2- 172.21.15.37:0/3574918147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf08162f00 0x7fdf080d24d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 -- 172.21.15.37:0/3574918147 >> 172.21.15.37:0/3574918147 conn(0x7fdf080b6b20 msgr2=0x7fdf080d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 -- 172.21.15.37:0/3574918147 shutdown_connections 2024-09-21T06:34:40.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.338+0000 7fdf0e000640 1 -- 172.21.15.37:0/3574918147 wait complete. 2024-09-21T06:34:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.340+0000 7fdf0e000640 1 Processor -- start 2024-09-21T06:34:40.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.340+0000 7fdf0e000640 1 -- start start 2024-09-21T06:34:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf080b2e40 0x7fdf081f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf08162280 0x7fdf0814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf080b3dd0 con 0x7fdf0815ea50 2024-09-21T06:34:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf080b3a20 con 0x7fdf080b2e40 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf0e000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdf080b3f50 con 0x7fdf08162280 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53430/0 (socket says 172.21.15.37:53430) 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 -- 172.21.15.37:0/2710950876 learned_addr learned my addr 172.21.15.37:0/2710950876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdeffe00640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf08162280 0x7fdf0814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 -- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf08162280 msgr2=0x7fdf0814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf08162280 0x7fdf0814d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 -- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf080b2e40 msgr2=0x7fdf081f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf080b2e40 0x7fdf081f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.342+0000 7fdf07e00640 1 -- 172.21.15.37:0/2710950876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0815f3a0 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdf07e00640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ea00 tx=0x7fdf0000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000cde0 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf00004b20 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00005020 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081f9720 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.343+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0814e090 con 0x7fdf0815ea50 2024-09-21T06:34:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.344+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf080b9690 con 0x7fdf0815ea50 2024-09-21T06:34:40.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.345+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf000051c0 con 0x7fdf0815ea50 2024-09-21T06:34:40.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.346+0000 7fdeff400640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 0x7fdee407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.346+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6391+0+0 (secure 0 0 0) 0x7fdf0009bbe0 con 0x7fdf0815ea50 2024-09-21T06:34:40.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.347+0000 7fdf07400640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 0x7fdee407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.347+0000 7fdf07400640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 0x7fdee407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdef8033000 tx=0x7fdef80046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.351+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf080b9690 con 0x7fdf0815ea50 2024-09-21T06:34:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.587+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf08162f00 con 0x7fdf0815ea50 2024-09-21T06:34:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.587+0000 7fdeff400640 1 -- 172.21.15.37:0/2710950876 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5275 (secure 0 0 0) 0x7fdf0815f3a0 con 0x7fdf0815ea50 2024-09-21T06:34:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:40.590 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"btime":"2024-09-21T06:34:20:272222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:33:46.562653+0000","modified":"2024-09-21T06:34:20.121129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7392,"mds_1":7389,"mds_2":7096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7096":{"gid":7096,"name":"b","rank":2,"incarnation":93,"state":"up:active","state_seq":10,"addr":"172.21.15.150:6837/3722358982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":3722358982},{"type":"v1","addr":"172.21.15.150:6837","nonce":3722358982}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7389":{"gid":7389,"name":"a","rank":1,"incarnation":91,"state":"up:active","state_seq":11,"addr":"172.21.15.37:6835/1748999037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1748999037},{"type":"v1","addr":"172.21.15.37:6835","nonce":1748999037}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7392":{"gid":7392,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6836/2979710501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2979710501},{"type":"v1","addr":"172.21.15.150:6836","nonce":2979710501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7392,"qdb_cluster":[7392,7389,7096]},"id":6}]} 2024-09-21T06:34:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 99 2024-09-21T06:34:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 msgr2=0x7fdee407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 0x7fdee407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdef8033000 tx=0x7fdef80046c0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 msgr2=0x7fdf0815ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ea00 tx=0x7fdf0000eec0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 shutdown_connections 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdee4078b90 0x7fdee407b0a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf0815ea50 0x7fdf0815ee60 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf08162280 0x7fdf0814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 --2- 172.21.15.37:0/2710950876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf080b2e40 0x7fdf081f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 >> 172.21.15.37:0/2710950876 conn(0x7fdf080b6b20 msgr2=0x7fdf0815d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 shutdown_connections 2024-09-21T06:34:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.590+0000 7fdf0e000640 1 -- 172.21.15.37:0/2710950876 wait complete. 2024-09-21T06:34:40.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 -- 172.21.15.37:0/282596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 msgr2=0x7f3b000b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 --2- 172.21.15.37:0/282596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b000b06f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3b00072f40 tx=0x7f3af002a540 comp rx=0 tx=0).stop 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 -- 172.21.15.37:0/282596706 shutdown_connections 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 --2- 172.21.15.37:0/282596706 >> v2:172.21.15.150:3300/0 conn(0x7f3b000af210 0x7f3b000aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 --2- 172.21.15.37:0/282596706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000b0c30 0x7f3b000ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 --2- 172.21.15.37:0/282596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b000b06f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 -- 172.21.15.37:0/282596706 >> 172.21.15.37:0/282596706 conn(0x7f3b000b5140 msgr2=0x7f3b000b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 -- 172.21.15.37:0/282596706 shutdown_connections 2024-09-21T06:34:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.850+0000 7f3b06200640 1 -- 172.21.15.37:0/282596706 wait complete. 2024-09-21T06:34:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.852+0000 7f3b06200640 1 Processor -- start 2024-09-21T06:34:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.852+0000 7f3b06200640 1 -- start start 2024-09-21T06:34:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000af210 0x7f3b000cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b000b0c30 0x7f3b001eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000d07b0 con 0x7f3b000acfd0 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000d0630 con 0x7f3b000b0c30 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b06200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3b000d0930 con 0x7f3b000af210 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53442/0 (socket says 172.21.15.37:53442) 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 -- 172.21.15.37:0/2765824048 learned_addr learned my addr 172.21.15.37:0/2765824048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3af7e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000af210 0x7f3b000cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b05800640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b000b0c30 0x7f3b001eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 -- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000af210 msgr2=0x7f3b000cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000af210 0x7f3b000cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 -- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b000b0c30 msgr2=0x7f3b001eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.854+0000 7f3b04e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b000b0c30 0x7f3b001eae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3b04e00640 1 -- 172.21.15.37:0/2765824048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001e62d0 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3b04e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3af00099d0 tx=0x7f3af00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0031b50 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3af0031de0 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b000ca8b0 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0033330 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.855+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b000cae70 con 0x7f3b000acfd0 2024-09-21T06:34:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.856+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000cdc50 con 0x7f3b000acfd0 2024-09-21T06:34:40.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.858+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3af00334d0 con 0x7f3b000acfd0 2024-09-21T06:34:40.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.859+0000 7f3afd600640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 0x7f3adc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.859+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6391+0+0 (secure 0 0 0) 0x7f3af00bc5f0 con 0x7f3b000acfd0 2024-09-21T06:34:40.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.859+0000 7f3af7e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 0x7f3adc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.860+0000 7f3af7e00640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 0x7f3adc07b0a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8005d70 tx=0x7f3ae8005d00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:40.863+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b000cdc50 con 0x7f3b000acfd0 2024-09-21T06:34:41.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.062+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b000b48c0 con 0x7f3b000acfd0 2024-09-21T06:34:41.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.063+0000 7f3afd600640 1 -- 172.21.15.37:0/2765824048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17486 (secure 0 0 0) 0x7f3b000b48c0 con 0x7f3b000acfd0 2024-09-21T06:34:41.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":93,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:34:40.590780+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:33:44.503000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T06:33:45.519092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3","mappings":[{"from":6,"to":4}]},{"pgid":"14.10","mappings":[{"from":1,"to":3}]},{"pgid":"14.12","mappings":[{"from":7,"to":2}]},{"pgid":"14.1f","mappings":[{"from":7,"to":0}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":1,"to":0}]},{"pgid":"14.2e","mappings":[{"from":7,"to":3}]},{"pgid":"14.38","mappings":[{"from":4,"to":0}]},{"pgid":"14.39","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.065+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 msgr2=0x7f3adc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.065+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 0x7f3adc07b0a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8005d70 tx=0x7f3ae8005d00 comp rx=0 tx=0).stop 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.065+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 msgr2=0x7f3b001e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.065+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3af00099d0 tx=0x7f3af00052f0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 shutdown_connections 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3adc078b90 0x7f3adc07b0a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b000b0c30 0x7f3b001eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b000af210 0x7f3b000cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 --2- 172.21.15.37:0/2765824048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b000acfd0 0x7f3b001e5a80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 >> 172.21.15.37:0/2765824048 conn(0x7f3b000b5140 msgr2=0x7f3b0015bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:41.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 shutdown_connections 2024-09-21T06:34:41.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.066+0000 7f3b06200640 1 -- 172.21.15.37:0/2765824048 wait complete. 2024-09-21T06:34:41.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.318+0000 7f0c05000640 1 -- 172.21.15.37:0/775746388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 msgr2=0x7f0be4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.318+0000 7f0c05000640 1 --2- 172.21.15.37:0/775746388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0be4004800 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001830 tx=0x7f0be4005830 comp rx=0 tx=0).stop 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.318+0000 7f0bffe00640 1 -- 172.21.15.37:0/775746388 reap_dead start 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.318+0000 7f0c05000640 1 -- 172.21.15.37:0/775746388 shutdown_connections 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.319+0000 7f0c05000640 1 -- 172.21.15.37:0/775746388 >> 172.21.15.37:0/775746388 conn(0x7f0c000d05c0 msgr2=0x7f0c0006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.319+0000 7f0c05000640 1 -- 172.21.15.37:0/775746388 shutdown_connections 2024-09-21T06:34:41.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.319+0000 7f0c05000640 1 -- 172.21.15.37:0/775746388 wait complete. 2024-09-21T06:34:41.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.321+0000 7f0c05000640 1 Processor -- start 2024-09-21T06:34:41.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.321+0000 7f0c05000640 1 -- start start 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.322+0000 7f0c05000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0c05000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c001401e0 0x7f0c0006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0c05000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 0x7f0c000c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0c05000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000b3820 con 0x7f0be4004230 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0c05000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000707b0 con 0x7f0c0015bbd0 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0c05000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000b39a0 con 0x7f0c001401e0 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53460/0 (socket says 172.21.15.37:53460) 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 -- 172.21.15.37:0/2484930657 learned_addr learned my addr 172.21.15.37:0/2484930657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bf7e00640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c001401e0 0x7f0c0006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bffe00640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 0x7f0c000c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 -- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c001401e0 msgr2=0x7f0c0006d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c001401e0 0x7f0c0006d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 -- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 msgr2=0x7f0c000c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 0x7f0c000c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bff400640 1 -- 172.21.15.37:0/2484930657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001436a0 con 0x7f0be4004230 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.323+0000 7f0bffe00640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 0x7f0c000c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0bff400640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011ae0 tx=0x7f0bf0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf001e070 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf000fe70 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0004290 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c00143980 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.324+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c00143f40 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.325+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c000b3f30 con 0x7f0be4004230 2024-09-21T06:34:41.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.326+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bf0021070 con 0x7f0be4004230 2024-09-21T06:34:41.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.327+0000 7f0bf7400640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.328+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6391+0+0 (secure 0 0 0) 0x7f0bf009bae0 con 0x7f0be4004230 2024-09-21T06:34:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.328+0000 7f0bf7e00640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.329+0000 7f0bf7e00640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0be8032000 tx=0x7f0be8001360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:41.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.332+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bf0063640 con 0x7f0be4004230 2024-09-21T06:34:41.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.566+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c000b52f0 con 0x7f0be4004230 2024-09-21T06:34:41.569 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:34:41.567+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:34:41.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.569+0000 7f0bf7400640 1 -- 172.21.15.37:0/2484930657 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7f0bf0068070 con 0x7f0be4004230 2024-09-21T06:34:41.571 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:34:41.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.572+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 msgr2=0x7f0bdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.572+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0be8032000 tx=0x7f0be8001360 comp rx=0 tx=0).stop 2024-09-21T06:34:41.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.572+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 msgr2=0x7f0c00144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.572+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011ae0 tx=0x7f0bf0009e90 comp rx=0 tx=0).stop 2024-09-21T06:34:41.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 shutdown_connections 2024-09-21T06:34:41.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0015bbd0 0x7f0c000c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c001401e0 0x7f0c0006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 --2- 172.21.15.37:0/2484930657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be4004230 0x7f0c00144460 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 >> 172.21.15.37:0/2484930657 conn(0x7f0c000d05c0 msgr2=0x7f0c00145720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 shutdown_connections 2024-09-21T06:34:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.573+0000 7f0c05000640 1 -- 172.21.15.37:0/2484930657 wait complete. 2024-09-21T06:34:41.588 INFO:tasks.ceph.mds.b.smithi150.stderr: -1359> 2024-09-21T06:34:15.994+0000 7f2617600640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1) n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e14e422000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.422912+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.589 INFO:tasks.ceph.mds.b.smithi150.stderr: -1337> 2024-09-21T06:34:15.995+0000 7f2617600640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/AsLsXsFsx@2},l=7515 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e14e422580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.589 INFO:tasks.ceph.mds.a.smithi037.stderr: -1482> 2024-09-21T06:34:15.994+0000 7f7e2bc00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1) n(v2 rc2024-09-21T06:34:00.701908+0000 4=0+4) (inest lock) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55606b8f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.419912+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:34:00.701908+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.590 INFO:tasks.ceph.mds.a.smithi037.stderr: -1332> 2024-09-21T06:34:15.996+0000 7f7e33e00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1) n(v1 rc2024-09-21T06:34:00.701908+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7515=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55606b8f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.422912+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.420912+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:34:00.701908+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.597 INFO:tasks.ceph.mds.d.smithi150.stderr: -3007> 2024-09-21T06:34:15.993+0000 7fa6d2a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1) n(v3 rc2024-09-21T06:34:00.701908+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7515=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7515 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5600e7f5f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.416912+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:34:00.701908+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.597 INFO:tasks.ceph.mds.d.smithi150.stderr: -2874> 2024-09-21T06:34:15.995+0000 7fa6dac00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1) n(v4 rc2024-09-21T06:34:00.701908+0000 5=0+5) (inest sync->mix g=1,2) caps={7515=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5600e7f5f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:34:00.701908+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:34:00.417912+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:34:00.701908+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:34:41.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:34:41.709 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:34:41.724 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:34:41.737 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:34:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 -- 172.21.15.37:0/2794690534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 msgr2=0x7fc54c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 --2- 172.21.15.37:0/2794690534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c06ddf0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0b2c70 tx=0x7fc53802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 -- 172.21.15.37:0/2794690534 shutdown_connections 2024-09-21T06:34:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 --2- 172.21.15.37:0/2794690534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c06ddf0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 --2- 172.21.15.37:0/2794690534 >> v2:172.21.15.150:3300/0 conn(0x7fc54c156a50 0x7fc54c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 -- 172.21.15.37:0/2794690534 >> 172.21.15.37:0/2794690534 conn(0x7fc54c0b1b90 msgr2=0x7fc54c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 -- 172.21.15.37:0/2794690534 shutdown_connections 2024-09-21T06:34:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.836+0000 7fc554600640 1 -- 172.21.15.37:0/2794690534 wait complete. 2024-09-21T06:34:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.838+0000 7fc554600640 1 Processor -- start 2024-09-21T06:34:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.838+0000 7fc554600640 1 -- start start 2024-09-21T06:34:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 0x7fc54c1f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 0x7fc54c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0ce500 con 0x7fc54c074b60 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0ce380 con 0x7fc54c06e330 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc554600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0ce680 con 0x7fc54c156a50 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc551200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc551c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 0x7fc54c1f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc551200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53504/0 (socket says 172.21.15.37:53504) 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc551200640 1 -- 172.21.15.37:0/239795543 learned_addr learned my addr 172.21.15.37:0/239795543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.840+0000 7fc552600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 0x7fc54c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 -- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 msgr2=0x7fc54c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 0x7fc54c0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 -- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 msgr2=0x7fc54c1f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 0x7fc54c1f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 -- 172.21.15.37:0/239795543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0c0870 con 0x7fc54c074b60 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc552600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 0x7fc54c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551c00640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 0x7fc54c1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc551200640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.841+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53803e030 con 0x7fc54c074b60 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.842+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc538003d10 con 0x7fc54c074b60 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.842+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538035040 con 0x7fc54c074b60 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.842+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c09ee90 con 0x7fc54c074b60 2024-09-21T06:34:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.842+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c09f450 con 0x7fc54c074b60 2024-09-21T06:34:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.843+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c0c05e0 con 0x7fc54c074b60 2024-09-21T06:34:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.844+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc538041030 con 0x7fc54c074b60 2024-09-21T06:34:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.845+0000 7fc549600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 0x7fc52c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.845+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6649+0+0 (secure 0 0 0) 0x7fc5380bb890 con 0x7fc54c074b60 2024-09-21T06:34:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.845+0000 7fc551c00640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 0x7fc52c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.846+0000 7fc551c00640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 0x7fc52c07b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc540009f20 tx=0x7fc540005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:41.850+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc538083230 con 0x7fc54c074b60 2024-09-21T06:34:42.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.098+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc54c06ddf0 con 0x7fc54c074b60 2024-09-21T06:34:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.613+0000 7fc549600640 1 -- 172.21.15.37:0/239795543 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) ==== 104+0+0 (secure 0 0 0) 0x7fc538087c60 con 0x7fc54c074b60 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 msgr2=0x7fc52c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 0x7fc52c07b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc540009f20 tx=0x7fc540005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 msgr2=0x7fc54c0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538003820 comp rx=0 tx=0).stop 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 shutdown_connections 2024-09-21T06:34:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc52c078bc0 0x7fc52c07b0d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc54c074b60 0x7fc54c0b3a10 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc54c156a50 0x7fc54c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 --2- 172.21.15.37:0/239795543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc54c06e330 0x7fc54c1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 >> 172.21.15.37:0/239795543 conn(0x7fc54c0b1b90 msgr2=0x7fc54c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.616+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 shutdown_connections 2024-09-21T06:34:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.617+0000 7fc554600640 1 -- 172.21.15.37:0/239795543 wait complete. 2024-09-21T06:34:42.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.876+0000 7fe33da00640 1 -- 172.21.15.37:0/479635346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe33815f720 msgr2=0x7fe338155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.876+0000 7fe33da00640 1 --2- 172.21.15.37:0/479635346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe33815f720 0x7fe338155570 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe328009860 tx=0x7fe32802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.877+0000 7fe33da00640 1 -- 172.21.15.37:0/479635346 shutdown_connections 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.877+0000 7fe33da00640 1 --2- 172.21.15.37:0/479635346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe33815f720 0x7fe338155570 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.877+0000 7fe33da00640 1 -- 172.21.15.37:0/479635346 >> 172.21.15.37:0/479635346 conn(0x7fe3380d0460 msgr2=0x7fe33806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.877+0000 7fe33da00640 1 -- 172.21.15.37:0/479635346 shutdown_connections 2024-09-21T06:34:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.877+0000 7fe33da00640 1 -- 172.21.15.37:0/479635346 wait complete. 2024-09-21T06:34:42.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.879+0000 7fe33da00640 1 Processor -- start 2024-09-21T06:34:42.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.879+0000 7fe33da00640 1 -- start start 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.880+0000 7fe33da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 0x7fe3381425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.880+0000 7fe33da00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 0x7fe3381591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.880+0000 7fe33da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.880+0000 7fe33da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380af3a0 con 0x7fe33815f720 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.880+0000 7fe33da00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380af220 con 0x7fe3380732a0 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33da00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380af520 con 0x7fe3380b49b0 2024-09-21T06:34:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53508/0 (socket says 172.21.15.37:53508) 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 -- 172.21.15.37:0/1372769475 learned_addr learned my addr 172.21.15.37:0/1372769475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 -- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 msgr2=0x7fe3381591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe337e00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 0x7fe3381425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe32fe00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 0x7fe3381591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 0x7fe3381591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 -- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 msgr2=0x7fe3381425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 0x7fe3381425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe33d000640 1 -- 172.21.15.37:0/1372769475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe338142e40 con 0x7fe33815f720 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe337e00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 0x7fe3381425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.881+0000 7fe32fe00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 0x7fe3381591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe33d000640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe33000c6c0 tx=0x7fe33000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330017070 con 0x7fe33815f720 2024-09-21T06:34:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe330003c60 con 0x7fe33815f720 2024-09-21T06:34:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300052d0 con 0x7fe33815f720 2024-09-21T06:34:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe338143120 con 0x7fe33815f720 2024-09-21T06:34:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.882+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381436e0 con 0x7fe33815f720 2024-09-21T06:34:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.883+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe338155570 con 0x7fe33815f720 2024-09-21T06:34:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.884+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe33001a070 con 0x7fe33815f720 2024-09-21T06:34:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.885+0000 7fe334c00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 0x7fe31407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.886+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6649+0+0 (secure 0 0 0) 0x7fe33009a7b0 con 0x7fe33815f720 2024-09-21T06:34:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.886+0000 7fe337e00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 0x7fe31407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.887+0000 7fe337e00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 0x7fe31407b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe328034000 tx=0x7fe328004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:42.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:42.891+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3300627a0 con 0x7fe33815f720 2024-09-21T06:34:43.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.092+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe3380cfc10 con 0x7fe33815f720 2024-09-21T06:34:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.620+0000 7fe334c00640 1 -- 172.21.15.37:0/1372769475 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7fe3300620a0 con 0x7fe33815f720 2024-09-21T06:34:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 msgr2=0x7fe31407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 0x7fe31407b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe328034000 tx=0x7fe328004730 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 msgr2=0x7fe338144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe33000c6c0 tx=0x7fe33000cb80 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 shutdown_connections 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe314078b90 0x7fe31407b0a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe33815f720 0x7fe338144c80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3380b49b0 0x7fe3381591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 --2- 172.21.15.37:0/1372769475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3380732a0 0x7fe3381425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.624+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 >> 172.21.15.37:0/1372769475 conn(0x7fe3380d0460 msgr2=0x7fe3380af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.625+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 shutdown_connections 2024-09-21T06:34:43.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.625+0000 7fe33da00640 1 -- 172.21.15.37:0/1372769475 wait complete. 2024-09-21T06:34:43.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:34:43.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 -- 172.21.15.37:0/3151280954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 msgr2=0x7fbb2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:43.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 --2- 172.21.15.37:0/3151280954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb2400b9c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0ca640 tx=0x7fbb24005830 comp rx=0 tx=0).stop 2024-09-21T06:34:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 -- 172.21.15.37:0/3151280954 shutdown_connections 2024-09-21T06:34:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 --2- 172.21.15.37:0/3151280954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb2400b9c0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 -- 172.21.15.37:0/3151280954 >> 172.21.15.37:0/3151280954 conn(0x7fbb3c0ccac0 msgr2=0x7fbb3c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 -- 172.21.15.37:0/3151280954 shutdown_connections 2024-09-21T06:34:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.885+0000 7fbb43e00640 1 -- 172.21.15.37:0/3151280954 wait complete. 2024-09-21T06:34:43.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.887+0000 7fbb43e00640 1 Processor -- start 2024-09-21T06:34:43.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.887+0000 7fbb43e00640 1 -- start start 2024-09-21T06:34:43.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c0c9c00 0x7fbb3c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c0b1250 0x7fbb3c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c160280 con 0x7fbb24005250 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c160100 con 0x7fbb3c0b1250 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb43e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c160400 con 0x7fbb3c0c9c00 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60654/0 (socket says 172.21.15.37:60654) 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 -- 172.21.15.37:0/2807675358 learned_addr learned my addr 172.21.15.37:0/2807675358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c0b1250 0x7fbb3c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb40a00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c0c9c00 0x7fbb3c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 -- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c0c9c00 msgr2=0x7fbb3c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:43.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c0c9c00 0x7fbb3c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 -- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c0b1250 msgr2=0x7fbb3c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:43.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c0b1250 0x7fbb3c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:43.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.889+0000 7fbb41400640 1 -- 172.21.15.37:0/2807675358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c0b1660 con 0x7fbb24005250 2024-09-21T06:34:43.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb41400640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbb30011ae0 tx=0x7fbb30009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3001e070 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb30004480 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c1f4a60 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000fc60 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.890+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1f3f80 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.891+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c160dd0 con 0x7fbb24005250 2024-09-21T06:34:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.893+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb30021070 con 0x7fbb24005250 2024-09-21T06:34:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.893+0000 7fbb39600640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 0x7fbb1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.894+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6182+0+0 (secure 0 0 0) 0x7fbb3009c1c0 con 0x7fbb24005250 2024-09-21T06:34:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.894+0000 7fbb40a00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 0x7fbb1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.895+0000 7fbb40a00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 0x7fbb1c07b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c157e90 tx=0x7fbb28006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:43.897+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb3c0b1660 con 0x7fbb24005250 2024-09-21T06:34:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.093+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb3c0b5f20 con 0x7fbb24005250 2024-09-21T06:34:44.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.628+0000 7fbb39600640 1 -- 172.21.15.37:0/2807675358 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fbb30063e40 con 0x7fbb24005250 2024-09-21T06:34:44.630 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 msgr2=0x7fbb1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 0x7fbb1c07b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c157e90 tx=0x7fbb28006b70 comp rx=0 tx=0).stop 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 msgr2=0x7fbb3c1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbb30011ae0 tx=0x7fbb30009e40 comp rx=0 tx=0).stop 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 shutdown_connections 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb1c078b90 0x7fbb1c07b0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c0b1250 0x7fbb3c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c0c9c00 0x7fbb3c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 --2- 172.21.15.37:0/2807675358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb24005250 0x7fbb3c1550e0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.631+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 >> 172.21.15.37:0/2807675358 conn(0x7fbb3c0ccac0 msgr2=0x7fbb3c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:44.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.632+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 shutdown_connections 2024-09-21T06:34:44.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.632+0000 7fbb43e00640 1 -- 172.21.15.37:0/2807675358 wait complete. 2024-09-21T06:34:44.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 -- 172.21.15.37:0/930602925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0074b60 msgr2=0x7f01c006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 --2- 172.21.15.37:0/930602925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0074b60 0x7f01c006ddf0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f01c00b2a20 tx=0x7f01b002a560 comp rx=0 tx=0).stop 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 -- 172.21.15.37:0/930602925 shutdown_connections 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 --2- 172.21.15.37:0/930602925 >> v2:172.21.15.150:3300/0 conn(0x7f01c006e330 0x7f01c00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 --2- 172.21.15.37:0/930602925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0074b60 0x7f01c006ddf0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 --2- 172.21.15.37:0/930602925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0156a50 0x7f01c0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 -- 172.21.15.37:0/930602925 >> 172.21.15.37:0/930602925 conn(0x7f01c00b1b90 msgr2=0x7f01c00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 -- 172.21.15.37:0/930602925 shutdown_connections 2024-09-21T06:34:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.901+0000 7f01c8800640 1 -- 172.21.15.37:0/930602925 wait complete. 2024-09-21T06:34:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.904+0000 7f01c8800640 1 Processor -- start 2024-09-21T06:34:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.904+0000 7f01c8800640 1 -- start start 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c006e330 0x7f01c01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0074b60 0x7f01c00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00ce330 con 0x7f01c0156a50 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00ce1b0 con 0x7f01c006e330 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c8800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f01c00ce4b0 con 0x7f01c0074b60 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.905+0000 7f01c6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60670/0 (socket says 172.21.15.37:60670) 2024-09-21T06:34:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 -- 172.21.15.37:0/386155023 learned_addr learned my addr 172.21.15.37:0/386155023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c5e00640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c006e330 0x7f01c01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c5400640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0074b60 0x7f01c00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 -- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0074b60 msgr2=0x7f01c00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0074b60 0x7f01c00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 -- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c006e330 msgr2=0x7f01c01556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c006e330 0x7f01c01556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 -- 172.21.15.37:0/386155023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0155c10 con 0x7f01c0156a50 2024-09-21T06:34:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.906+0000 7f01c6800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00d8d0 tx=0x7f01bc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.907+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc016070 con 0x7f01c0156a50 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.907+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01bc00bd40 con 0x7f01c0156a50 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.907+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc003bd0 con 0x7f01c0156a50 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.907+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c00c9e80 con 0x7f01c0156a50 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.907+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c00ca440 con 0x7f01c0156a50 2024-09-21T06:34:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.908+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c00c9bf0 con 0x7f01c0156a50 2024-09-21T06:34:44.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.909+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01bc010460 con 0x7f01c0156a50 2024-09-21T06:34:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.910+0000 7f01ba000640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 0x7f01a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.910+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f01bc09b2f0 con 0x7f01c0156a50 2024-09-21T06:34:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.911+0000 7f01c5e00640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 0x7f01a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.911+0000 7f01c5e00640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 0x7f01a407b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005fd0 tx=0x7f01b400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:44.914+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01bc063080 con 0x7f01c0156a50 2024-09-21T06:34:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.115+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f01c00cfc00 con 0x7f01c0156a50 2024-09-21T06:34:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.119+0000 7f01ba000640 1 -- 172.21.15.37:0/386155023 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f01bc067ab0 con 0x7f01c0156a50 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 msgr2=0x7f01a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 0x7f01a407b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005fd0 tx=0x7f01b400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 msgr2=0x7f01c015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00d8d0 tx=0x7f01bc00dd90 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 shutdown_connections 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f01a4078b90 0x7f01a407b0a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0156a50 0x7f01c015ee60 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01c0074b60 0x7f01c00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 --2- 172.21.15.37:0/386155023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c006e330 0x7f01c01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.121+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 >> 172.21.15.37:0/386155023 conn(0x7f01c00b1b90 msgr2=0x7f01c00afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.122+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 shutdown_connections 2024-09-21T06:34:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.122+0000 7f01c8800640 1 -- 172.21.15.37:0/386155023 wait complete. 2024-09-21T06:34:45.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-21T06:34:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 -- 172.21.15.37:0/1717227519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 msgr2=0x7fbf0015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 --2- 172.21.15.37:0/1717227519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf0015ca00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005fd0 tx=0x7fbef002a530 comp rx=0 tx=0).stop 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 -- 172.21.15.37:0/1717227519 shutdown_connections 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 --2- 172.21.15.37:0/1717227519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf0006e500 0x7fbf000ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 --2- 172.21.15.37:0/1717227519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf0015ca00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 -- 172.21.15.37:0/1717227519 >> 172.21.15.37:0/1717227519 conn(0x7fbf00073400 msgr2=0x7fbf00073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 -- 172.21.15.37:0/1717227519 shutdown_connections 2024-09-21T06:34:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.371+0000 7fbeff200640 1 -- 172.21.15.37:0/1717227519 wait complete. 2024-09-21T06:34:45.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.373+0000 7fbeff200640 1 Processor -- start 2024-09-21T06:34:45.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.374+0000 7fbeff200640 1 -- start start 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf000afb80 0x7fbf001f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf000c9dc0 0x7fbf001f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000cf450 con 0x7fbf00073f20 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000cf2d0 con 0x7fbf000c9dc0 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbeff200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbf000cf5d0 con 0x7fbf000afb80 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbefde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbefde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60688/0 (socket says 172.21.15.37:60688) 2024-09-21T06:34:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.375+0000 7fbefde00640 1 -- 172.21.15.37:0/2620186299 learned_addr learned my addr 172.21.15.37:0/2620186299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 -- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf000afb80 msgr2=0x7fbf001f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf000afb80 0x7fbf001f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 -- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf000c9dc0 msgr2=0x7fbf001f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf000c9dc0 0x7fbf001f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 -- 172.21.15.37:0/2620186299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf001f7110 con 0x7fbf00073f20 2024-09-21T06:34:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbefde00640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbef0034040 tx=0x7fbef00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef003e030 con 0x7fbf00073f20 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef0003d90 con 0x7fbf00073f20 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0035040 con 0x7fbf00073f20 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.376+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf000b5500 con 0x7fbf00073f20 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.377+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf000b5ac0 con 0x7fbf00073f20 2024-09-21T06:34:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.378+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf000cfaa0 con 0x7fbf00073f20 2024-09-21T06:34:45.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.379+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbef0041030 con 0x7fbf00073f20 2024-09-21T06:34:45.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.380+0000 7fbef6000640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 0x7fbee007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.380+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fbef00bb8a0 con 0x7fbf00073f20 2024-09-21T06:34:45.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.380+0000 7fbefd400640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 0x7fbee007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.381+0000 7fbefd400640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 0x7fbee007b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006fd0 tx=0x7fbeec006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:45.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.384+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbef0083d00 con 0x7fbf00073f20 2024-09-21T06:34:45.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:45.655+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7fbf000b0400 con 0x7fbf00073f20 2024-09-21T06:34:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.632+0000 7fbef6000640 1 -- 172.21.15.37:0/2620186299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v364) ==== 154+0+0 (secure 0 0 0) 0x7fbef0083600 con 0x7fbf00073f20 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.635+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 msgr2=0x7fbee007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.635+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 0x7fbee007b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006fd0 tx=0x7fbeec006bc0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.635+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 msgr2=0x7fbf00157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.635+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbef0034040 tx=0x7fbef00038a0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 shutdown_connections 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbee0078b90 0x7fbee007b0a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf000c9dc0 0x7fbf001f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf000afb80 0x7fbf001f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 --2- 172.21.15.37:0/2620186299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf00073f20 0x7fbf00157bc0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 >> 172.21.15.37:0/2620186299 conn(0x7fbf00073400 msgr2=0x7fbf00159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 shutdown_connections 2024-09-21T06:34:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.636+0000 7fbeff200640 1 -- 172.21.15.37:0/2620186299 wait complete. 2024-09-21T06:34:46.664 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:34:46.671 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-21T06:34:46.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 -- 172.21.15.37:0/2460252958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc61815dfe0 msgr2=0x7fc618160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 --2- 172.21.15.37:0/2460252958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc61815dfe0 0x7fc618160a40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc608009830 tx=0x7fc60802a480 comp rx=0 tx=0).stop 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 -- 172.21.15.37:0/2460252958 shutdown_connections 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 --2- 172.21.15.37:0/2460252958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 0x7fc61806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 --2- 172.21.15.37:0/2460252958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc61815dfe0 0x7fc618160a40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 -- 172.21.15.37:0/2460252958 >> 172.21.15.37:0/2460252958 conn(0x7fc61806f940 msgr2=0x7fc6180b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 -- 172.21.15.37:0/2460252958 shutdown_connections 2024-09-21T06:34:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.903+0000 7fc61d800640 1 -- 172.21.15.37:0/2460252958 wait complete. 2024-09-21T06:34:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.906+0000 7fc61d800640 1 Processor -- start 2024-09-21T06:34:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.906+0000 7fc61d800640 1 -- start start 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 0x7fc618073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 0x7fc6181357b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180b4c50 con 0x7fc61815dfe0 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180b4ad0 con 0x7fc6180cc8d0 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.907+0000 7fc61d800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180b4dd0 con 0x7fc61806e160 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60702/0 (socket says 172.21.15.37:60702) 2024-09-21T06:34:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 -- 172.21.15.37:0/1597851494 learned_addr learned my addr 172.21.15.37:0/1597851494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc617e00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 0x7fc618073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc60f400640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 0x7fc6181357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 -- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 msgr2=0x7fc618073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 0x7fc618073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 -- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 msgr2=0x7fc6181357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 0x7fc6181357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc61ce00640 1 -- 172.21.15.37:0/1597851494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618135cf0 con 0x7fc61815dfe0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.908+0000 7fc60f400640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 0x7fc6181357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc61ce00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc61000ca50 tx=0x7fc61000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610016020 con 0x7fc61815dfe0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc61000aca0 con 0x7fc61815dfe0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6181f5a40 con 0x7fc61815dfe0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610012680 con 0x7fc61815dfe0 2024-09-21T06:34:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.909+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181f6000 con 0x7fc61815dfe0 2024-09-21T06:34:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.910+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6180b5310 con 0x7fc61815dfe0 2024-09-21T06:34:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.911+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc610019020 con 0x7fc61815dfe0 2024-09-21T06:34:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.913+0000 7fc614c00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 0x7fc5f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.913+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fc61009af70 con 0x7fc61815dfe0 2024-09-21T06:34:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.913+0000 7fc617e00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 0x7fc5f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.914+0000 7fc617e00640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 0x7fc5f407b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc608034000 tx=0x7fc608004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:46.918+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc610020030 con 0x7fc61815dfe0 2024-09-21T06:34:47.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.115+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6180cc0e0 con 0x7fc61815dfe0 2024-09-21T06:34:47.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.116+0000 7fc614c00640 1 -- 172.21.15.37:0/1597851494 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13582 (secure 0 0 0) 0x7fc610063460 con 0x7fc61815dfe0 2024-09-21T06:34:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:47.119 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":96,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:34:44.621680+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6837/3722358982":"2024-09-22T06:34:41.568706+0000","172.21.15.150:6835/3722358982":"2024-09-22T06:34:41.568706+0000","172.21.15.37:6835/1748999037":"2024-09-22T06:34:41.568724+0000","172.21.15.150:6836/2979710501":"2024-09-22T06:34:41.568730+0000","172.21.15.37:6832/1748999037":"2024-09-22T06:34:41.568724+0000","172.21.15.150:6834/2979710501":"2024-09-22T06:34:41.568730+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.118+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 msgr2=0x7fc5f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.118+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 0x7fc5f407b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc608034000 tx=0x7fc608004730 comp rx=0 tx=0).stop 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.118+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 msgr2=0x7fc6181f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.118+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc61000ca50 tx=0x7fc61000cf10 comp rx=0 tx=0).stop 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 shutdown_connections 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc5f4078b90 0x7fc5f407b0a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc61815dfe0 0x7fc6181f9010 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6180cc8d0 0x7fc6181357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 --2- 172.21.15.37:0/1597851494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc61806e160 0x7fc618073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 >> 172.21.15.37:0/1597851494 conn(0x7fc61806f940 msgr2=0x7fc6180afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 shutdown_connections 2024-09-21T06:34:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.119+0000 7fc61d800640 1 -- 172.21.15.37:0/1597851494 wait complete. 2024-09-21T06:34:47.144 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:27:49.118619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:34:47.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:34:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 -- 172.21.15.37:0/2034531875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a180b5640 msgr2=0x7f7a180b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 --2- 172.21.15.37:0/2034531875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a180b5640 0x7f7a180b5a50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7a18070b20 tx=0x7f7a1002aa30 comp rx=0 tx=0).stop 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 -- 172.21.15.37:0/2034531875 shutdown_connections 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 --2- 172.21.15.37:0/2034531875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a180b5640 0x7f7a180b5a50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 --2- 172.21.15.37:0/2034531875 >> v2:172.21.15.150:3300/0 conn(0x7f7a18072b90 0x7f7a18072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 --2- 172.21.15.37:0/2034531875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a181533b0 0x7f7a180cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 -- 172.21.15.37:0/2034531875 >> 172.21.15.37:0/2034531875 conn(0x7f7a180ccd70 msgr2=0x7f7a180cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 -- 172.21.15.37:0/2034531875 shutdown_connections 2024-09-21T06:34:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.371+0000 7f7a17e00640 1 -- 172.21.15.37:0/2034531875 wait complete. 2024-09-21T06:34:47.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.373+0000 7f7a17e00640 1 Processor -- start 2024-09-21T06:34:47.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.373+0000 7f7a17e00640 1 -- start start 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 0x7f7a18153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a180b5640 0x7f7a181564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a180cc0d0 con 0x7f7a181533b0 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a180cbf50 con 0x7f7a18072b90 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a18160050 con 0x7f7a180b5640 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60720/0 (socket says 172.21.15.37:60720) 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a17400640 1 -- 172.21.15.37:0/3487371392 learned_addr learned my addr 172.21.15.37:0/3487371392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a16a00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 0x7f7a18153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.375+0000 7f7a0fe00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a180b5640 0x7f7a181564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 -- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a180b5640 msgr2=0x7f7a181564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a180b5640 0x7f7a181564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 -- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 msgr2=0x7f7a18153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 0x7f7a18153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 -- 172.21.15.37:0/3487371392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a180b43f0 con 0x7f7a181533b0 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a16a00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 0x7f7a18153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17400640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d60 tx=0x7f7a10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10033050 con 0x7f7a181533b0 2024-09-21T06:34:47.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a10004450 con 0x7f7a181533b0 2024-09-21T06:34:47.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a180b46d0 con 0x7f7a181533b0 2024-09-21T06:34:47.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10038450 con 0x7f7a181533b0 2024-09-21T06:34:47.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.376+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181f5270 con 0x7f7a181533b0 2024-09-21T06:34:47.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.377+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18154220 con 0x7f7a181533b0 2024-09-21T06:34:47.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.380+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a10036070 con 0x7f7a181533b0 2024-09-21T06:34:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.381+0000 7f7a0ea00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 0x7f79f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.381+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f7a100bc130 con 0x7f7a181533b0 2024-09-21T06:34:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.381+0000 7f7a16a00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 0x7f79f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.381+0000 7f7a16a00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 0x7f79f407b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009b80 tx=0x7f7a08005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.385+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a100845a0 con 0x7f7a181533b0 2024-09-21T06:34:47.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.580+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a181f54d0 con 0x7f7a181533b0 2024-09-21T06:34:47.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.580+0000 7f7a0ea00640 1 -- 172.21.15.37:0/3487371392 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f7a10083ea0 con 0x7f7a181533b0 2024-09-21T06:34:47.583 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-21T06:34:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 msgr2=0x7f79f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 0x7f79f407b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009b80 tx=0x7f7a08005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 msgr2=0x7f7a181508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d60 tx=0x7f7a10003f10 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 shutdown_connections 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f79f4078b90 0x7f79f407b0a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a181533b0 0x7f7a181508c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a180b5640 0x7f7a181564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 --2- 172.21.15.37:0/3487371392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18072b90 0x7f7a18153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 >> 172.21.15.37:0/3487371392 conn(0x7f7a180ccd70 msgr2=0x7f7a18152940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 shutdown_connections 2024-09-21T06:34:47.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.583+0000 7f7a17e00640 1 -- 172.21.15.37:0/3487371392 wait complete. 2024-09-21T06:34:47.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 -- 172.21.15.37:0/2433351211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc06e500 msgr2=0x7f71cc0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 --2- 172.21.15.37:0/2433351211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc06e500 0x7f71cc0cc020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005fd0 tx=0x7f71c402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 -- 172.21.15.37:0/2433351211 shutdown_connections 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 --2- 172.21.15.37:0/2433351211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc06e500 0x7f71cc0cc020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 --2- 172.21.15.37:0/2433351211 >> v2:172.21.15.150:3300/0 conn(0x7f71cc074cb0 0x7f71cc073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 -- 172.21.15.37:0/2433351211 >> 172.21.15.37:0/2433351211 conn(0x7f71cc0cca40 msgr2=0x7f71cc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 -- 172.21.15.37:0/2433351211 shutdown_connections 2024-09-21T06:34:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.835+0000 7f71d3e00640 1 -- 172.21.15.37:0/2433351211 wait complete. 2024-09-21T06:34:47.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.837+0000 7f71d3e00640 1 Processor -- start 2024-09-21T06:34:47.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.837+0000 7f71d3e00640 1 -- start start 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71cc0cbc10 0x7f71cc1f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc157120 0x7f71cc09cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc06f3f0 con 0x7f71cc074cb0 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc06f270 con 0x7f71cc0cbc10 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d3e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc06f570 con 0x7f71cc157120 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60734/0 (socket says 172.21.15.37:60734) 2024-09-21T06:34:47.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 -- 172.21.15.37:0/3247192902 learned_addr learned my addr 172.21.15.37:0/3247192902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc157120 0x7f71cc09cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d0a00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71cc0cbc10 0x7f71cc1f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 -- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc157120 msgr2=0x7f71cc09cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc157120 0x7f71cc09cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 -- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71cc0cbc10 msgr2=0x7f71cc1f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71cc0cbc10 0x7f71cc1f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 -- 172.21.15.37:0/3247192902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1f37a0 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.839+0000 7f71d1400640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f71c000e790 tx=0x7f71c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.840+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c000eea0 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.840+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71c000ccc0 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.840+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc09c4e0 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.840+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0004280 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.840+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc09c780 con 0x7f71cc074cb0 2024-09-21T06:34:47.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.841+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc1f3970 con 0x7f71cc074cb0 2024-09-21T06:34:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.843+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71c00035f0 con 0x7f71cc074cb0 2024-09-21T06:34:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.844+0000 7f71c9600640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 0x7f71ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.844+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f71c009b670 con 0x7f71cc074cb0 2024-09-21T06:34:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.844+0000 7f71d0a00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 0x7f71ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.844+0000 7f71d0a00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 0x7f71ac07b0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f71b8007fd0 tx=0x7f71b8001390 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:47.848+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71c0063400 con 0x7f71cc074cb0 2024-09-21T06:34:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.123+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7f71cc06fa90 con 0x7f71cc074cb0 2024-09-21T06:34:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.642+0000 7f71c9600640 1 -- 172.21.15.37:0/3247192902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v366) ==== 168+0+0 (secure 0 0 0) 0x7f71c006c020 con 0x7f71cc074cb0 2024-09-21T06:34:48.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 msgr2=0x7f71ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:48.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 0x7f71ac07b0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f71b8007fd0 tx=0x7f71b8001390 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 msgr2=0x7f71cc156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f71c000e790 tx=0x7f71c000ec50 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 shutdown_connections 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71ac078b90 0x7f71ac07b0a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71cc157120 0x7f71cc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71cc0cbc10 0x7f71cc1f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 --2- 172.21.15.37:0/3247192902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71cc074cb0 0x7f71cc156590 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 >> 172.21.15.37:0/3247192902 conn(0x7f71cc0cca40 msgr2=0x7f71cc06dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 shutdown_connections 2024-09-21T06:34:48.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.645+0000 7f71d3e00640 1 -- 172.21.15.37:0/3247192902 wait complete. 2024-09-21T06:34:48.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:34:48.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 -- 172.21.15.37:0/1918704820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 msgr2=0x7f663c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 --2- 172.21.15.37:0/1918704820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c15e8a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f663c06dfa0 tx=0x7f662c02b1e0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 -- 172.21.15.37:0/1918704820 shutdown_connections 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 --2- 172.21.15.37:0/1918704820 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c0ccd60 0x7f663c0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 --2- 172.21.15.37:0/1918704820 >> v2:172.21.15.150:3300/0 conn(0x7f663c15ede0 0x7f663c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 --2- 172.21.15.37:0/1918704820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c15e8a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 -- 172.21.15.37:0/1918704820 >> 172.21.15.37:0/1918704820 conn(0x7f663c06f940 msgr2=0x7f663c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 -- 172.21.15.37:0/1918704820 shutdown_connections 2024-09-21T06:34:48.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.900+0000 7f6642000640 1 -- 172.21.15.37:0/1918704820 wait complete. 2024-09-21T06:34:48.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.902+0000 7f6642000640 1 Processor -- start 2024-09-21T06:34:48.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.902+0000 7f6642000640 1 -- start start 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f663c0ccd60 0x7f663c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c15ede0 0x7f663c151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f663c070e70 con 0x7f663c15b820 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f663c070cf0 con 0x7f663c0ccd60 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6642000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0ce300 con 0x7f663c15ede0 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6633e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6633e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60752/0 (socket says 172.21.15.37:60752) 2024-09-21T06:34:48.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6633e00640 1 -- 172.21.15.37:0/289456583 learned_addr learned my addr 172.21.15.37:0/289456583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6641600640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c15ede0 0x7f663c151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.904+0000 7f6640c00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f663c0ccd60 0x7f663c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 -- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c15ede0 msgr2=0x7f663c151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c15ede0 0x7f663c151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 -- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f663c0ccd60 msgr2=0x7f663c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f663c0ccd60 0x7f663c156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 -- 172.21.15.37:0/289456583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c156c80 con 0x7f663c15b820 2024-09-21T06:34:48.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6633e00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f662400d740 tx=0x7f662400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624017070 con 0x7f663c15b820 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6624009d20 con 0x7f663c15b820 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66240106c0 con 0x7f663c15b820 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.905+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c156f60 con 0x7f663c15b820 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.906+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c157520 con 0x7f663c15b820 2024-09-21T06:34:48.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.906+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0ce820 con 0x7f663c15b820 2024-09-21T06:34:48.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.908+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f662401a070 con 0x7f663c15b820 2024-09-21T06:34:48.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.909+0000 7f6639600640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 0x7f661807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:48.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.909+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f662409ad80 con 0x7f663c15b820 2024-09-21T06:34:48.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.910+0000 7f6640c00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 0x7f661807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:48.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.911+0000 7f6640c00640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 0x7f661807b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f662c033000 tx=0x7f662c035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:48.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:48.914+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6624062b90 con 0x7f663c15b820 2024-09-21T06:34:49.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.114+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f663c1569d0 con 0x7f663c15b820 2024-09-21T06:34:49.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.114+0000 7f6639600640 1 -- 172.21.15.37:0/289456583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f66240675c0 con 0x7f663c15b820 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 2888.132894 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2906.480142 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3257.554989 2024-09-21T06:34:49.116 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 2886.725239 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4011.204707 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7128.087440 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4154.847666 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.748683 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:34:49.117 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:34:49.118 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-21T06:34:49.118 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-21T06:34:49.118 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-21T06:34:49.118 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:34:49.118 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 msgr2=0x7f661807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 0x7f661807b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f662c033000 tx=0x7f662c035040 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 msgr2=0x7f663c147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f662400d740 tx=0x7f662400dc00 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 shutdown_connections 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6618078b90 0x7f661807b0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c15b820 0x7f663c147250 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f663c15ede0 0x7f663c151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 --2- 172.21.15.37:0/289456583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f663c0ccd60 0x7f663c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 >> 172.21.15.37:0/289456583 conn(0x7f663c06f940 msgr2=0x7f663c15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 shutdown_connections 2024-09-21T06:34:49.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.117+0000 7f6642000640 1 -- 172.21.15.37:0/289456583 wait complete. 2024-09-21T06:34:49.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.373+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1344456552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a00ccef0 msgr2=0x7fb7a00cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.373+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1344456552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a00ccef0 0x7fb7a00cddc0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00b2950 tx=0x7fb78c02a560 comp rx=0 tx=0).stop 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.373+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1344456552 shutdown_connections 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.374+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1344456552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a00ccef0 0x7fb7a00cddc0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.374+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1344456552 >> v2:172.21.15.150:3300/0 conn(0x7fb7a0160a00 0x7fb7a006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.374+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1344456552 >> 172.21.15.37:0/1344456552 conn(0x7fb7a006f920 msgr2=0x7fb7a00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.374+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1344456552 shutdown_connections 2024-09-21T06:34:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.374+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1344456552 wait complete. 2024-09-21T06:34:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.376+0000 7fb7a4e00640 1 Processor -- start 2024-09-21T06:34:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.376+0000 7fb7a4e00640 1 -- start start 2024-09-21T06:34:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.377+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a0160a00 0x7fb7a015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a015b1c0 0x7fb7a0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a006d4d0 con 0x7fb7a006e2c0 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a006d650 con 0x7fb7a0160a00 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a006d7d0 con 0x7fb7a015b1c0 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60770/0 (socket says 172.21.15.37:60770) 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 -- 172.21.15.37:0/1899164702 learned_addr learned my addr 172.21.15.37:0/1899164702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 -- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a015b1c0 msgr2=0x7fb7a0157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a015b1c0 0x7fb7a0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 -- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a0160a00 msgr2=0x7fb7a015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a0160a00 0x7fb7a015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.378+0000 7fb79f400640 1 -- 172.21.15.37:0/1899164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a0132aa0 con 0x7fb7a006e2c0 2024-09-21T06:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb79f400640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b420 tx=0x7fb79000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790011830 con 0x7fb7a006e2c0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb790016070 con 0x7fb7a006e2c0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790011d60 con 0x7fb7a006e2c0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a0132d80 con 0x7fb7a006e2c0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.379+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a0157570 con 0x7fb7a006e2c0 2024-09-21T06:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.380+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a006dc60 con 0x7fb7a006e2c0 2024-09-21T06:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.381+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb79001b460 con 0x7fb7a006e2c0 2024-09-21T06:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.382+0000 7fb797400640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 0x7fb78007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.382+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fb79009a890 con 0x7fb7a006e2c0 2024-09-21T06:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.383+0000 7fb79ea00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 0x7fb78007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.383+0000 7fb79ea00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 0x7fb78007b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb78c033000 tx=0x7fb78c0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:49.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.387+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb790020020 con 0x7fb7a006e2c0 2024-09-21T06:34:49.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.583+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb7a006fe30 con 0x7fb7a006e2c0 2024-09-21T06:34:49.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.587+0000 7fb797400640 1 -- 172.21.15.37:0/1899164702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7fb790062d00 con 0x7fb7a006e2c0 2024-09-21T06:34:49.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 msgr2=0x7fb78007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 0x7fb78007b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb78c033000 tx=0x7fb78c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 msgr2=0x7fb7a01302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b420 tx=0x7fb79000b8e0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 shutdown_connections 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb780078b90 0x7fb78007b0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a015b1c0 0x7fb7a0157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a0160a00 0x7fb7a015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 --2- 172.21.15.37:0/1899164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006e2c0 0x7fb7a01302c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 >> 172.21.15.37:0/1899164702 conn(0x7fb7a006f920 msgr2=0x7fb7a0145980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 shutdown_connections 2024-09-21T06:34:49.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:49.590+0000 7fb7a4e00640 1 -- 172.21.15.37:0/1899164702 wait complete. 2024-09-21T06:34:49.617 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:34:49.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:34:49.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:34:49.803 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:34:49.803 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:34:49.803 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:49.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:34:49.838 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:34:49.838 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:49.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:34:49.876 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:34:49.876 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:49.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:34:49.915 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:34:49.916 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:49.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:34:49.952 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:34:49.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 -- 172.21.15.37:0/1488491892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cfbc0 msgr2=0x7fb2b815e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 --2- 172.21.15.37:0/1488491892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cfbc0 0x7fb2b815e5e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009860 tx=0x7fb2a002a410 comp rx=0 tx=0).stop 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 -- 172.21.15.37:0/1488491892 shutdown_connections 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 --2- 172.21.15.37:0/1488491892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cfbc0 0x7fb2b815e5e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 -- 172.21.15.37:0/1488491892 >> 172.21.15.37:0/1488491892 conn(0x7fb2b80d4360 msgr2=0x7fb2b80d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 -- 172.21.15.37:0/1488491892 shutdown_connections 2024-09-21T06:34:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.181+0000 7fb2bce00640 1 -- 172.21.15.37:0/1488491892 wait complete. 2024-09-21T06:34:50.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.184+0000 7fb2bce00640 1 Processor -- start 2024-09-21T06:34:50.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.184+0000 7fb2bce00640 1 -- start start 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 0x7fb2b815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80cd660 0x7fb2b80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b8f20 con 0x7fb2b80ceff0 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b8da0 con 0x7fb2b80cd660 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.185+0000 7fb2bce00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80b90a0 con 0x7fb2b80d2520 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60782/0 (socket says 172.21.15.37:60782) 2024-09-21T06:34:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 -- 172.21.15.37:0/2326371999 learned_addr learned my addr 172.21.15.37:0/2326371999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2afe00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 0x7fb2b815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7e00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80cd660 0x7fb2b80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 -- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 msgr2=0x7fb2b815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 0x7fb2b815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 -- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80cd660 msgr2=0x7fb2b80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80cd660 0x7fb2b80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2b7400640 1 -- 172.21.15.37:0/2326371999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80ce070 con 0x7fb2b80ceff0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.186+0000 7fb2afe00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 0x7fb2b815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2b7400640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4d0 tx=0x7fb2a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8016020 con 0x7fb2b80ceff0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2a8012920 con 0x7fb2b80ceff0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8012d70 con 0x7fb2b80ceff0 2024-09-21T06:34:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b80aadb0 con 0x7fb2b80ceff0 2024-09-21T06:34:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.187+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b80ab370 con 0x7fb2b80ceff0 2024-09-21T06:34:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.188+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b80aabd0 con 0x7fb2b80ceff0 2024-09-21T06:34:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.189+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2a8019020 con 0x7fb2b80ceff0 2024-09-21T06:34:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.190+0000 7fb2af400640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 0x7fb29407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.190+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fb2a809a8f0 con 0x7fb2b80ceff0 2024-09-21T06:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.190+0000 7fb2afe00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 0x7fb29407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.191+0000 7fb2afe00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 0x7fb29407b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0034000 tx=0x7fb2a0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:50.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.194+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2b80aabd0 con 0x7fb2b80ceff0 2024-09-21T06:34:50.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.428+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2b815e5e0 con 0x7fb2b80ceff0 2024-09-21T06:34:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.429+0000 7fb2af400640 1 -- 172.21.15.37:0/2326371999 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+3918 (secure 0 0 0) 0x7fb2a8020030 con 0x7fb2b80ceff0 2024-09-21T06:34:50.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:50.430 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":101,"btime":"2024-09-21T06:34:42:608794+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/763522983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":763522983},{"type":"v1","addr":"172.21.15.37:6834","nonce":763522983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2655994640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2655994640},{"type":"v1","addr":"172.21.15.150:6835","nonce":2655994640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7723,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2010125352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2010125352},{"type":"v1","addr":"172.21.15.150:6837","nonce":2010125352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2175872352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2175872352},{"type":"v1","addr":"172.21.15.37:6835","nonce":2175872352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[]} 2024-09-21T06:34:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 101 2024-09-21T06:34:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 msgr2=0x7fb29407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 0x7fb29407b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0034000 tx=0x7fb2a0004730 comp rx=0 tx=0).stop 2024-09-21T06:34:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 msgr2=0x7fb2b80ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4d0 tx=0x7fb2a800c990 comp rx=0 tx=0).stop 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 shutdown_connections 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb294078bc0 0x7fb29407b0d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80cd660 0x7fb2b80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80d2520 0x7fb2b815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 --2- 172.21.15.37:0/2326371999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2b80ceff0 0x7fb2b80ab890 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 >> 172.21.15.37:0/2326371999 conn(0x7fb2b80d4360 msgr2=0x7fb2b8162a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 shutdown_connections 2024-09-21T06:34:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.431+0000 7fb2bce00640 1 -- 172.21.15.37:0/2326371999 wait complete. 2024-09-21T06:34:50.455 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:34:50.455 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:34:50.456 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:34:50.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:34:50.456 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:34:50.456 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:34:50.456 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:34:50.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:34:50.456 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:34:50.457 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:34:50.457 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:34:50.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:34:50.457 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:34:50.457 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:34:50.457 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:34:50.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:34:50.483 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:34:50.484 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:34:50.484 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:34:50.484 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:34:50.489 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:34:50.489 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:34:50.490 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:34:50.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:34:50.528 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:34:50.528 DEBUG:teuthology.parallel:result is None 2024-09-21T06:34:50.531 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:34:50.531 DEBUG:teuthology.parallel:result is None 2024-09-21T06:34:50.534 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:34:50.534 DEBUG:teuthology.parallel:result is None 2024-09-21T06:34:50.537 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:34:50.537 DEBUG:teuthology.parallel:result is None 2024-09-21T06:34:50.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:34:50.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:34:50.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:34:50.731 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-09-21T06:34:50.731 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:34:50.735 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:34:50.738 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:34:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 -- 172.21.15.37:0/582702595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 msgr2=0x7f378015e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 --2- 172.21.15.37:0/582702595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378015e920 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f378006fb20 tx=0x7f377002a560 comp rx=0 tx=0).stop 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 -- 172.21.15.37:0/582702595 shutdown_connections 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 --2- 172.21.15.37:0/582702595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3780071090 0x7f37801624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 --2- 172.21.15.37:0/582702595 >> v2:172.21.15.150:3300/0 conn(0x7f378006ff00 0x7f3780070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 --2- 172.21.15.37:0/582702595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378015e920 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 -- 172.21.15.37:0/582702595 >> 172.21.15.37:0/582702595 conn(0x7f37800cdca0 msgr2=0x7f37800ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 -- 172.21.15.37:0/582702595 shutdown_connections 2024-09-21T06:34:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.776+0000 7f3787600640 1 -- 172.21.15.37:0/582702595 wait complete. 2024-09-21T06:34:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.779+0000 7f3787600640 1 Processor -- start 2024-09-21T06:34:50.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.779+0000 7f3787600640 1 -- start start 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f378006ff00 0x7f3780161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3780071090 0x7f37800c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37801629e0 con 0x7f37800706e0 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f37800adff0 con 0x7f3780071090 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3787600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3780162b60 con 0x7f378006ff00 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.780+0000 7f3777e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60862/0 (socket says 172.21.15.37:60862) 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 -- 172.21.15.37:0/2707612554 learned_addr learned my addr 172.21.15.37:0/2707612554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3784c00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f378006ff00 0x7f3780161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3785600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3780071090 0x7f37800c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 -- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f378006ff00 msgr2=0x7f3780161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f378006ff00 0x7f3780161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 -- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3780071090 msgr2=0x7f37800c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:50.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3780071090 0x7f37800c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 -- 172.21.15.37:0/2707612554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378014fbe0 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.781+0000 7f3777e00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f376800e710 tx=0x7f376800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.782+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376800cab0 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.782+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f376800cd40 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.782+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768010690 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.782+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37800c5230 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.782+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378015f430 con 0x7f37800706e0 2024-09-21T06:34:50.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.783+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378015ea60 con 0x7f37800706e0 2024-09-21T06:34:50.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.784+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3768010a40 con 0x7f37800706e0 2024-09-21T06:34:50.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.785+0000 7f377d600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 0x7f375c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:50.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.785+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f376809b810 con 0x7f37800706e0 2024-09-21T06:34:50.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.786+0000 7f3784c00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 0x7f375c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:50.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.786+0000 7f3784c00640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 0x7f375c07b0d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3770033000 tx=0x7f3770004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:50.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:50.790+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f378015ea60 con 0x7f37800706e0 2024-09-21T06:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.001+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f37801630c0 con 0x7f37800706e0 2024-09-21T06:34:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.664+0000 7f377d600640 1 -- 172.21.15.37:0/2707612554 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) ==== 97+0+0 (secure 0 0 0) 0x7f378014fbe0 con 0x7f37800706e0 2024-09-21T06:34:51.666 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-21T06:34:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 msgr2=0x7f375c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 0x7f375c07b0d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3770033000 tx=0x7f3770004730 comp rx=0 tx=0).stop 2024-09-21T06:34:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 msgr2=0x7f378014f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f376800e710 tx=0x7f376800ebd0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 shutdown_connections 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f375c078bc0 0x7f375c07b0d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37800706e0 0x7f378014f6a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3780071090 0x7f37800c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 --2- 172.21.15.37:0/2707612554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f378006ff00 0x7f3780161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 >> 172.21.15.37:0/2707612554 conn(0x7f37800cdca0 msgr2=0x7f37800ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 shutdown_connections 2024-09-21T06:34:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.668+0000 7f3787600640 1 -- 172.21.15.37:0/2707612554 wait complete. 2024-09-21T06:34:51.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:34:51.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 -- 172.21.15.37:0/681320080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f799815f4c0 msgr2=0x7f79980b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 --2- 172.21.15.37:0/681320080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f799815f4c0 0x7f79980b2e40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f7990009830 tx=0x7f799002a530 comp rx=0 tx=0).stop 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 -- 172.21.15.37:0/681320080 shutdown_connections 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 --2- 172.21.15.37:0/681320080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f799815f4c0 0x7f79980b2e40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 -- 172.21.15.37:0/681320080 >> 172.21.15.37:0/681320080 conn(0x7f79980b6b20 msgr2=0x7f79980d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 -- 172.21.15.37:0/681320080 shutdown_connections 2024-09-21T06:34:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.968+0000 7f799ea00640 1 -- 172.21.15.37:0/681320080 wait complete. 2024-09-21T06:34:51.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.970+0000 7f799ea00640 1 Processor -- start 2024-09-21T06:34:51.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.971+0000 7f799ea00640 1 -- start start 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f799815f4c0 0x7f79981f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799806d1a0 0x7f799806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79980b3710 con 0x7f7998162280 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79980b3590 con 0x7f799806d1a0 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f79980b3890 con 0x7f799815f4c0 2024-09-21T06:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f798f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f798f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60890/0 (socket says 172.21.15.37:60890) 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f798f400640 1 -- 172.21.15.37:0/2121437484 learned_addr learned my addr 172.21.15.37:0/2121437484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f7997e00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f799815f4c0 0x7f79981f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.972+0000 7f799ca00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799806d1a0 0x7f799806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 -- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f799815f4c0 msgr2=0x7f79981f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f799815f4c0 0x7f79981f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 -- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799806d1a0 msgr2=0x7f799806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799806d1a0 0x7f799806d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 -- 172.21.15.37:0/2121437484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7998161ec0 con 0x7f7998162280 2024-09-21T06:34:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f798f400640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f798000cba0 tx=0x7f798000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980016020 con 0x7f7998162280 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f798000adf0 con 0x7f7998162280 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.973+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980012660 con 0x7f7998162280 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.974+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7998161040 con 0x7f7998162280 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.974+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981604c0 con 0x7f7998162280 2024-09-21T06:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.975+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7998162f00 con 0x7f7998162280 2024-09-21T06:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.976+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7980019020 con 0x7f7998162280 2024-09-21T06:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.977+0000 7f7994c00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.977+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f798009b0c0 con 0x7f7998162280 2024-09-21T06:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.977+0000 7f7997e00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.978+0000 7f7997e00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 0x7f797407b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7988001120 tx=0x7f7988009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:51.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:51.981+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7980020020 con 0x7f7998162280 2024-09-21T06:34:52.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.252+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f799815def0 con 0x7f7998162280 2024-09-21T06:34:52.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.253+0000 7f7994c00640 1 -- 172.21.15.37:0/2121437484 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f79800630f0 con 0x7f7998162280 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mtUeSHxAAELykzEcF0L6bChYPj57qdg==", 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:34:52.254 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m4wGJOxAAErxsMOb4h8Nw6xU4GUMP8w==", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5mHL6KLRAAGPI/6VSknyhbUVtITC/W+g==", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mQhK8DRAALbhlnXj50vfz0W5agaFLgg==", 2024-09-21T06:34:52.256 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAwZ+5mx3laORAAEyf6jMNlCbku4IwWGl98Vw==", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.257 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA0Z+5mHMwvGxAAjQq9h1aIIJyJCKIbbSeHxw==", 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-21T06:34:52.258 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA3Z+5m+nFyKhAAaAF1buiRPuWcfsSf043qJg==", 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6Z+5mkRRdORAADEK9tE1ujqYSC1XBv9rndw==", 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.260 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBBZ+5m9BQRAhAA3FTbN9HZw4HdtzN3BEdX5w==", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBEZ+5mhUldEBAASZGduSSGmbnNn9WVC7V+BA==", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBHZ+5mFQnIKxAAPQn5NgV3kCYyon74E2c0FA==", 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-21T06:34:52.262 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBKZ+5mHUQSOxAApitoUQcYXuyEzAOd+o8TlQ==", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA==", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:34:52.263 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mVPDGLxAAVk3KJBtgZtdCsbYMKsCtdA==", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-21T06:34:52.264 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mJC4kJRAA7gKlEHWHUU9M+Fpnt6eq1Q==", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAtZ+5mz3pMOhAA4VQHyz3JLqsHzHLR2iN6hA==", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-21T06:34:52.265 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQArZ+5mexylHRAAemlF4vTUZMe3W+OD6tX6ZA==", 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.266 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5md/egFBAA63aTsT/ExS2EvJvDkZ3z8g==", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQAsZ+5m8hQ4CBAA7Je6NpFJxrJBbT810AKsiQ==", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-21T06:34:52.267 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-21T06:34:52.268 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.268 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.268 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-21T06:34:52.268 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 msgr2=0x7f797407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 0x7f797407b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7988001120 tx=0x7f7988009040 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 msgr2=0x7f79980b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f798000cba0 tx=0x7f798000a430 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 shutdown_connections 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7998162280 0x7f79980b2960 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799806d1a0 0x7f799806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 --2- 172.21.15.37:0/2121437484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f799815f4c0 0x7f79981f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 >> 172.21.15.37:0/2121437484 conn(0x7f79980b6b20 msgr2=0x7f799806e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 shutdown_connections 2024-09-21T06:34:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.255+0000 7f799ea00640 1 -- 172.21.15.37:0/2121437484 wait complete. 2024-09-21T06:34:52.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:34:52.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 -- 172.21.15.37:0/2988617243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 msgr2=0x7fbd68160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 --2- 172.21.15.37:0/2988617243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68160a40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009830 tx=0x7fbd5802a480 comp rx=0 tx=0).stop 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 -- 172.21.15.37:0/2988617243 shutdown_connections 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 --2- 172.21.15.37:0/2988617243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd680ccfe0 0x7fbd6806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 --2- 172.21.15.37:0/2988617243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68160a40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 -- 172.21.15.37:0/2988617243 >> 172.21.15.37:0/2988617243 conn(0x7fbd6806f940 msgr2=0x7fbd680b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 -- 172.21.15.37:0/2988617243 shutdown_connections 2024-09-21T06:34:52.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.505+0000 7fbd6da00640 1 -- 172.21.15.37:0/2988617243 wait complete. 2024-09-21T06:34:52.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.507+0000 7fbd6da00640 1 Processor -- start 2024-09-21T06:34:52.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.507+0000 7fbd6da00640 1 -- start start 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd6806e160 0x7fbd681597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd68158440 0x7fbd68158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd680b4b80 con 0x7fbd6815dfe0 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbd680b4a00 con 0x7fbd68158440 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6da00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbd680b4d00 con 0x7fbd6806e160 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60908/0 (socket says 172.21.15.37:60908) 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 -- 172.21.15.37:0/3227642949 learned_addr learned my addr 172.21.15.37:0/3227642949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd67e00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd6806e160 0x7fbd681597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd6d000640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd68158440 0x7fbd68158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 -- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd6806e160 msgr2=0x7fbd681597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd6806e160 0x7fbd681597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 -- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd68158440 msgr2=0x7fbd68158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd68158440 0x7fbd68158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.509+0000 7fbd5fe00640 1 -- 172.21.15.37:0/3227642949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd68158d90 con 0x7fbd6815dfe0 2024-09-21T06:34:52.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd5fe00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c3c0 tx=0x7fbd5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50017070 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd500035f0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003a80 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd68155be0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.510+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd681561a0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.511+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd680b51b0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.513+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd5001a070 con 0x7fbd6815dfe0 2024-09-21T06:34:52.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.514+0000 7fbd64c00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 0x7fbd4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.514+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7fbd50099fc0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.514+0000 7fbd67e00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 0x7fbd4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.515+0000 7fbd67e00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 0x7fbd4407b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd58034000 tx=0x7fbd58004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.517+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd50061fe0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.716+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbd680cc0e0 con 0x7fbd6815dfe0 2024-09-21T06:34:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.716+0000 7fbd64c00640 1 -- 172.21.15.37:0/3227642949 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7fbd50066a10 con 0x7fbd6815dfe0 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:34:52.718 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 msgr2=0x7fbd4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 0x7fbd4407b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd58034000 tx=0x7fbd58004730 comp rx=0 tx=0).stop 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 msgr2=0x7fbd68157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c3c0 tx=0x7fbd5000c880 comp rx=0 tx=0).stop 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 shutdown_connections 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd44078b90 0x7fbd4407b0a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd6815dfe0 0x7fbd68157f00 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd68158440 0x7fbd68158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 --2- 172.21.15.37:0/3227642949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd6806e160 0x7fbd681597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 >> 172.21.15.37:0/3227642949 conn(0x7fbd6806f940 msgr2=0x7fbd680b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.718+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 shutdown_connections 2024-09-21T06:34:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.719+0000 7fbd6da00640 1 -- 172.21.15.37:0/3227642949 wait complete. 2024-09-21T06:34:52.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:34:52.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:34:52.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 -- 172.21.15.37:0/4269656083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 msgr2=0x7f25d40cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 --2- 172.21.15.37:0/4269656083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d40cca30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009be0 tx=0x7f25cc02a840 comp rx=0 tx=0).stop 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 -- 172.21.15.37:0/4269656083 shutdown_connections 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 --2- 172.21.15.37:0/4269656083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d40cca30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 --2- 172.21.15.37:0/4269656083 >> v2:172.21.15.150:3300/0 conn(0x7f25d4070e70 0x7f25d415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 -- 172.21.15.37:0/4269656083 >> 172.21.15.37:0/4269656083 conn(0x7f25d40cd970 msgr2=0x7f25d40cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 -- 172.21.15.37:0/4269656083 shutdown_connections 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.975+0000 7f25d9200640 1 -- 172.21.15.37:0/4269656083 wait complete. 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.977+0000 7f25d9200640 1 Processor -- start 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.978+0000 7f25d9200640 1 -- start start 2024-09-21T06:34:52.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d4070e70 0x7f25d413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d4142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40cd0f0 con 0x7f25d4144b70 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40ccf70 con 0x7f25d4070e70 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d9200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f25d40cd270 con 0x7f25d40cc620 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.979+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60916/0 (socket says 172.21.15.37:60916) 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 -- 172.21.15.37:0/359732244 learned_addr learned my addr 172.21.15.37:0/359732244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3400640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d4070e70 0x7f25d413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d2a00640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d4142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 -- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 msgr2=0x7f25d4142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d4142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 -- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d4070e70 msgr2=0x7f25d413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d4070e70 0x7f25d413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 -- 172.21.15.37:0/359732244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d41500c0 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.980+0000 7f25d3e00640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.981+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc042020 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.981+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25cc004040 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.981+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d414f300 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.981+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc037040 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.981+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d414f8c0 con 0x7f25d4144b70 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.982+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d4142f30 con 0x7f25d4144b70 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.983+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25cc045020 con 0x7f25d4144b70 2024-09-21T06:34:52.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.984+0000 7f25cb400640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 0x7f25b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.984+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f25cc0bba10 con 0x7f25d4144b70 2024-09-21T06:34:52.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.984+0000 7f25d3400640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 0x7f25b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.985+0000 7f25d3400640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 0x7f25b407b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f25c4007fd0 tx=0x7f25c4001370 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:52.989+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25cc04c030 con 0x7f25d4144b70 2024-09-21T06:34:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.189+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f25d40cca30 con 0x7f25d4144b70 2024-09-21T06:34:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.675+0000 7f25cb400640 1 -- 172.21.15.37:0/359732244 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f25cc02f070 con 0x7f25d4144b70 2024-09-21T06:34:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-21T06:34:53.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 msgr2=0x7f25b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:53.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 0x7f25b407b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f25c4007fd0 tx=0x7f25c4001370 comp rx=0 tx=0).stop 2024-09-21T06:34:53.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 msgr2=0x7f25d40ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:53.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc033dd0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 shutdown_connections 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f25b4078b90 0x7f25b407b0a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25d4144b70 0x7f25d40ca260 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25d40cc620 0x7f25d4142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 --2- 172.21.15.37:0/359732244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d4070e70 0x7f25d413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 >> 172.21.15.37:0/359732244 conn(0x7f25d40cd970 msgr2=0x7f25d4141710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 shutdown_connections 2024-09-21T06:34:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.678+0000 7f25d9200640 1 -- 172.21.15.37:0/359732244 wait complete. 2024-09-21T06:34:53.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.943+0000 7f12cd800640 1 -- 172.21.15.37:0/3988397120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac004800 msgr2=0x7f12ac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.943+0000 7f12cd800640 1 --2- 172.21.15.37:0/3988397120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac004800 0x7f12ac004c10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f12c80cd8c0 tx=0x7f12ac005830 comp rx=0 tx=0).stop 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.943+0000 7f12cd800640 1 -- 172.21.15.37:0/3988397120 shutdown_connections 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.943+0000 7f12cd800640 1 --2- 172.21.15.37:0/3988397120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac004800 0x7f12ac004c10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.943+0000 7f12cd800640 1 -- 172.21.15.37:0/3988397120 >> 172.21.15.37:0/3988397120 conn(0x7f12c80cca40 msgr2=0x7f12c80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.944+0000 7f12cd800640 1 -- 172.21.15.37:0/3988397120 shutdown_connections 2024-09-21T06:34:53.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.944+0000 7f12cd800640 1 -- 172.21.15.37:0/3988397120 wait complete. 2024-09-21T06:34:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.946+0000 7f12cd800640 1 Processor -- start 2024-09-21T06:34:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.946+0000 7f12cd800640 1 -- start start 2024-09-21T06:34:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12ac004800 0x7f12c80b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80afc90 0x7f12c809d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12c806e180 con 0x7f12c80c28d0 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12c806e000 con 0x7f12c80afc90 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12cd800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f12c80cbc10 con 0x7f12ac004800 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38154/0 (socket says 172.21.15.37:38154) 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 -- 172.21.15.37:0/3145231109 learned_addr learned my addr 172.21.15.37:0/3145231109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 -- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12ac004800 msgr2=0x7f12c80b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12ac004800 0x7f12c80b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 -- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80afc90 msgr2=0x7f12c809d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80afc90 0x7f12c809d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.948+0000 7f12c6000640 1 -- 172.21.15.37:0/3145231109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c80b00a0 con 0x7f12c80c28d0 2024-09-21T06:34:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12c6000640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009a70 tx=0x7f12b40336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4038040 con 0x7f12c80c28d0 2024-09-21T06:34:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b4041070 con 0x7f12c80c28d0 2024-09-21T06:34:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81f3a50 con 0x7f12c80c28d0 2024-09-21T06:34:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b403c430 con 0x7f12c80c28d0 2024-09-21T06:34:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.949+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81f4010 con 0x7f12c80c28d0 2024-09-21T06:34:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.950+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c81f37c0 con 0x7f12c80c28d0 2024-09-21T06:34:53.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.952+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12b403c5d0 con 0x7f12c80c28d0 2024-09-21T06:34:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.953+0000 7f12bea00640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 0x7f12a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.953+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5619+0+0 (secure 0 0 0) 0x7f12b40bb820 con 0x7f12c80c28d0 2024-09-21T06:34:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.954+0000 7f12c6a00640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 0x7f12a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.954+0000 7f12c6a00640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 0x7f12a807b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f12c806bb20 tx=0x7f12b80015b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:53.959+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12c80b00a0 con 0x7f12c80c28d0 2024-09-21T06:34:54.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.157+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12c80cc170 con 0x7f12c80c28d0 2024-09-21T06:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.683+0000 7f12bea00640 1 -- 172.21.15.37:0/3145231109 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f12b4083ca0 con 0x7f12c80c28d0 2024-09-21T06:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-21T06:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.687+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 msgr2=0x7f12a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.687+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 0x7f12a807b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f12c806bb20 tx=0x7f12b80015b0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.687+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 msgr2=0x7f12c80af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.687+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009a70 tx=0x7f12b40336f0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 shutdown_connections 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f12a8078b90 0x7f12a807b0a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12c80c28d0 0x7f12c80af750 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80afc90 0x7f12c809d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 --2- 172.21.15.37:0/3145231109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12ac004800 0x7f12c80b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.688+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 >> 172.21.15.37:0/3145231109 conn(0x7f12c80cca40 msgr2=0x7f12c80ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.689+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 shutdown_connections 2024-09-21T06:34:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.691+0000 7f12cd800640 1 -- 172.21.15.37:0/3145231109 wait complete. 2024-09-21T06:34:54.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:34:54.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 -- 172.21.15.37:0/2609240519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2860156a50 msgr2=0x7f2860074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 --2- 172.21.15.37:0/2609240519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2860156a50 0x7f2860074620 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2860074160 tx=0x7f285002a530 comp rx=0 tx=0).stop 2024-09-21T06:34:54.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 -- 172.21.15.37:0/2609240519 shutdown_connections 2024-09-21T06:34:54.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 --2- 172.21.15.37:0/2609240519 >> v2:172.21.15.150:3300/0 conn(0x7f2860074b60 0x7f286006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 --2- 172.21.15.37:0/2609240519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2860156a50 0x7f2860074620 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 -- 172.21.15.37:0/2609240519 >> 172.21.15.37:0/2609240519 conn(0x7f28600b1b90 msgr2=0x7f28600b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:54.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 -- 172.21.15.37:0/2609240519 shutdown_connections 2024-09-21T06:34:54.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.965+0000 7f2866600640 1 -- 172.21.15.37:0/2609240519 wait complete. 2024-09-21T06:34:54.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.967+0000 7f2866600640 1 Processor -- start 2024-09-21T06:34:54.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.967+0000 7f2866600640 1 -- start start 2024-09-21T06:34:54.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 0x7f286015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 0x7f2860159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28600ce500 con 0x7f286006e330 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28600ce380 con 0x7f2860156a50 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f2866600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f28600ce680 con 0x7f2860074b60 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38170/0 (socket says 172.21.15.37:38170) 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285f400640 1 -- 172.21.15.37:0/3242053461 learned_addr learned my addr 172.21.15.37:0/3242053461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285ea00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 0x7f286015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285fe00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 0x7f2860159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285f400640 1 -- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 msgr2=0x7f286015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.969+0000 7f285f400640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 0x7f286015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285f400640 1 -- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 msgr2=0x7f2860159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285f400640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 0x7f2860159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285f400640 1 -- 172.21.15.37:0/3242053461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286015f7d0 con 0x7f286006e330 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285ea00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 0x7f286015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285fe00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 0x7f2860159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f285f400640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f28500098f0 tx=0x7f2850033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850005c90 con 0x7f286006e330 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2850031e50 con 0x7f286006e330 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850005260 con 0x7f286006e330 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601f6950 con 0x7f286006e330 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.970+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601f3ef0 con 0x7f286006e330 2024-09-21T06:34:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.972+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286006ddf0 con 0x7f286006e330 2024-09-21T06:34:54.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.973+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2850005400 con 0x7f286006e330 2024-09-21T06:34:54.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.974+0000 7f2857400640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 0x7f284007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.974+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6001+0+0 (secure 0 0 0) 0x7f28500bbfe0 con 0x7f286006e330 2024-09-21T06:34:54.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.974+0000 7f285ea00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 0x7f284007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.975+0000 7f285ea00640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 0x7f284007b0d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2860070f50 tx=0x7f284c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:54.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:54.979+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f285003f030 con 0x7f286006e330 2024-09-21T06:34:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.215+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f28601579d0 con 0x7f286006e330 2024-09-21T06:34:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.686+0000 7f2857400640 1 -- 172.21.15.37:0/3242053461 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v104) ==== 174+0+0 (secure 0 0 0) 0x7f2850084290 con 0x7f286006e330 2024-09-21T06:34:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 15 and data pool 16 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.689+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 msgr2=0x7f284007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.689+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 0x7f284007b0d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2860070f50 tx=0x7f284c005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.689+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 msgr2=0x7f2860158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.689+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f28500098f0 tx=0x7f2850033c20 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 shutdown_connections 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2840078bc0 0x7f284007b0d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2860156a50 0x7f2860159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2860074b60 0x7f286015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 --2- 172.21.15.37:0/3242053461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286006e330 0x7f2860158100 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 >> 172.21.15.37:0/3242053461 conn(0x7f28600b1b90 msgr2=0x7f286015b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:55.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.690+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 shutdown_connections 2024-09-21T06:34:55.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.691+0000 7f2866600640 1 -- 172.21.15.37:0/3242053461 wait complete. 2024-09-21T06:34:55.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 -- 172.21.15.37:0/965063916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c0cb0c0 msgr2=0x7f696c0cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 --2- 172.21.15.37:0/965063916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c0cb0c0 0x7f696c0cb4d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6964005fd0 tx=0x7f696402a810 comp rx=0 tx=0).stop 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 -- 172.21.15.37:0/965063916 shutdown_connections 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 --2- 172.21.15.37:0/965063916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c0cb0c0 0x7f696c0cb4d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 --2- 172.21.15.37:0/965063916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c06e500 0x7f696c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 -- 172.21.15.37:0/965063916 >> 172.21.15.37:0/965063916 conn(0x7f696c073400 msgr2=0x7f696c073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 -- 172.21.15.37:0/965063916 shutdown_connections 2024-09-21T06:34:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.973+0000 7f6970c00640 1 -- 172.21.15.37:0/965063916 wait complete. 2024-09-21T06:34:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.976+0000 7f6970c00640 1 Processor -- start 2024-09-21T06:34:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.976+0000 7f6970c00640 1 -- start start 2024-09-21T06:34:55.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c0cb0c0 0x7f696c1f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c1f5de0 0x7f696c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0cbb90 con 0x7f696c073f20 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0cba10 con 0x7f696c1f5de0 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f6970c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0cbd10 con 0x7f696c0cb0c0 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f696b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f696b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38184/0 (socket says 172.21.15.37:38184) 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.977+0000 7f696b400640 1 -- 172.21.15.37:0/3600835810 learned_addr learned my addr 172.21.15.37:0/3600835810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f6962a00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c0cb0c0 0x7f696c1f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:55.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696be00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c1f5de0 0x7f696c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 -- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c0cb0c0 msgr2=0x7f696c1f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c0cb0c0 0x7f696c1f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 -- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c1f5de0 msgr2=0x7f696c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c1f5de0 0x7f696c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 -- 172.21.15.37:0/3600835810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c153340 con 0x7f696c073f20 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f696b400640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c430 tx=0x7f695c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.978+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c011790 con 0x7f696c073f20 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.979+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695c017070 con 0x7f696c073f20 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.979+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c153620 con 0x7f696c073f20 2024-09-21T06:34:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.979+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c011ce0 con 0x7f696c073f20 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.979+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c153be0 con 0x7f696c073f20 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.980+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c0cc1c0 con 0x7f696c073f20 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.981+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f695c01c460 con 0x7f696c073f20 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.982+0000 7f6963e00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 0x7f694807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.982+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f695c09adb0 con 0x7f696c073f20 2024-09-21T06:34:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.982+0000 7f6962a00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 0x7f694807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:55.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.983+0000 7f6962a00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 0x7f694807b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f696c0b5a10 tx=0x7f6954005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:55.987+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f695c09f070 con 0x7f696c073f20 2024-09-21T06:34:56.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.183+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f696c15b4f0 con 0x7f696c073f20 2024-09-21T06:34:56.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.184+0000 7f6963e00640 1 -- 172.21.15.37:0/3600835810 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16985 (secure 0 0 0) 0x7f695c062fc0 con 0x7f696c073f20 2024-09-21T06:34:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:56.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:34:55.216526+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:53.201605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:54.169408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.186+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 msgr2=0x7f694807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.186+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 0x7f694807b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f696c0b5a10 tx=0x7f6954005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 msgr2=0x7f696c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c430 tx=0x7f695c00c8f0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 shutdown_connections 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6948078b90 0x7f694807b0a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f696c1f5de0 0x7f696c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f696c0cb0c0 0x7f696c1f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 --2- 172.21.15.37:0/3600835810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c073f20 0x7f696c154e40 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 >> 172.21.15.37:0/3600835810 conn(0x7f696c073400 msgr2=0x7f696c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 shutdown_connections 2024-09-21T06:34:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.187+0000 7f6970c00640 1 -- 172.21.15.37:0/3600835810 wait complete. 2024-09-21T06:34:56.214 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 -- 172.21.15.37:0/3933467970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c15ca00 msgr2=0x7f8c3c0afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3933467970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c0afb80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8c34005fd0 tx=0x7f8c3402a8d0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 -- 172.21.15.37:0/3933467970 shutdown_connections 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3933467970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c0afb80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 -- 172.21.15.37:0/3933467970 >> 172.21.15.37:0/3933467970 conn(0x7f8c3c073400 msgr2=0x7f8c3c073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 -- 172.21.15.37:0/3933467970 shutdown_connections 2024-09-21T06:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.442+0000 7f8c41a00640 1 -- 172.21.15.37:0/3933467970 wait complete. 2024-09-21T06:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.444+0000 7f8c41a00640 1 Processor -- start 2024-09-21T06:34:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.445+0000 7f8c41a00640 1 -- start start 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c3c073f20 0x7f8c3c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c157070 0x7f8c3c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cf5f0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cf470 con 0x7f8c3c157070 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c41a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cf770 con 0x7f8c3c073f20 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38196/0 (socket says 172.21.15.37:38196) 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.446+0000 7f8c3b400640 1 -- 172.21.15.37:0/3181740681 learned_addr learned my addr 172.21.15.37:0/3181740681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c41000640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c157070 0x7f8c3c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3be00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c3c073f20 0x7f8c3c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 -- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c3c073f20 msgr2=0x7f8c3c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c3c073f20 0x7f8c3c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 -- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c157070 msgr2=0x7f8c3c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c157070 0x7f8c3c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 -- 172.21.15.37:0/3181740681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c1568e0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c3b400640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c500 tx=0x7f8c2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:56.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011790 con 0x7f8c3c15ca00 2024-09-21T06:34:56.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c28017070 con 0x7f8c3c15ca00 2024-09-21T06:34:56.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011c80 con 0x7f8c3c15ca00 2024-09-21T06:34:56.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.447+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3c157be0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.448+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3c1581a0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.449+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3c0c3ea0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.450+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c2801c460 con 0x7f8c3c15ca00 2024-09-21T06:34:56.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.451+0000 7f8c33e00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 0x7f8c1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.451+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f8c2809ac70 con 0x7f8c3c15ca00 2024-09-21T06:34:56.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.451+0000 7f8c3be00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 0x7f8c1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.452+0000 7f8c3be00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 0x7f8c1c07b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c001120 tx=0x7f8c2c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.456+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c280628d0 con 0x7f8c3c15ca00 2024-09-21T06:34:56.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.656+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c3c0cfc20 con 0x7f8c3c15ca00 2024-09-21T06:34:56.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.657+0000 7f8c33e00640 1 -- 172.21.15.37:0/3181740681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16985 (secure 0 0 0) 0x7f8c28067300 con 0x7f8c3c15ca00 2024-09-21T06:34:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:34:55.216526+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:53.201605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:54.169408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 msgr2=0x7f8c1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 0x7f8c1c07b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c001120 tx=0x7f8c2c009040 comp rx=0 tx=0).stop 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 msgr2=0x7f8c3c156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c500 tx=0x7f8c2800c9c0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 shutdown_connections 2024-09-21T06:34:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8c1c078b90 0x7f8c1c07b0a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3c15ca00 0x7f8c3c156b30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3c157070 0x7f8c3c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 --2- 172.21.15.37:0/3181740681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c3c073f20 0x7f8c3c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 >> 172.21.15.37:0/3181740681 conn(0x7f8c3c073400 msgr2=0x7f8c3c06dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 shutdown_connections 2024-09-21T06:34:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.660+0000 7f8c41a00640 1 -- 172.21.15.37:0/3181740681 wait complete. 2024-09-21T06:34:56.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 -- 172.21.15.37:0/4284000251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 msgr2=0x7fc4600b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 --2- 172.21.15.37:0/4284000251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc4600b4f90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc450005fd0 tx=0x7fc45002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 -- 172.21.15.37:0/4284000251 shutdown_connections 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 --2- 172.21.15.37:0/4284000251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4600ccfc0 0x7fc4600cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 --2- 172.21.15.37:0/4284000251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc4600b4f90 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 -- 172.21.15.37:0/4284000251 >> 172.21.15.37:0/4284000251 conn(0x7fc46006f920 msgr2=0x7fc4600b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:56.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 -- 172.21.15.37:0/4284000251 shutdown_connections 2024-09-21T06:34:56.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.914+0000 7fc45f200640 1 -- 172.21.15.37:0/4284000251 wait complete. 2024-09-21T06:34:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.917+0000 7fc45f200640 1 Processor -- start 2024-09-21T06:34:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.917+0000 7fc45f200640 1 -- start start 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc460156c60 0x7fc46014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc460151140 0x7fc460151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc46006d610 con 0x7fc46006e370 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc46006d490 con 0x7fc460156c60 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.918+0000 7fc45f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc46006d790 con 0x7fc460151140 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38216/0 (socket says 172.21.15.37:38216) 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 -- 172.21.15.37:0/1827021238 learned_addr learned my addr 172.21.15.37:0/1827021238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc455400640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc460156c60 0x7fc46014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45e800640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc460151140 0x7fc460151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 -- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc460151140 msgr2=0x7fc460151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc460151140 0x7fc460151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 -- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc460156c60 msgr2=0x7fc46014d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc460156c60 0x7fc46014d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.919+0000 7fc45de00640 1 -- 172.21.15.37:0/1827021238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601500a0 con 0x7fc46006e370 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc45de00640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc4500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45003e030 con 0x7fc46006e370 2024-09-21T06:34:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc450003ca0 con 0x7fc46006e370 2024-09-21T06:34:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc460150380 con 0x7fc46006e370 2024-09-21T06:34:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450035040 con 0x7fc46006e370 2024-09-21T06:34:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.920+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc460150940 con 0x7fc46006e370 2024-09-21T06:34:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.921+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46006dcb0 con 0x7fc46006e370 2024-09-21T06:34:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.922+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc450041030 con 0x7fc46006e370 2024-09-21T06:34:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.923+0000 7fc456a00640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 0x7fc43c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.923+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc4500bb510 con 0x7fc46006e370 2024-09-21T06:34:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.924+0000 7fc455400640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 0x7fc43c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.924+0000 7fc455400640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 0x7fc43c07b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc460156500 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:56.928+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc4500830d0 con 0x7fc46006e370 2024-09-21T06:34:57.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.166+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc46015dce0 con 0x7fc46006e370 2024-09-21T06:34:57.676 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:34:57.675+0000 7f3a2a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:34:57.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.717+0000 7fc456a00640 1 -- 172.21.15.37:0/1827021238 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v107) ==== 118+0+0 (secure 0 0 0) 0x7fc450087b00 con 0x7fc46006e370 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 msgr2=0x7fc43c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 0x7fc43c07b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc460156500 tx=0x7fc448009040 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 msgr2=0x7fc460159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc4500038a0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 shutdown_connections 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc43c078b90 0x7fc43c07b0a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc460151140 0x7fc460151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc460156c60 0x7fc46014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 --2- 172.21.15.37:0/1827021238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc46006e370 0x7fc460159e20 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 >> 172.21.15.37:0/1827021238 conn(0x7fc46006f920 msgr2=0x7fc46015e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 shutdown_connections 2024-09-21T06:34:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.720+0000 7fc45f200640 1 -- 172.21.15.37:0/1827021238 wait complete. 2024-09-21T06:34:57.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:34:57.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:34:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.974+0000 7fcdb6c00640 1 -- 172.21.15.37:0/1282126631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb00ccdd0 msgr2=0x7fcdb015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.974+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/1282126631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb00ccdd0 0x7fcdb015e790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009830 tx=0x7fcd9c02a510 comp rx=0 tx=0).stop 2024-09-21T06:34:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 -- 172.21.15.37:0/1282126631 shutdown_connections 2024-09-21T06:34:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/1282126631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015ecd0 0x7fcdb00cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/1282126631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb00ccdd0 0x7fcdb015e790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 -- 172.21.15.37:0/1282126631 >> 172.21.15.37:0/1282126631 conn(0x7fcdb006f920 msgr2=0x7fcdb00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 -- 172.21.15.37:0/1282126631 shutdown_connections 2024-09-21T06:34:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.975+0000 7fcdb6c00640 1 -- 172.21.15.37:0/1282126631 wait complete. 2024-09-21T06:34:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.977+0000 7fcdb6c00640 1 Processor -- start 2024-09-21T06:34:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.977+0000 7fcdb6c00640 1 -- start start 2024-09-21T06:34:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015b7a0 0x7fcdb0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb01f8a00 0x7fcdb01f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0070e70 con 0x7fcdb015ecd0 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0070cf0 con 0x7fcdb01f8a00 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00ce300 con 0x7fcdb015b7a0 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38232/0 (socket says 172.21.15.37:38232) 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 -- 172.21.15.37:0/3388633741 learned_addr learned my addr 172.21.15.37:0/3388633741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb5800640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015b7a0 0x7fcdb0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb6200640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb01f8a00 0x7fcdb01f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 -- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015b7a0 msgr2=0x7fcdb0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015b7a0 0x7fcdb0150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 -- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb01f8a00 msgr2=0x7fcdb01f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb01f8a00 0x7fcdb01f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.979+0000 7fcdb4e00640 1 -- 172.21.15.37:0/3388633741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb01f9350 con 0x7fcdb015ecd0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdb4e00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c02ab00 tx=0x7fcd9c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c005c90 con 0x7fcdb015ecd0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd9c031e50 con 0x7fcdb015ecd0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb0151040 con 0x7fcdb015ecd0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c005260 con 0x7fcdb015ecd0 2024-09-21T06:34:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.980+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb0153cc0 con 0x7fcdb015ecd0 2024-09-21T06:34:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.981+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb0153750 con 0x7fcdb015ecd0 2024-09-21T06:34:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.983+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd9c005400 con 0x7fcdb015ecd0 2024-09-21T06:34:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.983+0000 7fcdad600640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 0x7fcd9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.984+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fcd9c0bbfd0 con 0x7fcdb015ecd0 2024-09-21T06:34:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.984+0000 7fcdb5800640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 0x7fcd9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:57.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.985+0000 7fcdb5800640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 0x7fcd9007b0a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcda4001710 tx=0x7fcda4007c60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:57.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:57.988+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcdb01f9350 con 0x7fcdb015ecd0 2024-09-21T06:34:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.225+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcdb00ce820 con 0x7fcdb015ecd0 2024-09-21T06:34:58.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.719+0000 7fcdad600640 1 -- 172.21.15.37:0/3388633741 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v108) ==== 122+0+0 (secure 0 0 0) 0x7fcd9c03f030 con 0x7fcdb015ecd0 2024-09-21T06:34:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 msgr2=0x7fcd9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 0x7fcd9007b0a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcda4001710 tx=0x7fcda4007c60 comp rx=0 tx=0).stop 2024-09-21T06:34:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 msgr2=0x7fcdb013f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c02ab00 tx=0x7fcd9c033c20 comp rx=0 tx=0).stop 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 shutdown_connections 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd90078b90 0x7fcd9007b0a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb015ecd0 0x7fcdb013f950 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb01f8a00 0x7fcdb01f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 --2- 172.21.15.37:0/3388633741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdb015b7a0 0x7fcdb0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 >> 172.21.15.37:0/3388633741 conn(0x7fcdb006f920 msgr2=0x7fcdb0159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 shutdown_connections 2024-09-21T06:34:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.722+0000 7fcdb6c00640 1 -- 172.21.15.37:0/3388633741 wait complete. 2024-09-21T06:34:58.748 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:34:58.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:34:58.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 -- 172.21.15.37:0/1681361781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0d2520 msgr2=0x7f7d0c0cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 --2- 172.21.15.37:0/1681361781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0cf4b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c08d1a0 tx=0x7f7cf402a410 comp rx=0 tx=0).stop 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 -- 172.21.15.37:0/1681361781 shutdown_connections 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 --2- 172.21.15.37:0/1681361781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0cf4b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 --2- 172.21.15.37:0/1681361781 >> v2:172.21.15.150:3300/0 conn(0x7f7d0c0ceff0 0x7f7d0c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 -- 172.21.15.37:0/1681361781 >> 172.21.15.37:0/1681361781 conn(0x7f7d0c0d4360 msgr2=0x7f7d0c0d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 -- 172.21.15.37:0/1681361781 shutdown_connections 2024-09-21T06:34:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.975+0000 7f7d12000640 1 -- 172.21.15.37:0/1681361781 wait complete. 2024-09-21T06:34:58.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.977+0000 7f7d12000640 1 Processor -- start 2024-09-21T06:34:58.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.978+0000 7f7d12000640 1 -- start start 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d0c0ceff0 0x7f7d0c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1628d0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0b9510 con 0x7f7d0c0d2520 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d12000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c162a50 con 0x7f7d0c0ceff0 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38244/0 (socket says 172.21.15.37:38244) 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.979+0000 7f7d03e00640 1 -- 172.21.15.37:0/3132044776 learned_addr learned my addr 172.21.15.37:0/3132044776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d10c00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d0c0ceff0 0x7f7d0c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d11600640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 -- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d0c0ceff0 msgr2=0x7f7d0c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d0c0ceff0 0x7f7d0c1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 -- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 msgr2=0x7f7d0c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 -- 172.21.15.37:0/3132044776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c158c90 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d11600640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d03e00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009780 tx=0x7f7cf4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4031d60 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf4033e10 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.980+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4005260 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.981+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c1f59f0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.981+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c1580f0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.982+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c157b80 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.983+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cf4005400 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.984+0000 7f7d09600640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.984+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7cf40bc1e0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.984+0000 7f7d10c00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.985+0000 7f7d10c00640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 0x7f7ce807b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005fd0 tx=0x7f7cfc00b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:58.988+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7cf403f030 con 0x7f7d0c0cf9f0 2024-09-21T06:34:59.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.224+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0c15e5e0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.225+0000 7f7d09600640 1 -- 172.21.15.37:0/3132044776 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5234 (secure 0 0 0) 0x7f7cf4083da0 con 0x7f7d0c0cf9f0 2024-09-21T06:34:59.226 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:59.226 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":108,"btime":"2024-09-21T06:34:58:717858+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":8172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:34:58.226135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175]},"id":7}]} 2024-09-21T06:34:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 108 2024-09-21T06:34:59.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 msgr2=0x7f7ce807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 0x7f7ce807b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005fd0 tx=0x7f7cfc00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 msgr2=0x7f7d0c158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009780 tx=0x7f7cf4033c20 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 shutdown_connections 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0c0cf9f0 0x7f7d0c158750 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c0d2520 0x7f7d0c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 --2- 172.21.15.37:0/3132044776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d0c0ceff0 0x7f7d0c1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 >> 172.21.15.37:0/3132044776 conn(0x7f7d0c0d4360 msgr2=0x7f7d0c161750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 shutdown_connections 2024-09-21T06:34:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.227+0000 7f7d12000640 1 -- 172.21.15.37:0/3132044776 wait complete. 2024-09-21T06:34:59.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 -- 172.21.15.37:0/115503117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08200cede0 msgr2=0x7f08200cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 --2- 172.21.15.37:0/115503117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08200cede0 0x7f08200cf7d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0814009830 tx=0x7f081402a4a0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 -- 172.21.15.37:0/115503117 shutdown_connections 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 --2- 172.21.15.37:0/115503117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08201622a0 0x7f08201626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 --2- 172.21.15.37:0/115503117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08200cede0 0x7f08200cf7d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 -- 172.21.15.37:0/115503117 >> 172.21.15.37:0/115503117 conn(0x7f08200d4360 msgr2=0x7f08200d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 -- 172.21.15.37:0/115503117 shutdown_connections 2024-09-21T06:34:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.474+0000 7f0826600640 1 -- 172.21.15.37:0/115503117 wait complete. 2024-09-21T06:34:59.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.477+0000 7f0826600640 1 Processor -- start 2024-09-21T06:34:59.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.477+0000 7f0826600640 1 -- start start 2024-09-21T06:34:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08200cede0 0x7f0820153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 0x7f082015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0820155c50 con 0x7f08201622a0 2024-09-21T06:34:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0820163130 con 0x7f0820154f20 2024-09-21T06:34:59.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0826600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f08200cfd10 con 0x7f08200cede0 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.478+0000 7f0825c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f081fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 0x7f082015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38260/0 (socket says 172.21.15.37:38260) 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 -- 172.21.15.37:0/2913794099 learned_addr learned my addr 172.21.15.37:0/2913794099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825200640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08200cede0 0x7f0820153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 -- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08200cede0 msgr2=0x7f0820153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08200cede0 0x7f0820153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 -- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 msgr2=0x7f082015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 0x7f082015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 -- 172.21.15.37:0/2913794099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08201e8160 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f081fe00640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 0x7f082015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.479+0000 7f0825c00640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f081800cb10 tx=0x7f081800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.480+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818016020 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.480+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081800ad80 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.480+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818012680 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.480+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08201e8400 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.480+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201e89c0 con 0x7f08201622a0 2024-09-21T06:34:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.481+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08200cf7d0 con 0x7f08201622a0 2024-09-21T06:34:59.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.482+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0818019020 con 0x7f08201622a0 2024-09-21T06:34:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.483+0000 7f081cc00640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 0x7f080007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.483+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f081809b510 con 0x7f08201622a0 2024-09-21T06:34:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.484+0000 7f0825200640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 0x7f080007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.485+0000 7f0825200640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 0x7f080007b0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0814034000 tx=0x7f0814004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.488+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0818020020 con 0x7f08201622a0 2024-09-21T06:34:59.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.685+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08200ce620 con 0x7f08201622a0 2024-09-21T06:34:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.686+0000 7f081cc00640 1 -- 172.21.15.37:0/2913794099 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16972 (secure 0 0 0) 0x7f0820162bf0 con 0x7f08201622a0 2024-09-21T06:34:59.688 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:34:59.688 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":104,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:34:56.686475+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:53.201605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:54.169408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 msgr2=0x7f080007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 0x7f080007b0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0814034000 tx=0x7f0814004730 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 msgr2=0x7f082015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f081800cb10 tx=0x7f081800a060 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 shutdown_connections 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0800078b90 0x7f080007b0a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08201622a0 0x7f082015ba80 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0820154f20 0x7f082015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 --2- 172.21.15.37:0/2913794099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08200cede0 0x7f0820153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 >> 172.21.15.37:0/2913794099 conn(0x7f08200d4360 msgr2=0x7f082006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 shutdown_connections 2024-09-21T06:34:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.689+0000 7f0826600640 1 -- 172.21.15.37:0/2913794099 wait complete. 2024-09-21T06:34:59.717 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:34:59.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 -- 172.21.15.37:0/324259494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7220003a50 msgr2=0x7f7220003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 --2- 172.21.15.37:0/324259494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7220003a50 0x7f7220003ec0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f722c001690 tx=0x7f7220005830 comp rx=0 tx=0).stop 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 -- 172.21.15.37:0/324259494 shutdown_connections 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 --2- 172.21.15.37:0/324259494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7220003a50 0x7f7220003ec0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 -- 172.21.15.37:0/324259494 >> 172.21.15.37:0/324259494 conn(0x7f723c0cca40 msgr2=0x7f723c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 -- 172.21.15.37:0/324259494 shutdown_connections 2024-09-21T06:34:59.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.945+0000 7f7242400640 1 -- 172.21.15.37:0/324259494 wait complete. 2024-09-21T06:34:59.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.948+0000 7f7242400640 1 Processor -- start 2024-09-21T06:34:59.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.948+0000 7f7242400640 1 -- start start 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7220003a50 0x7f723c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f723c09eb50 0x7f723c09ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0cc100 con 0x7f723c159240 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0cbf80 con 0x7f723c09eb50 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7242400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0cc280 con 0x7f7220003a50 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.949+0000 7f7232a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38272/0 (socket says 172.21.15.37:38272) 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 -- 172.21.15.37:0/3994736058 learned_addr learned my addr 172.21.15.37:0/3994736058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f723be00640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f723c09eb50 0x7f723c09ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f723b400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7220003a50 0x7f723c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 -- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7220003a50 msgr2=0x7f723c158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7220003a50 0x7f723c158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 -- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f723c09eb50 msgr2=0x7f723c09ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f723c09eb50 0x7f723c09ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 -- 172.21.15.37:0/3994736058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0a0500 con 0x7f723c159240 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7232a00640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f722400e540 tx=0x7f722400ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.950+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224015020 con 0x7f723c159240 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.951+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7224010980 con 0x7f723c159240 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.951+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c1e68b0 con 0x7f723c159240 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.951+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224010dd0 con 0x7f723c159240 2024-09-21T06:34:59.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.951+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c0a0a80 con 0x7f723c159240 2024-09-21T06:34:59.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.952+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c0cdfa0 con 0x7f723c159240 2024-09-21T06:34:59.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.953+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7224018020 con 0x7f723c159240 2024-09-21T06:34:59.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.954+0000 7f7233e00640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 0x7f721807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:59.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.954+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f722409b2f0 con 0x7f723c159240 2024-09-21T06:34:59.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.954+0000 7f723b400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 0x7f721807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:59.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.955+0000 7f723b400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 0x7f721807b0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f722c032000 tx=0x7f722c02c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:59.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:34:59.958+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f723c0cdfa0 con 0x7f723c159240 2024-09-21T06:35:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.153+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f723c0a0df0 con 0x7f7218078bc0 2024-09-21T06:35:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.158+0000 7f7233e00640 1 -- 172.21.15.37:0/3994736058 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f723c0a0df0 con 0x7f7218078bc0 2024-09-21T06:35:00.162 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 msgr2=0x7f721807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 0x7f721807b0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f722c032000 tx=0x7f722c02c040 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 msgr2=0x7f723c09e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f722400e540 tx=0x7f722400ea00 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 shutdown_connections 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7218078bc0 0x7f721807b0d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f723c159240 0x7f723c09e610 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f723c09eb50 0x7f723c09ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 --2- 172.21.15.37:0/3994736058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7220003a50 0x7f723c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 >> 172.21.15.37:0/3994736058 conn(0x7f723c0cca40 msgr2=0x7f723c15f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 shutdown_connections 2024-09-21T06:35:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.162+0000 7f7242400640 1 -- 172.21.15.37:0/3994736058 wait complete. 2024-09-21T06:35:00.185 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T06:34:59.205143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260700,"kb_used_data":11160,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714020,"statfs":{"total":773094113280,"available":772827156480,"internally_reserved":0,"allocated":11427840,"data_stored":5911950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.525742"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491654+0000","last_change":"2024-09-21T06:34:55.702355+0000","last_active":"2024-09-21T06:34:57.491654+0000","last_peered":"2024-09-21T06:34:57.491654+0000","last_clean":"2024-09-21T06:34:57.491654+0000","last_became_active":"2024-09-21T06:34:55.701180+0000","last_became_peered":"2024-09-21T06:34:55.701180+0000","last_unstale":"2024-09-21T06:34:57.491654+0000","last_undegraded":"2024-09-21T06:34:57.491654+0000","last_fullsized":"2024-09-21T06:34:57.491654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:40.085758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690863+0000","last_change":"2024-09-21T06:34:55.734713+0000","last_active":"2024-09-21T06:34:56.690863+0000","last_peered":"2024-09-21T06:34:56.690863+0000","last_clean":"2024-09-21T06:34:56.690863+0000","last_became_active":"2024-09-21T06:34:55.734367+0000","last_became_peered":"2024-09-21T06:34:55.734367+0000","last_unstale":"2024-09-21T06:34:56.690863+0000","last_undegraded":"2024-09-21T06:34:56.690863+0000","last_fullsized":"2024-09-21T06:34:56.690863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.630265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689595+0000","last_change":"2024-09-21T06:34:54.689595+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689595+0000","last_undegraded":"2024-09-21T06:34:54.689595+0000","last_fullsized":"2024-09-21T06:34:54.689595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491281+0000","last_change":"2024-09-21T06:34:55.713592+0000","last_active":"2024-09-21T06:34:57.491281+0000","last_peered":"2024-09-21T06:34:57.491281+0000","last_clean":"2024-09-21T06:34:57.491281+0000","last_became_active":"2024-09-21T06:34:55.711240+0000","last_became_peered":"2024-09-21T06:34:55.711240+0000","last_unstale":"2024-09-21T06:34:57.491281+0000","last_undegraded":"2024-09-21T06:34:57.491281+0000","last_fullsized":"2024-09-21T06:34:57.491281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.722733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491268+0000","last_change":"2024-09-21T06:34:55.712441+0000","last_active":"2024-09-21T06:34:57.491268+0000","last_peered":"2024-09-21T06:34:57.491268+0000","last_clean":"2024-09-21T06:34:57.491268+0000","last_became_active":"2024-09-21T06:34:55.710543+0000","last_became_peered":"2024-09-21T06:34:55.710543+0000","last_unstale":"2024-09-21T06:34:57.491268+0000","last_undegraded":"2024-09-21T06:34:57.491268+0000","last_fullsized":"2024-09-21T06:34:57.491268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:02.131613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897116+0000","last_change":"2024-09-21T06:34:55.727939+0000","last_active":"2024-09-21T06:34:56.897116+0000","last_peered":"2024-09-21T06:34:56.897116+0000","last_clean":"2024-09-21T06:34:56.897116+0000","last_became_active":"2024-09-21T06:34:55.727032+0000","last_became_peered":"2024-09-21T06:34:55.727032+0000","last_unstale":"2024-09-21T06:34:56.897116+0000","last_undegraded":"2024-09-21T06:34:56.897116+0000","last_fullsized":"2024-09-21T06:34:56.897116+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.403443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.896947+0000","last_change":"2024-09-21T06:34:55.723696+0000","last_active":"2024-09-21T06:34:56.896947+0000","last_peered":"2024-09-21T06:34:56.896947+0000","last_clean":"2024-09-21T06:34:56.896947+0000","last_became_active":"2024-09-21T06:34:55.723259+0000","last_became_peered":"2024-09-21T06:34:55.723259+0000","last_unstale":"2024-09-21T06:34:56.896947+0000","last_undegraded":"2024-09-21T06:34:56.896947+0000","last_fullsized":"2024-09-21T06:34:56.896947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:07.748611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691935+0000","last_change":"2024-09-21T06:34:55.723651+0000","last_active":"2024-09-21T06:34:56.691935+0000","last_peered":"2024-09-21T06:34:56.691935+0000","last_clean":"2024-09-21T06:34:56.691935+0000","last_became_active":"2024-09-21T06:34:55.723288+0000","last_became_peered":"2024-09-21T06:34:55.723288+0000","last_unstale":"2024-09-21T06:34:56.691935+0000","last_undegraded":"2024-09-21T06:34:56.691935+0000","last_fullsized":"2024-09-21T06:34:56.691935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:30.694885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691167+0000","last_change":"2024-09-21T06:34:55.704172+0000","last_active":"2024-09-21T06:34:56.691167+0000","last_peered":"2024-09-21T06:34:56.691167+0000","last_clean":"2024-09-21T06:34:56.691167+0000","last_became_active":"2024-09-21T06:34:55.703829+0000","last_became_peered":"2024-09-21T06:34:55.703829+0000","last_unstale":"2024-09-21T06:34:56.691167+0000","last_undegraded":"2024-09-21T06:34:56.691167+0000","last_fullsized":"2024-09-21T06:34:56.691167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:10.923582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693214+0000","last_change":"2024-09-21T06:34:55.714365+0000","last_active":"2024-09-21T06:34:56.693214+0000","last_peered":"2024-09-21T06:34:56.693214+0000","last_clean":"2024-09-21T06:34:56.693214+0000","last_became_active":"2024-09-21T06:34:55.711411+0000","last_became_peered":"2024-09-21T06:34:55.711411+0000","last_unstale":"2024-09-21T06:34:56.693214+0000","last_undegraded":"2024-09-21T06:34:56.693214+0000","last_fullsized":"2024-09-21T06:34:56.693214+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:55.929081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721189+0000","last_change":"2024-09-21T06:34:55.712794+0000","last_active":"2024-09-21T06:34:55.721189+0000","last_peered":"2024-09-21T06:34:55.721189+0000","last_clean":"2024-09-21T06:34:55.721189+0000","last_became_active":"2024-09-21T06:34:55.712487+0000","last_became_peered":"2024-09-21T06:34:55.712487+0000","last_unstale":"2024-09-21T06:34:55.721189+0000","last_undegraded":"2024-09-21T06:34:55.721189+0000","last_fullsized":"2024-09-21T06:34:55.721189+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:48.749080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897088+0000","last_change":"2024-09-21T06:34:55.720101+0000","last_active":"2024-09-21T06:34:56.897088+0000","last_peered":"2024-09-21T06:34:56.897088+0000","last_clean":"2024-09-21T06:34:56.897088+0000","last_became_active":"2024-09-21T06:34:55.719743+0000","last_became_peered":"2024-09-21T06:34:55.719743+0000","last_unstale":"2024-09-21T06:34:56.897088+0000","last_undegraded":"2024-09-21T06:34:56.897088+0000","last_fullsized":"2024-09-21T06:34:56.897088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:52.305232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491183+0000","last_change":"2024-09-21T06:34:55.712044+0000","last_active":"2024-09-21T06:34:57.491183+0000","last_peered":"2024-09-21T06:34:57.491183+0000","last_clean":"2024-09-21T06:34:57.491183+0000","last_became_active":"2024-09-21T06:34:55.710142+0000","last_became_peered":"2024-09-21T06:34:55.710142+0000","last_unstale":"2024-09-21T06:34:57.491183+0000","last_undegraded":"2024-09-21T06:34:57.491183+0000","last_fullsized":"2024-09-21T06:34:57.491183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:42.402252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897146+0000","last_change":"2024-09-21T06:34:55.722424+0000","last_active":"2024-09-21T06:34:56.897146+0000","last_peered":"2024-09-21T06:34:56.897146+0000","last_clean":"2024-09-21T06:34:56.897146+0000","last_became_active":"2024-09-21T06:34:55.722091+0000","last_became_peered":"2024-09-21T06:34:55.722091+0000","last_unstale":"2024-09-21T06:34:56.897146+0000","last_undegraded":"2024-09-21T06:34:56.897146+0000","last_fullsized":"2024-09-21T06:34:56.897146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:23.664483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690751+0000","last_change":"2024-09-21T06:34:55.727747+0000","last_active":"2024-09-21T06:34:56.690751+0000","last_peered":"2024-09-21T06:34:56.690751+0000","last_clean":"2024-09-21T06:34:56.690751+0000","last_became_active":"2024-09-21T06:34:55.727447+0000","last_became_peered":"2024-09-21T06:34:55.727447+0000","last_unstale":"2024-09-21T06:34:56.690751+0000","last_undegraded":"2024-09-21T06:34:56.690751+0000","last_fullsized":"2024-09-21T06:34:56.690751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.086183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683012+0000","last_change":"2024-09-21T06:34:55.710554+0000","last_active":"2024-09-21T06:34:57.683012+0000","last_peered":"2024-09-21T06:34:57.683012+0000","last_clean":"2024-09-21T06:34:57.683012+0000","last_became_active":"2024-09-21T06:34:55.710216+0000","last_became_peered":"2024-09-21T06:34:55.710216+0000","last_unstale":"2024-09-21T06:34:57.683012+0000","last_undegraded":"2024-09-21T06:34:57.683012+0000","last_fullsized":"2024-09-21T06:34:57.683012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.693106+0000","last_change":"2024-09-21T06:34:54.693106+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.693106+0000","last_undegraded":"2024-09-21T06:34:54.693106+0000","last_fullsized":"2024-09-21T06:34:54.693106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690915+0000","last_change":"2024-09-21T06:34:55.702753+0000","last_active":"2024-09-21T06:34:56.690915+0000","last_peered":"2024-09-21T06:34:56.690915+0000","last_clean":"2024-09-21T06:34:56.690915+0000","last_became_active":"2024-09-21T06:34:55.702431+0000","last_became_peered":"2024-09-21T06:34:55.702431+0000","last_unstale":"2024-09-21T06:34:56.690915+0000","last_undegraded":"2024-09-21T06:34:56.690915+0000","last_fullsized":"2024-09-21T06:34:56.690915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:07.262224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491255+0000","last_change":"2024-09-21T06:34:55.731627+0000","last_active":"2024-09-21T06:34:57.491255+0000","last_peered":"2024-09-21T06:34:57.491255+0000","last_clean":"2024-09-21T06:34:57.491255+0000","last_became_active":"2024-09-21T06:34:55.731309+0000","last_became_peered":"2024-09-21T06:34:55.731309+0000","last_unstale":"2024-09-21T06:34:57.491255+0000","last_undegraded":"2024-09-21T06:34:57.491255+0000","last_fullsized":"2024-09-21T06:34:57.491255+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:24.150794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:55.710593+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:55.710307+0000","last_became_peered":"2024-09-21T06:34:55.710307+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:54.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689114+0000","last_change":"2024-09-21T06:34:54.689114+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689114+0000","last_undegraded":"2024-09-21T06:34:54.689114+0000","last_fullsized":"2024-09-21T06:34:54.689114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731832+0000","last_change":"2024-09-21T06:34:55.728815+0000","last_active":"2024-09-21T06:34:55.731832+0000","last_peered":"2024-09-21T06:34:55.731832+0000","last_clean":"2024-09-21T06:34:55.731832+0000","last_became_active":"2024-09-21T06:34:55.728227+0000","last_became_peered":"2024-09-21T06:34:55.728227+0000","last_unstale":"2024-09-21T06:34:55.731832+0000","last_undegraded":"2024-09-21T06:34:55.731832+0000","last_fullsized":"2024-09-21T06:34:55.731832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:32.261965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692011+0000","last_change":"2024-09-21T06:34:55.731414+0000","last_active":"2024-09-21T06:34:56.692011+0000","last_peered":"2024-09-21T06:34:56.692011+0000","last_clean":"2024-09-21T06:34:56.692011+0000","last_became_active":"2024-09-21T06:34:55.731103+0000","last_became_peered":"2024-09-21T06:34:55.731103+0000","last_unstale":"2024-09-21T06:34:56.692011+0000","last_undegraded":"2024-09-21T06:34:56.692011+0000","last_fullsized":"2024-09-21T06:34:56.692011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691979+0000","last_change":"2024-09-21T06:34:55.715389+0000","last_active":"2024-09-21T06:34:56.691979+0000","last_peered":"2024-09-21T06:34:56.691979+0000","last_clean":"2024-09-21T06:34:56.691979+0000","last_became_active":"2024-09-21T06:34:55.715071+0000","last_became_peered":"2024-09-21T06:34:55.715071+0000","last_unstale":"2024-09-21T06:34:56.691979+0000","last_undegraded":"2024-09-21T06:34:56.691979+0000","last_fullsized":"2024-09-21T06:34:56.691979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682949+0000","last_change":"2024-09-21T06:34:55.717327+0000","last_active":"2024-09-21T06:34:57.682949+0000","last_peered":"2024-09-21T06:34:57.682949+0000","last_clean":"2024-09-21T06:34:57.682949+0000","last_became_active":"2024-09-21T06:34:55.716998+0000","last_became_peered":"2024-09-21T06:34:55.716998+0000","last_unstale":"2024-09-21T06:34:57.682949+0000","last_undegraded":"2024-09-21T06:34:57.682949+0000","last_fullsized":"2024-09-21T06:34:57.682949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683071+0000","last_change":"2024-09-21T06:34:55.717181+0000","last_active":"2024-09-21T06:34:57.683071+0000","last_peered":"2024-09-21T06:34:57.683071+0000","last_clean":"2024-09-21T06:34:57.683071+0000","last_became_active":"2024-09-21T06:34:55.716922+0000","last_became_peered":"2024-09-21T06:34:55.716922+0000","last_unstale":"2024-09-21T06:34:57.683071+0000","last_undegraded":"2024-09-21T06:34:57.683071+0000","last_fullsized":"2024-09-21T06:34:57.683071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683004+0000","last_change":"2024-09-21T06:34:55.715886+0000","last_active":"2024-09-21T06:34:57.683004+0000","last_peered":"2024-09-21T06:34:57.683004+0000","last_clean":"2024-09-21T06:34:57.683004+0000","last_became_active":"2024-09-21T06:34:55.715533+0000","last_became_peered":"2024-09-21T06:34:55.715533+0000","last_unstale":"2024-09-21T06:34:57.683004+0000","last_undegraded":"2024-09-21T06:34:57.683004+0000","last_fullsized":"2024-09-21T06:34:57.683004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897335+0000","last_change":"2024-09-21T06:34:55.730908+0000","last_active":"2024-09-21T06:34:56.897335+0000","last_peered":"2024-09-21T06:34:56.897335+0000","last_clean":"2024-09-21T06:34:56.897335+0000","last_became_active":"2024-09-21T06:34:55.730593+0000","last_became_peered":"2024-09-21T06:34:55.730593+0000","last_unstale":"2024-09-21T06:34:56.897335+0000","last_undegraded":"2024-09-21T06:34:56.897335+0000","last_fullsized":"2024-09-21T06:34:56.897335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692651+0000","last_change":"2024-09-21T06:34:55.728164+0000","last_active":"2024-09-21T06:34:56.692651+0000","last_peered":"2024-09-21T06:34:56.692651+0000","last_clean":"2024-09-21T06:34:56.692651+0000","last_became_active":"2024-09-21T06:34:55.727836+0000","last_became_peered":"2024-09-21T06:34:55.727836+0000","last_unstale":"2024-09-21T06:34:56.692651+0000","last_undegraded":"2024-09-21T06:34:56.692651+0000","last_fullsized":"2024-09-21T06:34:56.692651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.918656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721753+0000","last_change":"2024-09-21T06:34:55.714924+0000","last_active":"2024-09-21T06:34:55.721753+0000","last_peered":"2024-09-21T06:34:55.721753+0000","last_clean":"2024-09-21T06:34:55.721753+0000","last_became_active":"2024-09-21T06:34:55.713658+0000","last_became_peered":"2024-09-21T06:34:55.713658+0000","last_unstale":"2024-09-21T06:34:55.721753+0000","last_undegraded":"2024-09-21T06:34:55.721753+0000","last_fullsized":"2024-09-21T06:34:55.721753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:07.310461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897057+0000","last_change":"2024-09-21T06:34:55.718692+0000","last_active":"2024-09-21T06:34:56.897057+0000","last_peered":"2024-09-21T06:34:56.897057+0000","last_clean":"2024-09-21T06:34:56.897057+0000","last_became_active":"2024-09-21T06:34:55.718323+0000","last_became_peered":"2024-09-21T06:34:55.718323+0000","last_unstale":"2024-09-21T06:34:56.897057+0000","last_undegraded":"2024-09-21T06:34:56.897057+0000","last_fullsized":"2024-09-21T06:34:56.897057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:21.216504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.715947+0000","last_change":"2024-09-21T06:34:55.711836+0000","last_active":"2024-09-21T06:34:55.715947+0000","last_peered":"2024-09-21T06:34:55.715947+0000","last_clean":"2024-09-21T06:34:55.715947+0000","last_became_active":"2024-09-21T06:34:55.711556+0000","last_became_peered":"2024-09-21T06:34:55.711556+0000","last_unstale":"2024-09-21T06:34:55.715947+0000","last_undegraded":"2024-09-21T06:34:55.715947+0000","last_fullsized":"2024-09-21T06:34:55.715947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:24.575657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.690579+0000","last_change":"2024-09-21T06:34:54.690579+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.690579+0000","last_undegraded":"2024-09-21T06:34:54.690579+0000","last_fullsized":"2024-09-21T06:34:54.690579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.730527+0000","last_change":"2024-09-21T06:34:54.714493+0000","last_active":"2024-09-21T06:34:55.730527+0000","last_peered":"2024-09-21T06:34:55.730527+0000","last_clean":"2024-09-21T06:34:55.730527+0000","last_became_active":"2024-09-21T06:34:54.714069+0000","last_became_peered":"2024-09-21T06:34:54.714069+0000","last_unstale":"2024-09-21T06:34:55.730527+0000","last_undegraded":"2024-09-21T06:34:55.730527+0000","last_fullsized":"2024-09-21T06:34:55.730527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:49.070679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692607+0000","last_change":"2024-09-21T06:34:55.731634+0000","last_active":"2024-09-21T06:34:56.692607+0000","last_peered":"2024-09-21T06:34:56.692607+0000","last_clean":"2024-09-21T06:34:56.692607+0000","last_became_active":"2024-09-21T06:34:55.731317+0000","last_became_peered":"2024-09-21T06:34:55.731317+0000","last_unstale":"2024-09-21T06:34:56.692607+0000","last_undegraded":"2024-09-21T06:34:56.692607+0000","last_fullsized":"2024-09-21T06:34:56.692607+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.714077+0000","last_change":"2024-09-21T06:34:54.710899+0000","last_active":"2024-09-21T06:34:54.714077+0000","last_peered":"2024-09-21T06:34:54.714077+0000","last_clean":"2024-09-21T06:34:54.714077+0000","last_became_active":"2024-09-21T06:34:54.710535+0000","last_became_peered":"2024-09-21T06:34:54.710535+0000","last_unstale":"2024-09-21T06:34:54.714077+0000","last_undegraded":"2024-09-21T06:34:54.714077+0000","last_fullsized":"2024-09-21T06:34:54.714077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:53.792219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691077+0000","last_change":"2024-09-21T06:34:55.707231+0000","last_active":"2024-09-21T06:34:56.691077+0000","last_peered":"2024-09-21T06:34:56.691077+0000","last_clean":"2024-09-21T06:34:56.691077+0000","last_became_active":"2024-09-21T06:34:55.706910+0000","last_became_peered":"2024-09-21T06:34:55.706910+0000","last_unstale":"2024-09-21T06:34:56.691077+0000","last_undegraded":"2024-09-21T06:34:56.691077+0000","last_fullsized":"2024-09-21T06:34:56.691077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:31.422885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691219+0000","last_change":"2024-09-21T06:34:54.704122+0000","last_active":"2024-09-21T06:34:56.691219+0000","last_peered":"2024-09-21T06:34:56.691219+0000","last_clean":"2024-09-21T06:34:56.691219+0000","last_became_active":"2024-09-21T06:34:54.703803+0000","last_became_peered":"2024-09-21T06:34:54.703803+0000","last_unstale":"2024-09-21T06:34:56.691219+0000","last_undegraded":"2024-09-21T06:34:56.691219+0000","last_fullsized":"2024-09-21T06:34:56.691219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:49.261246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.737546+0000","last_change":"2024-09-21T06:34:55.731772+0000","last_active":"2024-09-21T06:34:55.737546+0000","last_peered":"2024-09-21T06:34:55.737546+0000","last_clean":"2024-09-21T06:34:55.737546+0000","last_became_active":"2024-09-21T06:34:55.731464+0000","last_became_peered":"2024-09-21T06:34:55.731464+0000","last_unstale":"2024-09-21T06:34:55.737546+0000","last_undegraded":"2024-09-21T06:34:55.737546+0000","last_fullsized":"2024-09-21T06:34:55.737546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:41.639926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692910+0000","last_change":"2024-09-21T06:34:54.701225+0000","last_active":"2024-09-21T06:34:56.692910+0000","last_peered":"2024-09-21T06:34:56.692910+0000","last_clean":"2024-09-21T06:34:56.692910+0000","last_became_active":"2024-09-21T06:34:54.700905+0000","last_became_peered":"2024-09-21T06:34:54.700905+0000","last_unstale":"2024-09-21T06:34:56.692910+0000","last_undegraded":"2024-09-21T06:34:56.692910+0000","last_fullsized":"2024-09-21T06:34:56.692910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491690+0000","last_change":"2024-09-21T06:34:55.698335+0000","last_active":"2024-09-21T06:34:57.491690+0000","last_peered":"2024-09-21T06:34:57.491690+0000","last_clean":"2024-09-21T06:34:57.491690+0000","last_became_active":"2024-09-21T06:34:55.697982+0000","last_became_peered":"2024-09-21T06:34:55.697982+0000","last_unstale":"2024-09-21T06:34:57.491690+0000","last_undegraded":"2024-09-21T06:34:57.491690+0000","last_fullsized":"2024-09-21T06:34:57.491690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:33.941883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.732028+0000","last_change":"2024-09-21T06:34:54.714529+0000","last_active":"2024-09-21T06:34:55.732028+0000","last_peered":"2024-09-21T06:34:55.732028+0000","last_clean":"2024-09-21T06:34:55.732028+0000","last_became_active":"2024-09-21T06:34:54.714159+0000","last_became_peered":"2024-09-21T06:34:54.714159+0000","last_unstale":"2024-09-21T06:34:55.732028+0000","last_undegraded":"2024-09-21T06:34:55.732028+0000","last_fullsized":"2024-09-21T06:34:55.732028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.164452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491696+0000","last_change":"2024-09-21T06:34:55.708931+0000","last_active":"2024-09-21T06:34:57.491696+0000","last_peered":"2024-09-21T06:34:57.491696+0000","last_clean":"2024-09-21T06:34:57.491696+0000","last_became_active":"2024-09-21T06:34:55.708602+0000","last_became_peered":"2024-09-21T06:34:55.708602+0000","last_unstale":"2024-09-21T06:34:57.491696+0000","last_undegraded":"2024-09-21T06:34:57.491696+0000","last_fullsized":"2024-09-21T06:34:57.491696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:53.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:54.709657+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:54.709345+0000","last_became_peered":"2024-09-21T06:34:54.709345+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.714451+0000","last_change":"2024-09-21T06:34:55.709527+0000","last_active":"2024-09-21T06:34:55.714451+0000","last_peered":"2024-09-21T06:34:55.714451+0000","last_clean":"2024-09-21T06:34:55.714451+0000","last_became_active":"2024-09-21T06:34:55.709199+0000","last_became_peered":"2024-09-21T06:34:55.709199+0000","last_unstale":"2024-09-21T06:34:55.714451+0000","last_undegraded":"2024-09-21T06:34:55.714451+0000","last_fullsized":"2024-09-21T06:34:55.714451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:07.792379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682816+0000","last_change":"2024-09-21T06:34:54.708914+0000","last_active":"2024-09-21T06:34:57.682816+0000","last_peered":"2024-09-21T06:34:57.682816+0000","last_clean":"2024-09-21T06:34:57.682816+0000","last_became_active":"2024-09-21T06:34:54.706923+0000","last_became_peered":"2024-09-21T06:34:54.706923+0000","last_unstale":"2024-09-21T06:34:57.682816+0000","last_undegraded":"2024-09-21T06:34:57.682816+0000","last_fullsized":"2024-09-21T06:34:57.682816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491887+0000","last_change":"2024-09-21T06:34:55.710127+0000","last_active":"2024-09-21T06:34:57.491887+0000","last_peered":"2024-09-21T06:34:57.491887+0000","last_clean":"2024-09-21T06:34:57.491887+0000","last_became_active":"2024-09-21T06:34:55.709577+0000","last_became_peered":"2024-09-21T06:34:55.709577+0000","last_unstale":"2024-09-21T06:34:57.491887+0000","last_undegraded":"2024-09-21T06:34:57.491887+0000","last_fullsized":"2024-09-21T06:34:57.491887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.912010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.702796+0000","last_change":"2024-09-21T06:34:54.711684+0000","last_active":"2024-09-21T06:34:55.702796+0000","last_peered":"2024-09-21T06:34:55.702796+0000","last_clean":"2024-09-21T06:34:55.702796+0000","last_became_active":"2024-09-21T06:34:54.711307+0000","last_became_peered":"2024-09-21T06:34:54.711307+0000","last_unstale":"2024-09-21T06:34:55.702796+0000","last_undegraded":"2024-09-21T06:34:55.702796+0000","last_fullsized":"2024-09-21T06:34:55.702796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:08.787586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490933+0000","last_change":"2024-09-21T06:34:55.718040+0000","last_active":"2024-09-21T06:34:57.490933+0000","last_peered":"2024-09-21T06:34:57.490933+0000","last_clean":"2024-09-21T06:34:57.490933+0000","last_became_active":"2024-09-21T06:34:55.717355+0000","last_became_peered":"2024-09-21T06:34:55.717355+0000","last_unstale":"2024-09-21T06:34:57.490933+0000","last_undegraded":"2024-09-21T06:34:57.490933+0000","last_fullsized":"2024-09-21T06:34:57.490933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897542+0000","last_change":"2024-09-21T06:34:54.709082+0000","last_active":"2024-09-21T06:34:56.897542+0000","last_peered":"2024-09-21T06:34:56.897542+0000","last_clean":"2024-09-21T06:34:56.897542+0000","last_became_active":"2024-09-21T06:34:54.708646+0000","last_became_peered":"2024-09-21T06:34:54.708646+0000","last_unstale":"2024-09-21T06:34:56.897542+0000","last_undegraded":"2024-09-21T06:34:56.897542+0000","last_fullsized":"2024-09-21T06:34:56.897542+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:59.547312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490987+0000","last_change":"2024-09-21T06:34:55.706722+0000","last_active":"2024-09-21T06:34:57.490987+0000","last_peered":"2024-09-21T06:34:57.490987+0000","last_clean":"2024-09-21T06:34:57.490987+0000","last_became_active":"2024-09-21T06:34:55.706428+0000","last_became_peered":"2024-09-21T06:34:55.706428+0000","last_unstale":"2024-09-21T06:34:57.490987+0000","last_undegraded":"2024-09-21T06:34:57.490987+0000","last_fullsized":"2024-09-21T06:34:57.490987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:17.159683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693179+0000","last_change":"2024-09-21T06:34:54.700087+0000","last_active":"2024-09-21T06:34:56.693179+0000","last_peered":"2024-09-21T06:34:56.693179+0000","last_clean":"2024-09-21T06:34:56.693179+0000","last_became_active":"2024-09-21T06:34:54.699715+0000","last_became_peered":"2024-09-21T06:34:54.699715+0000","last_unstale":"2024-09-21T06:34:56.693179+0000","last_undegraded":"2024-09-21T06:34:56.693179+0000","last_fullsized":"2024-09-21T06:34:56.693179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693783+0000","last_change":"2024-09-21T06:34:55.714419+0000","last_active":"2024-09-21T06:34:56.693783+0000","last_peered":"2024-09-21T06:34:56.693783+0000","last_clean":"2024-09-21T06:34:56.693783+0000","last_became_active":"2024-09-21T06:34:55.714040+0000","last_became_peered":"2024-09-21T06:34:55.714040+0000","last_unstale":"2024-09-21T06:34:56.693783+0000","last_undegraded":"2024-09-21T06:34:56.693783+0000","last_fullsized":"2024-09-21T06:34:56.693783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.757308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897468+0000","last_change":"2024-09-21T06:34:54.708697+0000","last_active":"2024-09-21T06:34:56.897468+0000","last_peered":"2024-09-21T06:34:56.897468+0000","last_clean":"2024-09-21T06:34:56.897468+0000","last_became_active":"2024-09-21T06:34:54.708338+0000","last_became_peered":"2024-09-21T06:34:54.708338+0000","last_unstale":"2024-09-21T06:34:56.897468+0000","last_undegraded":"2024-09-21T06:34:56.897468+0000","last_fullsized":"2024-09-21T06:34:56.897468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:29.524412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.733897+0000","last_change":"2024-09-21T06:34:55.732505+0000","last_active":"2024-09-21T06:34:55.733897+0000","last_peered":"2024-09-21T06:34:55.733897+0000","last_clean":"2024-09-21T06:34:55.733897+0000","last_became_active":"2024-09-21T06:34:55.732209+0000","last_became_peered":"2024-09-21T06:34:55.732209+0000","last_unstale":"2024-09-21T06:34:55.733897+0000","last_undegraded":"2024-09-21T06:34:55.733897+0000","last_fullsized":"2024-09-21T06:34:55.733897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:36.296181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.740620+0000","last_change":"2024-09-21T06:34:54.704500+0000","last_active":"2024-09-21T06:34:55.740620+0000","last_peered":"2024-09-21T06:34:55.740620+0000","last_clean":"2024-09-21T06:34:55.740620+0000","last_became_active":"2024-09-21T06:34:54.704165+0000","last_became_peered":"2024-09-21T06:34:54.704165+0000","last_unstale":"2024-09-21T06:34:55.740620+0000","last_undegraded":"2024-09-21T06:34:55.740620+0000","last_fullsized":"2024-09-21T06:34:55.740620+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.649075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690939+0000","last_change":"2024-09-21T06:34:55.705237+0000","last_active":"2024-09-21T06:34:56.690939+0000","last_peered":"2024-09-21T06:34:56.690939+0000","last_clean":"2024-09-21T06:34:56.690939+0000","last_became_active":"2024-09-21T06:34:55.704951+0000","last_became_peered":"2024-09-21T06:34:55.704951+0000","last_unstale":"2024-09-21T06:34:56.690939+0000","last_undegraded":"2024-09-21T06:34:56.690939+0000","last_fullsized":"2024-09-21T06:34:56.690939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491461+0000","last_change":"2024-09-21T06:34:54.709537+0000","last_active":"2024-09-21T06:34:57.491461+0000","last_peered":"2024-09-21T06:34:57.491461+0000","last_clean":"2024-09-21T06:34:57.491461+0000","last_became_active":"2024-09-21T06:34:54.706229+0000","last_became_peered":"2024-09-21T06:34:54.706229+0000","last_unstale":"2024-09-21T06:34:57.491461+0000","last_undegraded":"2024-09-21T06:34:57.491461+0000","last_fullsized":"2024-09-21T06:34:57.491461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:57.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691227+0000","last_change":"2024-09-21T06:34:55.699196+0000","last_active":"2024-09-21T06:34:56.691227+0000","last_peered":"2024-09-21T06:34:56.691227+0000","last_clean":"2024-09-21T06:34:56.691227+0000","last_became_active":"2024-09-21T06:34:55.698813+0000","last_became_peered":"2024-09-21T06:34:55.698813+0000","last_unstale":"2024-09-21T06:34:56.691227+0000","last_undegraded":"2024-09-21T06:34:56.691227+0000","last_fullsized":"2024-09-21T06:34:56.691227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:46.110588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693829+0000","last_change":"2024-09-21T06:34:54.701370+0000","last_active":"2024-09-21T06:34:56.693829+0000","last_peered":"2024-09-21T06:34:56.693829+0000","last_clean":"2024-09-21T06:34:56.693829+0000","last_became_active":"2024-09-21T06:34:54.700999+0000","last_became_peered":"2024-09-21T06:34:54.700999+0000","last_unstale":"2024-09-21T06:34:56.693829+0000","last_undegraded":"2024-09-21T06:34:56.693829+0000","last_fullsized":"2024-09-21T06:34:56.693829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731581+0000","last_change":"2024-09-21T06:34:55.728639+0000","last_active":"2024-09-21T06:34:55.731581+0000","last_peered":"2024-09-21T06:34:55.731581+0000","last_clean":"2024-09-21T06:34:55.731581+0000","last_became_active":"2024-09-21T06:34:55.728220+0000","last_became_peered":"2024-09-21T06:34:55.728220+0000","last_unstale":"2024-09-21T06:34:55.731581+0000","last_undegraded":"2024-09-21T06:34:55.731581+0000","last_fullsized":"2024-09-21T06:34:55.731581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.666244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490873+0000","last_change":"2024-09-21T06:34:54.708770+0000","last_active":"2024-09-21T06:34:57.490873+0000","last_peered":"2024-09-21T06:34:57.490873+0000","last_clean":"2024-09-21T06:34:57.490873+0000","last_became_active":"2024-09-21T06:34:54.708363+0000","last_became_peered":"2024-09-21T06:34:54.708363+0000","last_unstale":"2024-09-21T06:34:57.490873+0000","last_undegraded":"2024-09-21T06:34:57.490873+0000","last_fullsized":"2024-09-21T06:34:57.490873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:08.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.692465+0000","last_change":"2024-09-21T06:34:54.692465+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.692465+0000","last_undegraded":"2024-09-21T06:34:54.692465+0000","last_fullsized":"2024-09-21T06:34:54.692465+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.734554+0000","last_change":"2024-09-21T06:34:54.713801+0000","last_active":"2024-09-21T06:34:55.734554+0000","last_peered":"2024-09-21T06:34:55.734554+0000","last_clean":"2024-09-21T06:34:55.734554+0000","last_became_active":"2024-09-21T06:34:54.712759+0000","last_became_peered":"2024-09-21T06:34:54.712759+0000","last_unstale":"2024-09-21T06:34:55.734554+0000","last_undegraded":"2024-09-21T06:34:55.734554+0000","last_fullsized":"2024-09-21T06:34:55.734554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:47.339290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692225+0000","last_change":"2024-09-21T06:34:55.732582+0000","last_active":"2024-09-21T06:34:56.692225+0000","last_peered":"2024-09-21T06:34:56.692225+0000","last_clean":"2024-09-21T06:34:56.692225+0000","last_became_active":"2024-09-21T06:34:55.732328+0000","last_became_peered":"2024-09-21T06:34:55.732328+0000","last_unstale":"2024-09-21T06:34:56.692225+0000","last_undegraded":"2024-09-21T06:34:56.692225+0000","last_fullsized":"2024-09-21T06:34:56.692225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897574+0000","last_change":"2024-09-21T06:34:54.710621+0000","last_active":"2024-09-21T06:34:56.897574+0000","last_peered":"2024-09-21T06:34:56.897574+0000","last_clean":"2024-09-21T06:34:56.897574+0000","last_became_active":"2024-09-21T06:34:54.710305+0000","last_became_peered":"2024-09-21T06:34:54.710305+0000","last_unstale":"2024-09-21T06:34:56.897574+0000","last_undegraded":"2024-09-21T06:34:56.897574+0000","last_fullsized":"2024-09-21T06:34:56.897574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:04.681733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683086+0000","last_change":"2024-09-21T06:34:55.716269+0000","last_active":"2024-09-21T06:34:57.683086+0000","last_peered":"2024-09-21T06:34:57.683086+0000","last_clean":"2024-09-21T06:34:57.683086+0000","last_became_active":"2024-09-21T06:34:55.706408+0000","last_became_peered":"2024-09-21T06:34:55.706408+0000","last_unstale":"2024-09-21T06:34:57.683086+0000","last_undegraded":"2024-09-21T06:34:57.683086+0000","last_fullsized":"2024-09-21T06:34:57.683086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683117+0000","last_change":"2024-09-21T06:34:54.712959+0000","last_active":"2024-09-21T06:34:57.683117+0000","last_peered":"2024-09-21T06:34:57.683117+0000","last_clean":"2024-09-21T06:34:57.683117+0000","last_became_active":"2024-09-21T06:34:54.711866+0000","last_became_peered":"2024-09-21T06:34:54.711866+0000","last_unstale":"2024-09-21T06:34:57.683117+0000","last_undegraded":"2024-09-21T06:34:57.683117+0000","last_fullsized":"2024-09-21T06:34:57.683117+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731728+0000","last_change":"2024-09-21T06:34:55.728437+0000","last_active":"2024-09-21T06:34:55.731728+0000","last_peered":"2024-09-21T06:34:55.731728+0000","last_clean":"2024-09-21T06:34:55.731728+0000","last_became_active":"2024-09-21T06:34:55.728111+0000","last_became_peered":"2024-09-21T06:34:55.728111+0000","last_unstale":"2024-09-21T06:34:55.731728+0000","last_undegraded":"2024-09-21T06:34:55.731728+0000","last_fullsized":"2024-09-21T06:34:55.731728+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:21.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897340+0000","last_change":"2024-09-21T06:34:54.710224+0000","last_active":"2024-09-21T06:34:56.897340+0000","last_peered":"2024-09-21T06:34:56.897340+0000","last_clean":"2024-09-21T06:34:56.897340+0000","last_became_active":"2024-09-21T06:34:54.709896+0000","last_became_peered":"2024-09-21T06:34:54.709896+0000","last_unstale":"2024-09-21T06:34:56.897340+0000","last_undegraded":"2024-09-21T06:34:56.897340+0000","last_fullsized":"2024-09-21T06:34:56.897340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:16.511663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491563+0000","last_change":"2024-09-21T06:34:55.708497+0000","last_active":"2024-09-21T06:34:57.491563+0000","last_peered":"2024-09-21T06:34:57.491563+0000","last_clean":"2024-09-21T06:34:57.491563+0000","last_became_active":"2024-09-21T06:34:55.706136+0000","last_became_peered":"2024-09-21T06:34:55.706136+0000","last_unstale":"2024-09-21T06:34:57.491563+0000","last_undegraded":"2024-09-21T06:34:57.491563+0000","last_fullsized":"2024-09-21T06:34:57.491563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:06.652871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713997+0000","last_change":"2024-09-21T06:34:54.712196+0000","last_active":"2024-09-21T06:34:54.713997+0000","last_peered":"2024-09-21T06:34:54.713997+0000","last_clean":"2024-09-21T06:34:54.713997+0000","last_became_active":"2024-09-21T06:34:54.711507+0000","last_became_peered":"2024-09-21T06:34:54.711507+0000","last_unstale":"2024-09-21T06:34:54.713997+0000","last_undegraded":"2024-09-21T06:34:54.713997+0000","last_fullsized":"2024-09-21T06:34:54.713997+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:41.618052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.725352+0000","last_change":"2024-09-21T06:34:55.718899+0000","last_active":"2024-09-21T06:34:55.725352+0000","last_peered":"2024-09-21T06:34:55.725352+0000","last_clean":"2024-09-21T06:34:55.725352+0000","last_became_active":"2024-09-21T06:34:55.718610+0000","last_became_peered":"2024-09-21T06:34:55.718610+0000","last_unstale":"2024-09-21T06:34:55.725352+0000","last_undegraded":"2024-09-21T06:34:55.725352+0000","last_fullsized":"2024-09-21T06:34:55.725352+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:14.255062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707453+0000","last_change":"2024-09-21T06:34:54.704910+0000","last_active":"2024-09-21T06:34:54.707453+0000","last_peered":"2024-09-21T06:34:54.707453+0000","last_clean":"2024-09-21T06:34:54.707453+0000","last_became_active":"2024-09-21T06:34:54.703410+0000","last_became_peered":"2024-09-21T06:34:54.703410+0000","last_unstale":"2024-09-21T06:34:54.707453+0000","last_undegraded":"2024-09-21T06:34:54.707453+0000","last_fullsized":"2024-09-21T06:34:54.707453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:42.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731552+0000","last_change":"2024-09-21T06:34:55.726247+0000","last_active":"2024-09-21T06:34:55.731552+0000","last_peered":"2024-09-21T06:34:55.731552+0000","last_clean":"2024-09-21T06:34:55.731552+0000","last_became_active":"2024-09-21T06:34:55.725850+0000","last_became_peered":"2024-09-21T06:34:55.725850+0000","last_unstale":"2024-09-21T06:34:55.731552+0000","last_undegraded":"2024-09-21T06:34:55.731552+0000","last_fullsized":"2024-09-21T06:34:55.731552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:40.564061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897179+0000","last_change":"2024-09-21T06:34:54.712284+0000","last_active":"2024-09-21T06:34:56.897179+0000","last_peered":"2024-09-21T06:34:56.897179+0000","last_clean":"2024-09-21T06:34:56.897179+0000","last_became_active":"2024-09-21T06:34:54.711828+0000","last_became_peered":"2024-09-21T06:34:54.711828+0000","last_unstale":"2024-09-21T06:34:56.897179+0000","last_undegraded":"2024-09-21T06:34:56.897179+0000","last_fullsized":"2024-09-21T06:34:56.897179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:29.075613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491685+0000","last_change":"2024-09-21T06:34:55.713268+0000","last_active":"2024-09-21T06:34:57.491685+0000","last_peered":"2024-09-21T06:34:57.491685+0000","last_clean":"2024-09-21T06:34:57.491685+0000","last_became_active":"2024-09-21T06:34:55.712940+0000","last_became_peered":"2024-09-21T06:34:55.712940+0000","last_unstale":"2024-09-21T06:34:57.491685+0000","last_undegraded":"2024-09-21T06:34:57.491685+0000","last_fullsized":"2024-09-21T06:34:57.491685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:20.633620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692250+0000","last_change":"2024-09-21T06:34:54.706724+0000","last_active":"2024-09-21T06:34:56.692250+0000","last_peered":"2024-09-21T06:34:56.692250+0000","last_clean":"2024-09-21T06:34:56.692250+0000","last_became_active":"2024-09-21T06:34:54.706045+0000","last_became_peered":"2024-09-21T06:34:54.706045+0000","last_unstale":"2024-09-21T06:34:56.692250+0000","last_undegraded":"2024-09-21T06:34:56.692250+0000","last_fullsized":"2024-09-21T06:34:56.692250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.717407+0000","last_change":"2024-09-21T06:34:55.716497+0000","last_active":"2024-09-21T06:34:55.717407+0000","last_peered":"2024-09-21T06:34:55.717407+0000","last_clean":"2024-09-21T06:34:55.717407+0000","last_became_active":"2024-09-21T06:34:55.716182+0000","last_became_peered":"2024-09-21T06:34:55.716182+0000","last_unstale":"2024-09-21T06:34:55.717407+0000","last_undegraded":"2024-09-21T06:34:55.717407+0000","last_fullsized":"2024-09-21T06:34:55.717407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:13.835969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.704018+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.703663+0000","last_became_peered":"2024-09-21T06:34:54.703663+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:45.237205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721388+0000","last_change":"2024-09-21T06:34:55.712873+0000","last_active":"2024-09-21T06:34:55.721388+0000","last_peered":"2024-09-21T06:34:55.721388+0000","last_clean":"2024-09-21T06:34:55.721388+0000","last_became_active":"2024-09-21T06:34:55.711695+0000","last_became_peered":"2024-09-21T06:34:55.711695+0000","last_unstale":"2024-09-21T06:34:55.721388+0000","last_undegraded":"2024-09-21T06:34:55.721388+0000","last_fullsized":"2024-09-21T06:34:55.721388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:31.987545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713460+0000","last_change":"2024-09-21T06:34:54.709868+0000","last_active":"2024-09-21T06:34:54.713460+0000","last_peered":"2024-09-21T06:34:54.713460+0000","last_clean":"2024-09-21T06:34:54.713460+0000","last_became_active":"2024-09-21T06:34:54.708447+0000","last_became_peered":"2024-09-21T06:34:54.708447+0000","last_unstale":"2024-09-21T06:34:54.713460+0000","last_undegraded":"2024-09-21T06:34:54.713460+0000","last_fullsized":"2024-09-21T06:34:54.713460+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:13.936837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693744+0000","last_change":"2024-09-21T06:34:55.729064+0000","last_active":"2024-09-21T06:34:56.693744+0000","last_peered":"2024-09-21T06:34:56.693744+0000","last_clean":"2024-09-21T06:34:56.693744+0000","last_became_active":"2024-09-21T06:34:55.728795+0000","last_became_peered":"2024-09-21T06:34:55.728795+0000","last_unstale":"2024-09-21T06:34:56.693744+0000","last_undegraded":"2024-09-21T06:34:56.693744+0000","last_fullsized":"2024-09-21T06:34:56.693744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:38.265711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.703782+0000","last_change":"2024-09-21T06:34:54.702424+0000","last_active":"2024-09-21T06:34:55.703782+0000","last_peered":"2024-09-21T06:34:55.703782+0000","last_clean":"2024-09-21T06:34:55.703782+0000","last_became_active":"2024-09-21T06:34:54.702065+0000","last_became_peered":"2024-09-21T06:34:54.702065+0000","last_unstale":"2024-09-21T06:34:55.703782+0000","last_undegraded":"2024-09-21T06:34:55.703782+0000","last_fullsized":"2024-09-21T06:34:55.703782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:51.535195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683128+0000","last_change":"2024-09-21T06:34:55.718743+0000","last_active":"2024-09-21T06:34:57.683128+0000","last_peered":"2024-09-21T06:34:57.683128+0000","last_clean":"2024-09-21T06:34:57.683128+0000","last_became_active":"2024-09-21T06:34:55.703205+0000","last_became_peered":"2024-09-21T06:34:55.703205+0000","last_unstale":"2024-09-21T06:34:57.683128+0000","last_undegraded":"2024-09-21T06:34:57.683128+0000","last_fullsized":"2024-09-21T06:34:57.683128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491730+0000","last_change":"2024-09-21T06:34:54.711548+0000","last_active":"2024-09-21T06:34:57.491730+0000","last_peered":"2024-09-21T06:34:57.491730+0000","last_clean":"2024-09-21T06:34:57.491730+0000","last_became_active":"2024-09-21T06:34:54.711141+0000","last_became_peered":"2024-09-21T06:34:54.711141+0000","last_unstale":"2024-09-21T06:34:57.491730+0000","last_undegraded":"2024-09-21T06:34:57.491730+0000","last_fullsized":"2024-09-21T06:34:57.491730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:27.210366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491185+0000","last_change":"2024-09-21T06:34:55.712582+0000","last_active":"2024-09-21T06:34:57.491185+0000","last_peered":"2024-09-21T06:34:57.491185+0000","last_clean":"2024-09-21T06:34:57.491185+0000","last_became_active":"2024-09-21T06:34:55.710833+0000","last_became_peered":"2024-09-21T06:34:55.710833+0000","last_unstale":"2024-09-21T06:34:57.491185+0000","last_undegraded":"2024-09-21T06:34:57.491185+0000","last_fullsized":"2024-09-21T06:34:57.491185+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.706570+0000","last_change":"2024-09-21T06:34:54.713168+0000","last_active":"2024-09-21T06:34:55.706570+0000","last_peered":"2024-09-21T06:34:55.706570+0000","last_clean":"2024-09-21T06:34:55.706570+0000","last_became_active":"2024-09-21T06:34:54.712837+0000","last_became_peered":"2024-09-21T06:34:54.712837+0000","last_unstale":"2024-09-21T06:34:55.706570+0000","last_undegraded":"2024-09-21T06:34:55.706570+0000","last_fullsized":"2024-09-21T06:34:55.706570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:29.731794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491694+0000","last_change":"2024-09-21T06:34:55.711645+0000","last_active":"2024-09-21T06:34:57.491694+0000","last_peered":"2024-09-21T06:34:57.491694+0000","last_clean":"2024-09-21T06:34:57.491694+0000","last_became_active":"2024-09-21T06:34:55.711312+0000","last_became_peered":"2024-09-21T06:34:55.711312+0000","last_unstale":"2024-09-21T06:34:57.491694+0000","last_undegraded":"2024-09-21T06:34:57.491694+0000","last_fullsized":"2024-09-21T06:34:57.491694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.705515+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.705153+0000","last_became_peered":"2024-09-21T06:34:54.705153+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:38.546287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897683+0000","last_change":"2024-09-21T06:34:55.729077+0000","last_active":"2024-09-21T06:34:56.897683+0000","last_peered":"2024-09-21T06:34:56.897683+0000","last_clean":"2024-09-21T06:34:56.897683+0000","last_became_active":"2024-09-21T06:34:55.728776+0000","last_became_peered":"2024-09-21T06:34:55.728776+0000","last_unstale":"2024-09-21T06:34:56.897683+0000","last_undegraded":"2024-09-21T06:34:56.897683+0000","last_fullsized":"2024-09-21T06:34:56.897683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683192+0000","last_change":"2024-09-21T06:34:54.703341+0000","last_active":"2024-09-21T06:34:57.683192+0000","last_peered":"2024-09-21T06:34:57.683192+0000","last_clean":"2024-09-21T06:34:57.683192+0000","last_became_active":"2024-09-21T06:34:54.703002+0000","last_became_peered":"2024-09-21T06:34:54.703002+0000","last_unstale":"2024-09-21T06:34:57.683192+0000","last_undegraded":"2024-09-21T06:34:57.683192+0000","last_fullsized":"2024-09-21T06:34:57.683192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691146+0000","last_change":"2024-09-21T06:34:55.737366+0000","last_active":"2024-09-21T06:34:56.691146+0000","last_peered":"2024-09-21T06:34:56.691146+0000","last_clean":"2024-09-21T06:34:56.691146+0000","last_became_active":"2024-09-21T06:34:55.734444+0000","last_became_peered":"2024-09-21T06:34:55.734444+0000","last_unstale":"2024-09-21T06:34:56.691146+0000","last_undegraded":"2024-09-21T06:34:56.691146+0000","last_fullsized":"2024-09-21T06:34:56.691146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:47.619903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707339+0000","last_change":"2024-09-21T06:34:54.702747+0000","last_active":"2024-09-21T06:34:54.707339+0000","last_peered":"2024-09-21T06:34:54.707339+0000","last_clean":"2024-09-21T06:34:54.707339+0000","last_became_active":"2024-09-21T06:34:54.702221+0000","last_became_peered":"2024-09-21T06:34:54.702221+0000","last_unstale":"2024-09-21T06:34:54.707339+0000","last_undegraded":"2024-09-21T06:34:54.707339+0000","last_fullsized":"2024-09-21T06:34:54.707339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:50.889270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897188+0000","last_change":"2024-09-21T06:34:55.724899+0000","last_active":"2024-09-21T06:34:56.897188+0000","last_peered":"2024-09-21T06:34:56.897188+0000","last_clean":"2024-09-21T06:34:56.897188+0000","last_became_active":"2024-09-21T06:34:55.724167+0000","last_became_peered":"2024-09-21T06:34:55.724167+0000","last_unstale":"2024-09-21T06:34:56.897188+0000","last_undegraded":"2024-09-21T06:34:56.897188+0000","last_fullsized":"2024-09-21T06:34:56.897188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491794+0000","last_change":"2024-09-21T06:34:54.707367+0000","last_active":"2024-09-21T06:34:57.491794+0000","last_peered":"2024-09-21T06:34:57.491794+0000","last_clean":"2024-09-21T06:34:57.491794+0000","last_became_active":"2024-09-21T06:34:54.705863+0000","last_became_peered":"2024-09-21T06:34:54.705863+0000","last_unstale":"2024-09-21T06:34:57.491794+0000","last_undegraded":"2024-09-21T06:34:57.491794+0000","last_fullsized":"2024-09-21T06:34:57.491794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.687054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682836+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:34:57.682836+0000","last_peered":"2024-09-21T06:34:57.682836+0000","last_clean":"2024-09-21T06:34:57.682836+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:34:57.682836+0000","last_undegraded":"2024-09-21T06:34:57.682836+0000","last_fullsized":"2024-09-21T06:34:57.682836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1277952,"data_stored":589850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509529,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":595924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1282048,"data_stored":590810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":1179658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":596472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1839104,"data_stored":1165732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:00.189 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:35:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 -- 172.21.15.37:0/149725937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800ccfe0 msgr2=0x7f7a8006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/149725937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800ccfe0 0x7f7a8006e570 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7a80074f80 tx=0x7f7a7802aa30 comp rx=0 tx=0).stop 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 -- 172.21.15.37:0/149725937 shutdown_connections 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/149725937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800ccfe0 0x7f7a8006e570 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/149725937 >> v2:172.21.15.150:3300/0 conn(0x7f7a8015dfe0 0x7f7a80160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 -- 172.21.15.37:0/149725937 >> 172.21.15.37:0/149725937 conn(0x7f7a8006f940 msgr2=0x7f7a800b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 -- 172.21.15.37:0/149725937 shutdown_connections 2024-09-21T06:35:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.451+0000 7f7a7fe00640 1 -- 172.21.15.37:0/149725937 wait complete. 2024-09-21T06:35:00.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.453+0000 7f7a7fe00640 1 Processor -- start 2024-09-21T06:35:00.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.453+0000 7f7a7fe00640 1 -- start start 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 0x7f7a800ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a8015dfe0 0x7f7a80074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800b4b70 con 0x7f7a80074970 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800b49f0 con 0x7f7a8006e160 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7fe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800b4cf0 con 0x7f7a8015dfe0 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38278/0 (socket says 172.21.15.37:38278) 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7f400640 1 -- 172.21.15.37:0/785328697 learned_addr learned my addr 172.21.15.37:0/785328697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a7ea00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 0x7f7a800ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.455+0000 7f7a76000640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a8015dfe0 0x7f7a80074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 -- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a8015dfe0 msgr2=0x7f7a80074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a8015dfe0 0x7f7a80074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 -- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 msgr2=0x7f7a800ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 0x7f7a800ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 -- 172.21.15.37:0/785328697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800aef00 con 0x7f7a80074970 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7ea00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 0x7f7a800ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7f400640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7a78005fd0 tx=0x7f7a78003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78033050 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a78004540 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78038450 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a80072c00 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.456+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a800731a0 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.457+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a800b5210 con 0x7f7a80074970 2024-09-21T06:35:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.459+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a78036070 con 0x7f7a80074970 2024-09-21T06:35:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.460+0000 7f7a77400640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 0x7f7a5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.460+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7a780bc240 con 0x7f7a80074970 2024-09-21T06:35:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.460+0000 7f7a7ea00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 0x7f7a5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.461+0000 7f7a7ea00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 0x7f7a5c07b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009b80 tx=0x7f7a70005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.464+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a800aef00 con 0x7f7a80074970 2024-09-21T06:35:00.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.656+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a80145c60 con 0x7f7a5c078b90 2024-09-21T06:35:00.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.661+0000 7f7a77400640 1 -- 172.21.15.37:0/785328697 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f7a80145c60 con 0x7f7a5c078b90 2024-09-21T06:35:00.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:00.666 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 msgr2=0x7f7a5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 0x7f7a5c07b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009b80 tx=0x7f7a70005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 msgr2=0x7f7a8015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7a78005fd0 tx=0x7f7a78003f90 comp rx=0 tx=0).stop 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 shutdown_connections 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a5c078b90 0x7f7a5c07b0a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a80074970 0x7f7a8015b500 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a8015dfe0 0x7f7a80074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 --2- 172.21.15.37:0/785328697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a8006e160 0x7f7a800ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 >> 172.21.15.37:0/785328697 conn(0x7f7a8006f940 msgr2=0x7f7a800d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 shutdown_connections 2024-09-21T06:35:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.665+0000 7f7a7fe00640 1 -- 172.21.15.37:0/785328697 wait complete. 2024-09-21T06:35:00.688 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T06:34:59.205143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260700,"kb_used_data":11160,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714020,"statfs":{"total":773094113280,"available":772827156480,"internally_reserved":0,"allocated":11427840,"data_stored":5911950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.525742"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491654+0000","last_change":"2024-09-21T06:34:55.702355+0000","last_active":"2024-09-21T06:34:57.491654+0000","last_peered":"2024-09-21T06:34:57.491654+0000","last_clean":"2024-09-21T06:34:57.491654+0000","last_became_active":"2024-09-21T06:34:55.701180+0000","last_became_peered":"2024-09-21T06:34:55.701180+0000","last_unstale":"2024-09-21T06:34:57.491654+0000","last_undegraded":"2024-09-21T06:34:57.491654+0000","last_fullsized":"2024-09-21T06:34:57.491654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:40.085758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690863+0000","last_change":"2024-09-21T06:34:55.734713+0000","last_active":"2024-09-21T06:34:56.690863+0000","last_peered":"2024-09-21T06:34:56.690863+0000","last_clean":"2024-09-21T06:34:56.690863+0000","last_became_active":"2024-09-21T06:34:55.734367+0000","last_became_peered":"2024-09-21T06:34:55.734367+0000","last_unstale":"2024-09-21T06:34:56.690863+0000","last_undegraded":"2024-09-21T06:34:56.690863+0000","last_fullsized":"2024-09-21T06:34:56.690863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.630265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689595+0000","last_change":"2024-09-21T06:34:54.689595+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689595+0000","last_undegraded":"2024-09-21T06:34:54.689595+0000","last_fullsized":"2024-09-21T06:34:54.689595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491281+0000","last_change":"2024-09-21T06:34:55.713592+0000","last_active":"2024-09-21T06:34:57.491281+0000","last_peered":"2024-09-21T06:34:57.491281+0000","last_clean":"2024-09-21T06:34:57.491281+0000","last_became_active":"2024-09-21T06:34:55.711240+0000","last_became_peered":"2024-09-21T06:34:55.711240+0000","last_unstale":"2024-09-21T06:34:57.491281+0000","last_undegraded":"2024-09-21T06:34:57.491281+0000","last_fullsized":"2024-09-21T06:34:57.491281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.722733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491268+0000","last_change":"2024-09-21T06:34:55.712441+0000","last_active":"2024-09-21T06:34:57.491268+0000","last_peered":"2024-09-21T06:34:57.491268+0000","last_clean":"2024-09-21T06:34:57.491268+0000","last_became_active":"2024-09-21T06:34:55.710543+0000","last_became_peered":"2024-09-21T06:34:55.710543+0000","last_unstale":"2024-09-21T06:34:57.491268+0000","last_undegraded":"2024-09-21T06:34:57.491268+0000","last_fullsized":"2024-09-21T06:34:57.491268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:02.131613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897116+0000","last_change":"2024-09-21T06:34:55.727939+0000","last_active":"2024-09-21T06:34:56.897116+0000","last_peered":"2024-09-21T06:34:56.897116+0000","last_clean":"2024-09-21T06:34:56.897116+0000","last_became_active":"2024-09-21T06:34:55.727032+0000","last_became_peered":"2024-09-21T06:34:55.727032+0000","last_unstale":"2024-09-21T06:34:56.897116+0000","last_undegraded":"2024-09-21T06:34:56.897116+0000","last_fullsized":"2024-09-21T06:34:56.897116+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.403443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.896947+0000","last_change":"2024-09-21T06:34:55.723696+0000","last_active":"2024-09-21T06:34:56.896947+0000","last_peered":"2024-09-21T06:34:56.896947+0000","last_clean":"2024-09-21T06:34:56.896947+0000","last_became_active":"2024-09-21T06:34:55.723259+0000","last_became_peered":"2024-09-21T06:34:55.723259+0000","last_unstale":"2024-09-21T06:34:56.896947+0000","last_undegraded":"2024-09-21T06:34:56.896947+0000","last_fullsized":"2024-09-21T06:34:56.896947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:07.748611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691935+0000","last_change":"2024-09-21T06:34:55.723651+0000","last_active":"2024-09-21T06:34:56.691935+0000","last_peered":"2024-09-21T06:34:56.691935+0000","last_clean":"2024-09-21T06:34:56.691935+0000","last_became_active":"2024-09-21T06:34:55.723288+0000","last_became_peered":"2024-09-21T06:34:55.723288+0000","last_unstale":"2024-09-21T06:34:56.691935+0000","last_undegraded":"2024-09-21T06:34:56.691935+0000","last_fullsized":"2024-09-21T06:34:56.691935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:30.694885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691167+0000","last_change":"2024-09-21T06:34:55.704172+0000","last_active":"2024-09-21T06:34:56.691167+0000","last_peered":"2024-09-21T06:34:56.691167+0000","last_clean":"2024-09-21T06:34:56.691167+0000","last_became_active":"2024-09-21T06:34:55.703829+0000","last_became_peered":"2024-09-21T06:34:55.703829+0000","last_unstale":"2024-09-21T06:34:56.691167+0000","last_undegraded":"2024-09-21T06:34:56.691167+0000","last_fullsized":"2024-09-21T06:34:56.691167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:10.923582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693214+0000","last_change":"2024-09-21T06:34:55.714365+0000","last_active":"2024-09-21T06:34:56.693214+0000","last_peered":"2024-09-21T06:34:56.693214+0000","last_clean":"2024-09-21T06:34:56.693214+0000","last_became_active":"2024-09-21T06:34:55.711411+0000","last_became_peered":"2024-09-21T06:34:55.711411+0000","last_unstale":"2024-09-21T06:34:56.693214+0000","last_undegraded":"2024-09-21T06:34:56.693214+0000","last_fullsized":"2024-09-21T06:34:56.693214+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:55.929081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721189+0000","last_change":"2024-09-21T06:34:55.712794+0000","last_active":"2024-09-21T06:34:55.721189+0000","last_peered":"2024-09-21T06:34:55.721189+0000","last_clean":"2024-09-21T06:34:55.721189+0000","last_became_active":"2024-09-21T06:34:55.712487+0000","last_became_peered":"2024-09-21T06:34:55.712487+0000","last_unstale":"2024-09-21T06:34:55.721189+0000","last_undegraded":"2024-09-21T06:34:55.721189+0000","last_fullsized":"2024-09-21T06:34:55.721189+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:48.749080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897088+0000","last_change":"2024-09-21T06:34:55.720101+0000","last_active":"2024-09-21T06:34:56.897088+0000","last_peered":"2024-09-21T06:34:56.897088+0000","last_clean":"2024-09-21T06:34:56.897088+0000","last_became_active":"2024-09-21T06:34:55.719743+0000","last_became_peered":"2024-09-21T06:34:55.719743+0000","last_unstale":"2024-09-21T06:34:56.897088+0000","last_undegraded":"2024-09-21T06:34:56.897088+0000","last_fullsized":"2024-09-21T06:34:56.897088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:52.305232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491183+0000","last_change":"2024-09-21T06:34:55.712044+0000","last_active":"2024-09-21T06:34:57.491183+0000","last_peered":"2024-09-21T06:34:57.491183+0000","last_clean":"2024-09-21T06:34:57.491183+0000","last_became_active":"2024-09-21T06:34:55.710142+0000","last_became_peered":"2024-09-21T06:34:55.710142+0000","last_unstale":"2024-09-21T06:34:57.491183+0000","last_undegraded":"2024-09-21T06:34:57.491183+0000","last_fullsized":"2024-09-21T06:34:57.491183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:42.402252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897146+0000","last_change":"2024-09-21T06:34:55.722424+0000","last_active":"2024-09-21T06:34:56.897146+0000","last_peered":"2024-09-21T06:34:56.897146+0000","last_clean":"2024-09-21T06:34:56.897146+0000","last_became_active":"2024-09-21T06:34:55.722091+0000","last_became_peered":"2024-09-21T06:34:55.722091+0000","last_unstale":"2024-09-21T06:34:56.897146+0000","last_undegraded":"2024-09-21T06:34:56.897146+0000","last_fullsized":"2024-09-21T06:34:56.897146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:23.664483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690751+0000","last_change":"2024-09-21T06:34:55.727747+0000","last_active":"2024-09-21T06:34:56.690751+0000","last_peered":"2024-09-21T06:34:56.690751+0000","last_clean":"2024-09-21T06:34:56.690751+0000","last_became_active":"2024-09-21T06:34:55.727447+0000","last_became_peered":"2024-09-21T06:34:55.727447+0000","last_unstale":"2024-09-21T06:34:56.690751+0000","last_undegraded":"2024-09-21T06:34:56.690751+0000","last_fullsized":"2024-09-21T06:34:56.690751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.086183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683012+0000","last_change":"2024-09-21T06:34:55.710554+0000","last_active":"2024-09-21T06:34:57.683012+0000","last_peered":"2024-09-21T06:34:57.683012+0000","last_clean":"2024-09-21T06:34:57.683012+0000","last_became_active":"2024-09-21T06:34:55.710216+0000","last_became_peered":"2024-09-21T06:34:55.710216+0000","last_unstale":"2024-09-21T06:34:57.683012+0000","last_undegraded":"2024-09-21T06:34:57.683012+0000","last_fullsized":"2024-09-21T06:34:57.683012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.693106+0000","last_change":"2024-09-21T06:34:54.693106+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.693106+0000","last_undegraded":"2024-09-21T06:34:54.693106+0000","last_fullsized":"2024-09-21T06:34:54.693106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690915+0000","last_change":"2024-09-21T06:34:55.702753+0000","last_active":"2024-09-21T06:34:56.690915+0000","last_peered":"2024-09-21T06:34:56.690915+0000","last_clean":"2024-09-21T06:34:56.690915+0000","last_became_active":"2024-09-21T06:34:55.702431+0000","last_became_peered":"2024-09-21T06:34:55.702431+0000","last_unstale":"2024-09-21T06:34:56.690915+0000","last_undegraded":"2024-09-21T06:34:56.690915+0000","last_fullsized":"2024-09-21T06:34:56.690915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:07.262224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491255+0000","last_change":"2024-09-21T06:34:55.731627+0000","last_active":"2024-09-21T06:34:57.491255+0000","last_peered":"2024-09-21T06:34:57.491255+0000","last_clean":"2024-09-21T06:34:57.491255+0000","last_became_active":"2024-09-21T06:34:55.731309+0000","last_became_peered":"2024-09-21T06:34:55.731309+0000","last_unstale":"2024-09-21T06:34:57.491255+0000","last_undegraded":"2024-09-21T06:34:57.491255+0000","last_fullsized":"2024-09-21T06:34:57.491255+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:24.150794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:55.710593+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:55.710307+0000","last_became_peered":"2024-09-21T06:34:55.710307+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:54.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689114+0000","last_change":"2024-09-21T06:34:54.689114+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689114+0000","last_undegraded":"2024-09-21T06:34:54.689114+0000","last_fullsized":"2024-09-21T06:34:54.689114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731832+0000","last_change":"2024-09-21T06:34:55.728815+0000","last_active":"2024-09-21T06:34:55.731832+0000","last_peered":"2024-09-21T06:34:55.731832+0000","last_clean":"2024-09-21T06:34:55.731832+0000","last_became_active":"2024-09-21T06:34:55.728227+0000","last_became_peered":"2024-09-21T06:34:55.728227+0000","last_unstale":"2024-09-21T06:34:55.731832+0000","last_undegraded":"2024-09-21T06:34:55.731832+0000","last_fullsized":"2024-09-21T06:34:55.731832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:32.261965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692011+0000","last_change":"2024-09-21T06:34:55.731414+0000","last_active":"2024-09-21T06:34:56.692011+0000","last_peered":"2024-09-21T06:34:56.692011+0000","last_clean":"2024-09-21T06:34:56.692011+0000","last_became_active":"2024-09-21T06:34:55.731103+0000","last_became_peered":"2024-09-21T06:34:55.731103+0000","last_unstale":"2024-09-21T06:34:56.692011+0000","last_undegraded":"2024-09-21T06:34:56.692011+0000","last_fullsized":"2024-09-21T06:34:56.692011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691979+0000","last_change":"2024-09-21T06:34:55.715389+0000","last_active":"2024-09-21T06:34:56.691979+0000","last_peered":"2024-09-21T06:34:56.691979+0000","last_clean":"2024-09-21T06:34:56.691979+0000","last_became_active":"2024-09-21T06:34:55.715071+0000","last_became_peered":"2024-09-21T06:34:55.715071+0000","last_unstale":"2024-09-21T06:34:56.691979+0000","last_undegraded":"2024-09-21T06:34:56.691979+0000","last_fullsized":"2024-09-21T06:34:56.691979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682949+0000","last_change":"2024-09-21T06:34:55.717327+0000","last_active":"2024-09-21T06:34:57.682949+0000","last_peered":"2024-09-21T06:34:57.682949+0000","last_clean":"2024-09-21T06:34:57.682949+0000","last_became_active":"2024-09-21T06:34:55.716998+0000","last_became_peered":"2024-09-21T06:34:55.716998+0000","last_unstale":"2024-09-21T06:34:57.682949+0000","last_undegraded":"2024-09-21T06:34:57.682949+0000","last_fullsized":"2024-09-21T06:34:57.682949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683071+0000","last_change":"2024-09-21T06:34:55.717181+0000","last_active":"2024-09-21T06:34:57.683071+0000","last_peered":"2024-09-21T06:34:57.683071+0000","last_clean":"2024-09-21T06:34:57.683071+0000","last_became_active":"2024-09-21T06:34:55.716922+0000","last_became_peered":"2024-09-21T06:34:55.716922+0000","last_unstale":"2024-09-21T06:34:57.683071+0000","last_undegraded":"2024-09-21T06:34:57.683071+0000","last_fullsized":"2024-09-21T06:34:57.683071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683004+0000","last_change":"2024-09-21T06:34:55.715886+0000","last_active":"2024-09-21T06:34:57.683004+0000","last_peered":"2024-09-21T06:34:57.683004+0000","last_clean":"2024-09-21T06:34:57.683004+0000","last_became_active":"2024-09-21T06:34:55.715533+0000","last_became_peered":"2024-09-21T06:34:55.715533+0000","last_unstale":"2024-09-21T06:34:57.683004+0000","last_undegraded":"2024-09-21T06:34:57.683004+0000","last_fullsized":"2024-09-21T06:34:57.683004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897335+0000","last_change":"2024-09-21T06:34:55.730908+0000","last_active":"2024-09-21T06:34:56.897335+0000","last_peered":"2024-09-21T06:34:56.897335+0000","last_clean":"2024-09-21T06:34:56.897335+0000","last_became_active":"2024-09-21T06:34:55.730593+0000","last_became_peered":"2024-09-21T06:34:55.730593+0000","last_unstale":"2024-09-21T06:34:56.897335+0000","last_undegraded":"2024-09-21T06:34:56.897335+0000","last_fullsized":"2024-09-21T06:34:56.897335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692651+0000","last_change":"2024-09-21T06:34:55.728164+0000","last_active":"2024-09-21T06:34:56.692651+0000","last_peered":"2024-09-21T06:34:56.692651+0000","last_clean":"2024-09-21T06:34:56.692651+0000","last_became_active":"2024-09-21T06:34:55.727836+0000","last_became_peered":"2024-09-21T06:34:55.727836+0000","last_unstale":"2024-09-21T06:34:56.692651+0000","last_undegraded":"2024-09-21T06:34:56.692651+0000","last_fullsized":"2024-09-21T06:34:56.692651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.918656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721753+0000","last_change":"2024-09-21T06:34:55.714924+0000","last_active":"2024-09-21T06:34:55.721753+0000","last_peered":"2024-09-21T06:34:55.721753+0000","last_clean":"2024-09-21T06:34:55.721753+0000","last_became_active":"2024-09-21T06:34:55.713658+0000","last_became_peered":"2024-09-21T06:34:55.713658+0000","last_unstale":"2024-09-21T06:34:55.721753+0000","last_undegraded":"2024-09-21T06:34:55.721753+0000","last_fullsized":"2024-09-21T06:34:55.721753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:07.310461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897057+0000","last_change":"2024-09-21T06:34:55.718692+0000","last_active":"2024-09-21T06:34:56.897057+0000","last_peered":"2024-09-21T06:34:56.897057+0000","last_clean":"2024-09-21T06:34:56.897057+0000","last_became_active":"2024-09-21T06:34:55.718323+0000","last_became_peered":"2024-09-21T06:34:55.718323+0000","last_unstale":"2024-09-21T06:34:56.897057+0000","last_undegraded":"2024-09-21T06:34:56.897057+0000","last_fullsized":"2024-09-21T06:34:56.897057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:21.216504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.715947+0000","last_change":"2024-09-21T06:34:55.711836+0000","last_active":"2024-09-21T06:34:55.715947+0000","last_peered":"2024-09-21T06:34:55.715947+0000","last_clean":"2024-09-21T06:34:55.715947+0000","last_became_active":"2024-09-21T06:34:55.711556+0000","last_became_peered":"2024-09-21T06:34:55.711556+0000","last_unstale":"2024-09-21T06:34:55.715947+0000","last_undegraded":"2024-09-21T06:34:55.715947+0000","last_fullsized":"2024-09-21T06:34:55.715947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:24.575657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.690579+0000","last_change":"2024-09-21T06:34:54.690579+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.690579+0000","last_undegraded":"2024-09-21T06:34:54.690579+0000","last_fullsized":"2024-09-21T06:34:54.690579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.730527+0000","last_change":"2024-09-21T06:34:54.714493+0000","last_active":"2024-09-21T06:34:55.730527+0000","last_peered":"2024-09-21T06:34:55.730527+0000","last_clean":"2024-09-21T06:34:55.730527+0000","last_became_active":"2024-09-21T06:34:54.714069+0000","last_became_peered":"2024-09-21T06:34:54.714069+0000","last_unstale":"2024-09-21T06:34:55.730527+0000","last_undegraded":"2024-09-21T06:34:55.730527+0000","last_fullsized":"2024-09-21T06:34:55.730527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:49.070679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692607+0000","last_change":"2024-09-21T06:34:55.731634+0000","last_active":"2024-09-21T06:34:56.692607+0000","last_peered":"2024-09-21T06:34:56.692607+0000","last_clean":"2024-09-21T06:34:56.692607+0000","last_became_active":"2024-09-21T06:34:55.731317+0000","last_became_peered":"2024-09-21T06:34:55.731317+0000","last_unstale":"2024-09-21T06:34:56.692607+0000","last_undegraded":"2024-09-21T06:34:56.692607+0000","last_fullsized":"2024-09-21T06:34:56.692607+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.714077+0000","last_change":"2024-09-21T06:34:54.710899+0000","last_active":"2024-09-21T06:34:54.714077+0000","last_peered":"2024-09-21T06:34:54.714077+0000","last_clean":"2024-09-21T06:34:54.714077+0000","last_became_active":"2024-09-21T06:34:54.710535+0000","last_became_peered":"2024-09-21T06:34:54.710535+0000","last_unstale":"2024-09-21T06:34:54.714077+0000","last_undegraded":"2024-09-21T06:34:54.714077+0000","last_fullsized":"2024-09-21T06:34:54.714077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:53.792219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691077+0000","last_change":"2024-09-21T06:34:55.707231+0000","last_active":"2024-09-21T06:34:56.691077+0000","last_peered":"2024-09-21T06:34:56.691077+0000","last_clean":"2024-09-21T06:34:56.691077+0000","last_became_active":"2024-09-21T06:34:55.706910+0000","last_became_peered":"2024-09-21T06:34:55.706910+0000","last_unstale":"2024-09-21T06:34:56.691077+0000","last_undegraded":"2024-09-21T06:34:56.691077+0000","last_fullsized":"2024-09-21T06:34:56.691077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:31.422885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691219+0000","last_change":"2024-09-21T06:34:54.704122+0000","last_active":"2024-09-21T06:34:56.691219+0000","last_peered":"2024-09-21T06:34:56.691219+0000","last_clean":"2024-09-21T06:34:56.691219+0000","last_became_active":"2024-09-21T06:34:54.703803+0000","last_became_peered":"2024-09-21T06:34:54.703803+0000","last_unstale":"2024-09-21T06:34:56.691219+0000","last_undegraded":"2024-09-21T06:34:56.691219+0000","last_fullsized":"2024-09-21T06:34:56.691219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:49.261246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.737546+0000","last_change":"2024-09-21T06:34:55.731772+0000","last_active":"2024-09-21T06:34:55.737546+0000","last_peered":"2024-09-21T06:34:55.737546+0000","last_clean":"2024-09-21T06:34:55.737546+0000","last_became_active":"2024-09-21T06:34:55.731464+0000","last_became_peered":"2024-09-21T06:34:55.731464+0000","last_unstale":"2024-09-21T06:34:55.737546+0000","last_undegraded":"2024-09-21T06:34:55.737546+0000","last_fullsized":"2024-09-21T06:34:55.737546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:41.639926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692910+0000","last_change":"2024-09-21T06:34:54.701225+0000","last_active":"2024-09-21T06:34:56.692910+0000","last_peered":"2024-09-21T06:34:56.692910+0000","last_clean":"2024-09-21T06:34:56.692910+0000","last_became_active":"2024-09-21T06:34:54.700905+0000","last_became_peered":"2024-09-21T06:34:54.700905+0000","last_unstale":"2024-09-21T06:34:56.692910+0000","last_undegraded":"2024-09-21T06:34:56.692910+0000","last_fullsized":"2024-09-21T06:34:56.692910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491690+0000","last_change":"2024-09-21T06:34:55.698335+0000","last_active":"2024-09-21T06:34:57.491690+0000","last_peered":"2024-09-21T06:34:57.491690+0000","last_clean":"2024-09-21T06:34:57.491690+0000","last_became_active":"2024-09-21T06:34:55.697982+0000","last_became_peered":"2024-09-21T06:34:55.697982+0000","last_unstale":"2024-09-21T06:34:57.491690+0000","last_undegraded":"2024-09-21T06:34:57.491690+0000","last_fullsized":"2024-09-21T06:34:57.491690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:33.941883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.732028+0000","last_change":"2024-09-21T06:34:54.714529+0000","last_active":"2024-09-21T06:34:55.732028+0000","last_peered":"2024-09-21T06:34:55.732028+0000","last_clean":"2024-09-21T06:34:55.732028+0000","last_became_active":"2024-09-21T06:34:54.714159+0000","last_became_peered":"2024-09-21T06:34:54.714159+0000","last_unstale":"2024-09-21T06:34:55.732028+0000","last_undegraded":"2024-09-21T06:34:55.732028+0000","last_fullsized":"2024-09-21T06:34:55.732028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.164452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491696+0000","last_change":"2024-09-21T06:34:55.708931+0000","last_active":"2024-09-21T06:34:57.491696+0000","last_peered":"2024-09-21T06:34:57.491696+0000","last_clean":"2024-09-21T06:34:57.491696+0000","last_became_active":"2024-09-21T06:34:55.708602+0000","last_became_peered":"2024-09-21T06:34:55.708602+0000","last_unstale":"2024-09-21T06:34:57.491696+0000","last_undegraded":"2024-09-21T06:34:57.491696+0000","last_fullsized":"2024-09-21T06:34:57.491696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:53.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:54.709657+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:54.709345+0000","last_became_peered":"2024-09-21T06:34:54.709345+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.714451+0000","last_change":"2024-09-21T06:34:55.709527+0000","last_active":"2024-09-21T06:34:55.714451+0000","last_peered":"2024-09-21T06:34:55.714451+0000","last_clean":"2024-09-21T06:34:55.714451+0000","last_became_active":"2024-09-21T06:34:55.709199+0000","last_became_peered":"2024-09-21T06:34:55.709199+0000","last_unstale":"2024-09-21T06:34:55.714451+0000","last_undegraded":"2024-09-21T06:34:55.714451+0000","last_fullsized":"2024-09-21T06:34:55.714451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:07.792379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682816+0000","last_change":"2024-09-21T06:34:54.708914+0000","last_active":"2024-09-21T06:34:57.682816+0000","last_peered":"2024-09-21T06:34:57.682816+0000","last_clean":"2024-09-21T06:34:57.682816+0000","last_became_active":"2024-09-21T06:34:54.706923+0000","last_became_peered":"2024-09-21T06:34:54.706923+0000","last_unstale":"2024-09-21T06:34:57.682816+0000","last_undegraded":"2024-09-21T06:34:57.682816+0000","last_fullsized":"2024-09-21T06:34:57.682816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491887+0000","last_change":"2024-09-21T06:34:55.710127+0000","last_active":"2024-09-21T06:34:57.491887+0000","last_peered":"2024-09-21T06:34:57.491887+0000","last_clean":"2024-09-21T06:34:57.491887+0000","last_became_active":"2024-09-21T06:34:55.709577+0000","last_became_peered":"2024-09-21T06:34:55.709577+0000","last_unstale":"2024-09-21T06:34:57.491887+0000","last_undegraded":"2024-09-21T06:34:57.491887+0000","last_fullsized":"2024-09-21T06:34:57.491887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.912010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.702796+0000","last_change":"2024-09-21T06:34:54.711684+0000","last_active":"2024-09-21T06:34:55.702796+0000","last_peered":"2024-09-21T06:34:55.702796+0000","last_clean":"2024-09-21T06:34:55.702796+0000","last_became_active":"2024-09-21T06:34:54.711307+0000","last_became_peered":"2024-09-21T06:34:54.711307+0000","last_unstale":"2024-09-21T06:34:55.702796+0000","last_undegraded":"2024-09-21T06:34:55.702796+0000","last_fullsized":"2024-09-21T06:34:55.702796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:08.787586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490933+0000","last_change":"2024-09-21T06:34:55.718040+0000","last_active":"2024-09-21T06:34:57.490933+0000","last_peered":"2024-09-21T06:34:57.490933+0000","last_clean":"2024-09-21T06:34:57.490933+0000","last_became_active":"2024-09-21T06:34:55.717355+0000","last_became_peered":"2024-09-21T06:34:55.717355+0000","last_unstale":"2024-09-21T06:34:57.490933+0000","last_undegraded":"2024-09-21T06:34:57.490933+0000","last_fullsized":"2024-09-21T06:34:57.490933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897542+0000","last_change":"2024-09-21T06:34:54.709082+0000","last_active":"2024-09-21T06:34:56.897542+0000","last_peered":"2024-09-21T06:34:56.897542+0000","last_clean":"2024-09-21T06:34:56.897542+0000","last_became_active":"2024-09-21T06:34:54.708646+0000","last_became_peered":"2024-09-21T06:34:54.708646+0000","last_unstale":"2024-09-21T06:34:56.897542+0000","last_undegraded":"2024-09-21T06:34:56.897542+0000","last_fullsized":"2024-09-21T06:34:56.897542+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:59.547312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490987+0000","last_change":"2024-09-21T06:34:55.706722+0000","last_active":"2024-09-21T06:34:57.490987+0000","last_peered":"2024-09-21T06:34:57.490987+0000","last_clean":"2024-09-21T06:34:57.490987+0000","last_became_active":"2024-09-21T06:34:55.706428+0000","last_became_peered":"2024-09-21T06:34:55.706428+0000","last_unstale":"2024-09-21T06:34:57.490987+0000","last_undegraded":"2024-09-21T06:34:57.490987+0000","last_fullsized":"2024-09-21T06:34:57.490987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:17.159683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693179+0000","last_change":"2024-09-21T06:34:54.700087+0000","last_active":"2024-09-21T06:34:56.693179+0000","last_peered":"2024-09-21T06:34:56.693179+0000","last_clean":"2024-09-21T06:34:56.693179+0000","last_became_active":"2024-09-21T06:34:54.699715+0000","last_became_peered":"2024-09-21T06:34:54.699715+0000","last_unstale":"2024-09-21T06:34:56.693179+0000","last_undegraded":"2024-09-21T06:34:56.693179+0000","last_fullsized":"2024-09-21T06:34:56.693179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693783+0000","last_change":"2024-09-21T06:34:55.714419+0000","last_active":"2024-09-21T06:34:56.693783+0000","last_peered":"2024-09-21T06:34:56.693783+0000","last_clean":"2024-09-21T06:34:56.693783+0000","last_became_active":"2024-09-21T06:34:55.714040+0000","last_became_peered":"2024-09-21T06:34:55.714040+0000","last_unstale":"2024-09-21T06:34:56.693783+0000","last_undegraded":"2024-09-21T06:34:56.693783+0000","last_fullsized":"2024-09-21T06:34:56.693783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.757308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897468+0000","last_change":"2024-09-21T06:34:54.708697+0000","last_active":"2024-09-21T06:34:56.897468+0000","last_peered":"2024-09-21T06:34:56.897468+0000","last_clean":"2024-09-21T06:34:56.897468+0000","last_became_active":"2024-09-21T06:34:54.708338+0000","last_became_peered":"2024-09-21T06:34:54.708338+0000","last_unstale":"2024-09-21T06:34:56.897468+0000","last_undegraded":"2024-09-21T06:34:56.897468+0000","last_fullsized":"2024-09-21T06:34:56.897468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:29.524412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.733897+0000","last_change":"2024-09-21T06:34:55.732505+0000","last_active":"2024-09-21T06:34:55.733897+0000","last_peered":"2024-09-21T06:34:55.733897+0000","last_clean":"2024-09-21T06:34:55.733897+0000","last_became_active":"2024-09-21T06:34:55.732209+0000","last_became_peered":"2024-09-21T06:34:55.732209+0000","last_unstale":"2024-09-21T06:34:55.733897+0000","last_undegraded":"2024-09-21T06:34:55.733897+0000","last_fullsized":"2024-09-21T06:34:55.733897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:36.296181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.740620+0000","last_change":"2024-09-21T06:34:54.704500+0000","last_active":"2024-09-21T06:34:55.740620+0000","last_peered":"2024-09-21T06:34:55.740620+0000","last_clean":"2024-09-21T06:34:55.740620+0000","last_became_active":"2024-09-21T06:34:54.704165+0000","last_became_peered":"2024-09-21T06:34:54.704165+0000","last_unstale":"2024-09-21T06:34:55.740620+0000","last_undegraded":"2024-09-21T06:34:55.740620+0000","last_fullsized":"2024-09-21T06:34:55.740620+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.649075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690939+0000","last_change":"2024-09-21T06:34:55.705237+0000","last_active":"2024-09-21T06:34:56.690939+0000","last_peered":"2024-09-21T06:34:56.690939+0000","last_clean":"2024-09-21T06:34:56.690939+0000","last_became_active":"2024-09-21T06:34:55.704951+0000","last_became_peered":"2024-09-21T06:34:55.704951+0000","last_unstale":"2024-09-21T06:34:56.690939+0000","last_undegraded":"2024-09-21T06:34:56.690939+0000","last_fullsized":"2024-09-21T06:34:56.690939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491461+0000","last_change":"2024-09-21T06:34:54.709537+0000","last_active":"2024-09-21T06:34:57.491461+0000","last_peered":"2024-09-21T06:34:57.491461+0000","last_clean":"2024-09-21T06:34:57.491461+0000","last_became_active":"2024-09-21T06:34:54.706229+0000","last_became_peered":"2024-09-21T06:34:54.706229+0000","last_unstale":"2024-09-21T06:34:57.491461+0000","last_undegraded":"2024-09-21T06:34:57.491461+0000","last_fullsized":"2024-09-21T06:34:57.491461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:57.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691227+0000","last_change":"2024-09-21T06:34:55.699196+0000","last_active":"2024-09-21T06:34:56.691227+0000","last_peered":"2024-09-21T06:34:56.691227+0000","last_clean":"2024-09-21T06:34:56.691227+0000","last_became_active":"2024-09-21T06:34:55.698813+0000","last_became_peered":"2024-09-21T06:34:55.698813+0000","last_unstale":"2024-09-21T06:34:56.691227+0000","last_undegraded":"2024-09-21T06:34:56.691227+0000","last_fullsized":"2024-09-21T06:34:56.691227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:46.110588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693829+0000","last_change":"2024-09-21T06:34:54.701370+0000","last_active":"2024-09-21T06:34:56.693829+0000","last_peered":"2024-09-21T06:34:56.693829+0000","last_clean":"2024-09-21T06:34:56.693829+0000","last_became_active":"2024-09-21T06:34:54.700999+0000","last_became_peered":"2024-09-21T06:34:54.700999+0000","last_unstale":"2024-09-21T06:34:56.693829+0000","last_undegraded":"2024-09-21T06:34:56.693829+0000","last_fullsized":"2024-09-21T06:34:56.693829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731581+0000","last_change":"2024-09-21T06:34:55.728639+0000","last_active":"2024-09-21T06:34:55.731581+0000","last_peered":"2024-09-21T06:34:55.731581+0000","last_clean":"2024-09-21T06:34:55.731581+0000","last_became_active":"2024-09-21T06:34:55.728220+0000","last_became_peered":"2024-09-21T06:34:55.728220+0000","last_unstale":"2024-09-21T06:34:55.731581+0000","last_undegraded":"2024-09-21T06:34:55.731581+0000","last_fullsized":"2024-09-21T06:34:55.731581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.666244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490873+0000","last_change":"2024-09-21T06:34:54.708770+0000","last_active":"2024-09-21T06:34:57.490873+0000","last_peered":"2024-09-21T06:34:57.490873+0000","last_clean":"2024-09-21T06:34:57.490873+0000","last_became_active":"2024-09-21T06:34:54.708363+0000","last_became_peered":"2024-09-21T06:34:54.708363+0000","last_unstale":"2024-09-21T06:34:57.490873+0000","last_undegraded":"2024-09-21T06:34:57.490873+0000","last_fullsized":"2024-09-21T06:34:57.490873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:08.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.692465+0000","last_change":"2024-09-21T06:34:54.692465+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.692465+0000","last_undegraded":"2024-09-21T06:34:54.692465+0000","last_fullsized":"2024-09-21T06:34:54.692465+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.734554+0000","last_change":"2024-09-21T06:34:54.713801+0000","last_active":"2024-09-21T06:34:55.734554+0000","last_peered":"2024-09-21T06:34:55.734554+0000","last_clean":"2024-09-21T06:34:55.734554+0000","last_became_active":"2024-09-21T06:34:54.712759+0000","last_became_peered":"2024-09-21T06:34:54.712759+0000","last_unstale":"2024-09-21T06:34:55.734554+0000","last_undegraded":"2024-09-21T06:34:55.734554+0000","last_fullsized":"2024-09-21T06:34:55.734554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:47.339290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692225+0000","last_change":"2024-09-21T06:34:55.732582+0000","last_active":"2024-09-21T06:34:56.692225+0000","last_peered":"2024-09-21T06:34:56.692225+0000","last_clean":"2024-09-21T06:34:56.692225+0000","last_became_active":"2024-09-21T06:34:55.732328+0000","last_became_peered":"2024-09-21T06:34:55.732328+0000","last_unstale":"2024-09-21T06:34:56.692225+0000","last_undegraded":"2024-09-21T06:34:56.692225+0000","last_fullsized":"2024-09-21T06:34:56.692225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897574+0000","last_change":"2024-09-21T06:34:54.710621+0000","last_active":"2024-09-21T06:34:56.897574+0000","last_peered":"2024-09-21T06:34:56.897574+0000","last_clean":"2024-09-21T06:34:56.897574+0000","last_became_active":"2024-09-21T06:34:54.710305+0000","last_became_peered":"2024-09-21T06:34:54.710305+0000","last_unstale":"2024-09-21T06:34:56.897574+0000","last_undegraded":"2024-09-21T06:34:56.897574+0000","last_fullsized":"2024-09-21T06:34:56.897574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:04.681733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683086+0000","last_change":"2024-09-21T06:34:55.716269+0000","last_active":"2024-09-21T06:34:57.683086+0000","last_peered":"2024-09-21T06:34:57.683086+0000","last_clean":"2024-09-21T06:34:57.683086+0000","last_became_active":"2024-09-21T06:34:55.706408+0000","last_became_peered":"2024-09-21T06:34:55.706408+0000","last_unstale":"2024-09-21T06:34:57.683086+0000","last_undegraded":"2024-09-21T06:34:57.683086+0000","last_fullsized":"2024-09-21T06:34:57.683086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683117+0000","last_change":"2024-09-21T06:34:54.712959+0000","last_active":"2024-09-21T06:34:57.683117+0000","last_peered":"2024-09-21T06:34:57.683117+0000","last_clean":"2024-09-21T06:34:57.683117+0000","last_became_active":"2024-09-21T06:34:54.711866+0000","last_became_peered":"2024-09-21T06:34:54.711866+0000","last_unstale":"2024-09-21T06:34:57.683117+0000","last_undegraded":"2024-09-21T06:34:57.683117+0000","last_fullsized":"2024-09-21T06:34:57.683117+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731728+0000","last_change":"2024-09-21T06:34:55.728437+0000","last_active":"2024-09-21T06:34:55.731728+0000","last_peered":"2024-09-21T06:34:55.731728+0000","last_clean":"2024-09-21T06:34:55.731728+0000","last_became_active":"2024-09-21T06:34:55.728111+0000","last_became_peered":"2024-09-21T06:34:55.728111+0000","last_unstale":"2024-09-21T06:34:55.731728+0000","last_undegraded":"2024-09-21T06:34:55.731728+0000","last_fullsized":"2024-09-21T06:34:55.731728+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:21.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897340+0000","last_change":"2024-09-21T06:34:54.710224+0000","last_active":"2024-09-21T06:34:56.897340+0000","last_peered":"2024-09-21T06:34:56.897340+0000","last_clean":"2024-09-21T06:34:56.897340+0000","last_became_active":"2024-09-21T06:34:54.709896+0000","last_became_peered":"2024-09-21T06:34:54.709896+0000","last_unstale":"2024-09-21T06:34:56.897340+0000","last_undegraded":"2024-09-21T06:34:56.897340+0000","last_fullsized":"2024-09-21T06:34:56.897340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:16.511663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491563+0000","last_change":"2024-09-21T06:34:55.708497+0000","last_active":"2024-09-21T06:34:57.491563+0000","last_peered":"2024-09-21T06:34:57.491563+0000","last_clean":"2024-09-21T06:34:57.491563+0000","last_became_active":"2024-09-21T06:34:55.706136+0000","last_became_peered":"2024-09-21T06:34:55.706136+0000","last_unstale":"2024-09-21T06:34:57.491563+0000","last_undegraded":"2024-09-21T06:34:57.491563+0000","last_fullsized":"2024-09-21T06:34:57.491563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:06.652871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713997+0000","last_change":"2024-09-21T06:34:54.712196+0000","last_active":"2024-09-21T06:34:54.713997+0000","last_peered":"2024-09-21T06:34:54.713997+0000","last_clean":"2024-09-21T06:34:54.713997+0000","last_became_active":"2024-09-21T06:34:54.711507+0000","last_became_peered":"2024-09-21T06:34:54.711507+0000","last_unstale":"2024-09-21T06:34:54.713997+0000","last_undegraded":"2024-09-21T06:34:54.713997+0000","last_fullsized":"2024-09-21T06:34:54.713997+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:41.618052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.725352+0000","last_change":"2024-09-21T06:34:55.718899+0000","last_active":"2024-09-21T06:34:55.725352+0000","last_peered":"2024-09-21T06:34:55.725352+0000","last_clean":"2024-09-21T06:34:55.725352+0000","last_became_active":"2024-09-21T06:34:55.718610+0000","last_became_peered":"2024-09-21T06:34:55.718610+0000","last_unstale":"2024-09-21T06:34:55.725352+0000","last_undegraded":"2024-09-21T06:34:55.725352+0000","last_fullsized":"2024-09-21T06:34:55.725352+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:14.255062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707453+0000","last_change":"2024-09-21T06:34:54.704910+0000","last_active":"2024-09-21T06:34:54.707453+0000","last_peered":"2024-09-21T06:34:54.707453+0000","last_clean":"2024-09-21T06:34:54.707453+0000","last_became_active":"2024-09-21T06:34:54.703410+0000","last_became_peered":"2024-09-21T06:34:54.703410+0000","last_unstale":"2024-09-21T06:34:54.707453+0000","last_undegraded":"2024-09-21T06:34:54.707453+0000","last_fullsized":"2024-09-21T06:34:54.707453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:42.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731552+0000","last_change":"2024-09-21T06:34:55.726247+0000","last_active":"2024-09-21T06:34:55.731552+0000","last_peered":"2024-09-21T06:34:55.731552+0000","last_clean":"2024-09-21T06:34:55.731552+0000","last_became_active":"2024-09-21T06:34:55.725850+0000","last_became_peered":"2024-09-21T06:34:55.725850+0000","last_unstale":"2024-09-21T06:34:55.731552+0000","last_undegraded":"2024-09-21T06:34:55.731552+0000","last_fullsized":"2024-09-21T06:34:55.731552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:40.564061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897179+0000","last_change":"2024-09-21T06:34:54.712284+0000","last_active":"2024-09-21T06:34:56.897179+0000","last_peered":"2024-09-21T06:34:56.897179+0000","last_clean":"2024-09-21T06:34:56.897179+0000","last_became_active":"2024-09-21T06:34:54.711828+0000","last_became_peered":"2024-09-21T06:34:54.711828+0000","last_unstale":"2024-09-21T06:34:56.897179+0000","last_undegraded":"2024-09-21T06:34:56.897179+0000","last_fullsized":"2024-09-21T06:34:56.897179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:29.075613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491685+0000","last_change":"2024-09-21T06:34:55.713268+0000","last_active":"2024-09-21T06:34:57.491685+0000","last_peered":"2024-09-21T06:34:57.491685+0000","last_clean":"2024-09-21T06:34:57.491685+0000","last_became_active":"2024-09-21T06:34:55.712940+0000","last_became_peered":"2024-09-21T06:34:55.712940+0000","last_unstale":"2024-09-21T06:34:57.491685+0000","last_undegraded":"2024-09-21T06:34:57.491685+0000","last_fullsized":"2024-09-21T06:34:57.491685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:20.633620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692250+0000","last_change":"2024-09-21T06:34:54.706724+0000","last_active":"2024-09-21T06:34:56.692250+0000","last_peered":"2024-09-21T06:34:56.692250+0000","last_clean":"2024-09-21T06:34:56.692250+0000","last_became_active":"2024-09-21T06:34:54.706045+0000","last_became_peered":"2024-09-21T06:34:54.706045+0000","last_unstale":"2024-09-21T06:34:56.692250+0000","last_undegraded":"2024-09-21T06:34:56.692250+0000","last_fullsized":"2024-09-21T06:34:56.692250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.717407+0000","last_change":"2024-09-21T06:34:55.716497+0000","last_active":"2024-09-21T06:34:55.717407+0000","last_peered":"2024-09-21T06:34:55.717407+0000","last_clean":"2024-09-21T06:34:55.717407+0000","last_became_active":"2024-09-21T06:34:55.716182+0000","last_became_peered":"2024-09-21T06:34:55.716182+0000","last_unstale":"2024-09-21T06:34:55.717407+0000","last_undegraded":"2024-09-21T06:34:55.717407+0000","last_fullsized":"2024-09-21T06:34:55.717407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:13.835969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.704018+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.703663+0000","last_became_peered":"2024-09-21T06:34:54.703663+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:45.237205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721388+0000","last_change":"2024-09-21T06:34:55.712873+0000","last_active":"2024-09-21T06:34:55.721388+0000","last_peered":"2024-09-21T06:34:55.721388+0000","last_clean":"2024-09-21T06:34:55.721388+0000","last_became_active":"2024-09-21T06:34:55.711695+0000","last_became_peered":"2024-09-21T06:34:55.711695+0000","last_unstale":"2024-09-21T06:34:55.721388+0000","last_undegraded":"2024-09-21T06:34:55.721388+0000","last_fullsized":"2024-09-21T06:34:55.721388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:31.987545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713460+0000","last_change":"2024-09-21T06:34:54.709868+0000","last_active":"2024-09-21T06:34:54.713460+0000","last_peered":"2024-09-21T06:34:54.713460+0000","last_clean":"2024-09-21T06:34:54.713460+0000","last_became_active":"2024-09-21T06:34:54.708447+0000","last_became_peered":"2024-09-21T06:34:54.708447+0000","last_unstale":"2024-09-21T06:34:54.713460+0000","last_undegraded":"2024-09-21T06:34:54.713460+0000","last_fullsized":"2024-09-21T06:34:54.713460+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:13.936837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693744+0000","last_change":"2024-09-21T06:34:55.729064+0000","last_active":"2024-09-21T06:34:56.693744+0000","last_peered":"2024-09-21T06:34:56.693744+0000","last_clean":"2024-09-21T06:34:56.693744+0000","last_became_active":"2024-09-21T06:34:55.728795+0000","last_became_peered":"2024-09-21T06:34:55.728795+0000","last_unstale":"2024-09-21T06:34:56.693744+0000","last_undegraded":"2024-09-21T06:34:56.693744+0000","last_fullsized":"2024-09-21T06:34:56.693744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:38.265711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.703782+0000","last_change":"2024-09-21T06:34:54.702424+0000","last_active":"2024-09-21T06:34:55.703782+0000","last_peered":"2024-09-21T06:34:55.703782+0000","last_clean":"2024-09-21T06:34:55.703782+0000","last_became_active":"2024-09-21T06:34:54.702065+0000","last_became_peered":"2024-09-21T06:34:54.702065+0000","last_unstale":"2024-09-21T06:34:55.703782+0000","last_undegraded":"2024-09-21T06:34:55.703782+0000","last_fullsized":"2024-09-21T06:34:55.703782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:51.535195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683128+0000","last_change":"2024-09-21T06:34:55.718743+0000","last_active":"2024-09-21T06:34:57.683128+0000","last_peered":"2024-09-21T06:34:57.683128+0000","last_clean":"2024-09-21T06:34:57.683128+0000","last_became_active":"2024-09-21T06:34:55.703205+0000","last_became_peered":"2024-09-21T06:34:55.703205+0000","last_unstale":"2024-09-21T06:34:57.683128+0000","last_undegraded":"2024-09-21T06:34:57.683128+0000","last_fullsized":"2024-09-21T06:34:57.683128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491730+0000","last_change":"2024-09-21T06:34:54.711548+0000","last_active":"2024-09-21T06:34:57.491730+0000","last_peered":"2024-09-21T06:34:57.491730+0000","last_clean":"2024-09-21T06:34:57.491730+0000","last_became_active":"2024-09-21T06:34:54.711141+0000","last_became_peered":"2024-09-21T06:34:54.711141+0000","last_unstale":"2024-09-21T06:34:57.491730+0000","last_undegraded":"2024-09-21T06:34:57.491730+0000","last_fullsized":"2024-09-21T06:34:57.491730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:27.210366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491185+0000","last_change":"2024-09-21T06:34:55.712582+0000","last_active":"2024-09-21T06:34:57.491185+0000","last_peered":"2024-09-21T06:34:57.491185+0000","last_clean":"2024-09-21T06:34:57.491185+0000","last_became_active":"2024-09-21T06:34:55.710833+0000","last_became_peered":"2024-09-21T06:34:55.710833+0000","last_unstale":"2024-09-21T06:34:57.491185+0000","last_undegraded":"2024-09-21T06:34:57.491185+0000","last_fullsized":"2024-09-21T06:34:57.491185+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.706570+0000","last_change":"2024-09-21T06:34:54.713168+0000","last_active":"2024-09-21T06:34:55.706570+0000","last_peered":"2024-09-21T06:34:55.706570+0000","last_clean":"2024-09-21T06:34:55.706570+0000","last_became_active":"2024-09-21T06:34:54.712837+0000","last_became_peered":"2024-09-21T06:34:54.712837+0000","last_unstale":"2024-09-21T06:34:55.706570+0000","last_undegraded":"2024-09-21T06:34:55.706570+0000","last_fullsized":"2024-09-21T06:34:55.706570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:29.731794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491694+0000","last_change":"2024-09-21T06:34:55.711645+0000","last_active":"2024-09-21T06:34:57.491694+0000","last_peered":"2024-09-21T06:34:57.491694+0000","last_clean":"2024-09-21T06:34:57.491694+0000","last_became_active":"2024-09-21T06:34:55.711312+0000","last_became_peered":"2024-09-21T06:34:55.711312+0000","last_unstale":"2024-09-21T06:34:57.491694+0000","last_undegraded":"2024-09-21T06:34:57.491694+0000","last_fullsized":"2024-09-21T06:34:57.491694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.705515+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.705153+0000","last_became_peered":"2024-09-21T06:34:54.705153+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:38.546287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897683+0000","last_change":"2024-09-21T06:34:55.729077+0000","last_active":"2024-09-21T06:34:56.897683+0000","last_peered":"2024-09-21T06:34:56.897683+0000","last_clean":"2024-09-21T06:34:56.897683+0000","last_became_active":"2024-09-21T06:34:55.728776+0000","last_became_peered":"2024-09-21T06:34:55.728776+0000","last_unstale":"2024-09-21T06:34:56.897683+0000","last_undegraded":"2024-09-21T06:34:56.897683+0000","last_fullsized":"2024-09-21T06:34:56.897683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683192+0000","last_change":"2024-09-21T06:34:54.703341+0000","last_active":"2024-09-21T06:34:57.683192+0000","last_peered":"2024-09-21T06:34:57.683192+0000","last_clean":"2024-09-21T06:34:57.683192+0000","last_became_active":"2024-09-21T06:34:54.703002+0000","last_became_peered":"2024-09-21T06:34:54.703002+0000","last_unstale":"2024-09-21T06:34:57.683192+0000","last_undegraded":"2024-09-21T06:34:57.683192+0000","last_fullsized":"2024-09-21T06:34:57.683192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691146+0000","last_change":"2024-09-21T06:34:55.737366+0000","last_active":"2024-09-21T06:34:56.691146+0000","last_peered":"2024-09-21T06:34:56.691146+0000","last_clean":"2024-09-21T06:34:56.691146+0000","last_became_active":"2024-09-21T06:34:55.734444+0000","last_became_peered":"2024-09-21T06:34:55.734444+0000","last_unstale":"2024-09-21T06:34:56.691146+0000","last_undegraded":"2024-09-21T06:34:56.691146+0000","last_fullsized":"2024-09-21T06:34:56.691146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:47.619903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707339+0000","last_change":"2024-09-21T06:34:54.702747+0000","last_active":"2024-09-21T06:34:54.707339+0000","last_peered":"2024-09-21T06:34:54.707339+0000","last_clean":"2024-09-21T06:34:54.707339+0000","last_became_active":"2024-09-21T06:34:54.702221+0000","last_became_peered":"2024-09-21T06:34:54.702221+0000","last_unstale":"2024-09-21T06:34:54.707339+0000","last_undegraded":"2024-09-21T06:34:54.707339+0000","last_fullsized":"2024-09-21T06:34:54.707339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:50.889270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897188+0000","last_change":"2024-09-21T06:34:55.724899+0000","last_active":"2024-09-21T06:34:56.897188+0000","last_peered":"2024-09-21T06:34:56.897188+0000","last_clean":"2024-09-21T06:34:56.897188+0000","last_became_active":"2024-09-21T06:34:55.724167+0000","last_became_peered":"2024-09-21T06:34:55.724167+0000","last_unstale":"2024-09-21T06:34:56.897188+0000","last_undegraded":"2024-09-21T06:34:56.897188+0000","last_fullsized":"2024-09-21T06:34:56.897188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491794+0000","last_change":"2024-09-21T06:34:54.707367+0000","last_active":"2024-09-21T06:34:57.491794+0000","last_peered":"2024-09-21T06:34:57.491794+0000","last_clean":"2024-09-21T06:34:57.491794+0000","last_became_active":"2024-09-21T06:34:54.705863+0000","last_became_peered":"2024-09-21T06:34:54.705863+0000","last_unstale":"2024-09-21T06:34:57.491794+0000","last_undegraded":"2024-09-21T06:34:57.491794+0000","last_fullsized":"2024-09-21T06:34:57.491794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.687054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682836+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:34:57.682836+0000","last_peered":"2024-09-21T06:34:57.682836+0000","last_clean":"2024-09-21T06:34:57.682836+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:34:57.682836+0000","last_undegraded":"2024-09-21T06:34:57.682836+0000","last_fullsized":"2024-09-21T06:34:57.682836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1277952,"data_stored":589850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509529,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":595924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1282048,"data_stored":590810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":1179658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":596472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1839104,"data_stored":1165732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:00.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:35:00.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 -- 172.21.15.37:0/683606960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce415dfe0 msgr2=0x7f9ce4160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 --2- 172.21.15.37:0/683606960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce415dfe0 0x7f9ce4160a40 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4153b50 tx=0x7f9cd802a560 comp rx=0 tx=0).stop 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 -- 172.21.15.37:0/683606960 shutdown_connections 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 --2- 172.21.15.37:0/683606960 >> v2:172.21.15.150:3300/0 conn(0x7f9ce406e160 0x7f9ce406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 --2- 172.21.15.37:0/683606960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce415dfe0 0x7f9ce4160a40 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 -- 172.21.15.37:0/683606960 >> 172.21.15.37:0/683606960 conn(0x7f9ce406f940 msgr2=0x7f9ce40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 -- 172.21.15.37:0/683606960 shutdown_connections 2024-09-21T06:35:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.954+0000 7f9ceb200640 1 -- 172.21.15.37:0/683606960 wait complete. 2024-09-21T06:35:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.956+0000 7f9ceb200640 1 Processor -- start 2024-09-21T06:35:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.956+0000 7f9ceb200640 1 -- start start 2024-09-21T06:35:00.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40cc8d0 0x7f9ce40aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce415dfe0 0x7f9ce40a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce40b4b70 con 0x7f9ce406e160 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce40b49f0 con 0x7f9ce40cc8d0 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ceb200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce40b4cf0 con 0x7f9ce415dfe0 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38300/0 (socket says 172.21.15.37:38300) 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9e00640 1 -- 172.21.15.37:0/769503197 learned_addr learned my addr 172.21.15.37:0/769503197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9400640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40cc8d0 0x7f9ce40aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9cea800640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce415dfe0 0x7f9ce40a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9e00640 1 -- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce415dfe0 msgr2=0x7f9ce40a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.958+0000 7f9ce9e00640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce415dfe0 0x7f9ce40a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ce9e00640 1 -- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40cc8d0 msgr2=0x7f9ce40aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ce9e00640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40cc8d0 0x7f9ce40aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ce9e00640 1 -- 172.21.15.37:0/769503197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce40aa990 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ce9e00640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8034040 tx=0x7f9cd8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd803e030 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cd8003e00 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41f6310 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8035040 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.959+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41f68d0 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.960+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce40b5210 con 0x7f9ce406e160 2024-09-21T06:35:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.961+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9cd8041030 con 0x7f9ce406e160 2024-09-21T06:35:00.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.963+0000 7f9cde000640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 0x7f9cc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.963+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f9cd80bb5d0 con 0x7f9ce406e160 2024-09-21T06:35:00.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.964+0000 7f9ce9400640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 0x7f9cc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.964+0000 7f9ce9400640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 0x7f9cc807b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4074130 tx=0x7f9cd400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:00.968+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9cd80832b0 con 0x7f9ce406e160 2024-09-21T06:35:01.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.160+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ce40cc0e0 con 0x7f9cc8078b90 2024-09-21T06:35:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.165+0000 7f9cde000640 1 -- 172.21.15.37:0/769503197 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f9ce40cc0e0 con 0x7f9cc8078b90 2024-09-21T06:35:01.169 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:01.170 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 msgr2=0x7f9cc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 0x7f9cc807b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4074130 tx=0x7f9cd400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 msgr2=0x7f9ce4074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8034040 tx=0x7f9cd8003910 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 shutdown_connections 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9cc8078b90 0x7f9cc807b0a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce415dfe0 0x7f9ce40a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40cc8d0 0x7f9ce40aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 --2- 172.21.15.37:0/769503197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ce406e160 0x7f9ce4074510 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 >> 172.21.15.37:0/769503197 conn(0x7f9ce406f940 msgr2=0x7f9ce40cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 shutdown_connections 2024-09-21T06:35:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.169+0000 7f9ceb200640 1 -- 172.21.15.37:0/769503197 wait complete. 2024-09-21T06:35:01.195 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T06:34:59.205143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260700,"kb_used_data":11160,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714020,"statfs":{"total":773094113280,"available":772827156480,"internally_reserved":0,"allocated":11427840,"data_stored":5911950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.525742"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491654+0000","last_change":"2024-09-21T06:34:55.702355+0000","last_active":"2024-09-21T06:34:57.491654+0000","last_peered":"2024-09-21T06:34:57.491654+0000","last_clean":"2024-09-21T06:34:57.491654+0000","last_became_active":"2024-09-21T06:34:55.701180+0000","last_became_peered":"2024-09-21T06:34:55.701180+0000","last_unstale":"2024-09-21T06:34:57.491654+0000","last_undegraded":"2024-09-21T06:34:57.491654+0000","last_fullsized":"2024-09-21T06:34:57.491654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:40.085758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690863+0000","last_change":"2024-09-21T06:34:55.734713+0000","last_active":"2024-09-21T06:34:56.690863+0000","last_peered":"2024-09-21T06:34:56.690863+0000","last_clean":"2024-09-21T06:34:56.690863+0000","last_became_active":"2024-09-21T06:34:55.734367+0000","last_became_peered":"2024-09-21T06:34:55.734367+0000","last_unstale":"2024-09-21T06:34:56.690863+0000","last_undegraded":"2024-09-21T06:34:56.690863+0000","last_fullsized":"2024-09-21T06:34:56.690863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.630265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689595+0000","last_change":"2024-09-21T06:34:54.689595+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689595+0000","last_undegraded":"2024-09-21T06:34:54.689595+0000","last_fullsized":"2024-09-21T06:34:54.689595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491281+0000","last_change":"2024-09-21T06:34:55.713592+0000","last_active":"2024-09-21T06:34:57.491281+0000","last_peered":"2024-09-21T06:34:57.491281+0000","last_clean":"2024-09-21T06:34:57.491281+0000","last_became_active":"2024-09-21T06:34:55.711240+0000","last_became_peered":"2024-09-21T06:34:55.711240+0000","last_unstale":"2024-09-21T06:34:57.491281+0000","last_undegraded":"2024-09-21T06:34:57.491281+0000","last_fullsized":"2024-09-21T06:34:57.491281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.722733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491268+0000","last_change":"2024-09-21T06:34:55.712441+0000","last_active":"2024-09-21T06:34:57.491268+0000","last_peered":"2024-09-21T06:34:57.491268+0000","last_clean":"2024-09-21T06:34:57.491268+0000","last_became_active":"2024-09-21T06:34:55.710543+0000","last_became_peered":"2024-09-21T06:34:55.710543+0000","last_unstale":"2024-09-21T06:34:57.491268+0000","last_undegraded":"2024-09-21T06:34:57.491268+0000","last_fullsized":"2024-09-21T06:34:57.491268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:02.131613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897116+0000","last_change":"2024-09-21T06:34:55.727939+0000","last_active":"2024-09-21T06:34:56.897116+0000","last_peered":"2024-09-21T06:34:56.897116+0000","last_clean":"2024-09-21T06:34:56.897116+0000","last_became_active":"2024-09-21T06:34:55.727032+0000","last_became_peered":"2024-09-21T06:34:55.727032+0000","last_unstale":"2024-09-21T06:34:56.897116+0000","last_undegraded":"2024-09-21T06:34:56.897116+0000","last_fullsized":"2024-09-21T06:34:56.897116+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.403443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.896947+0000","last_change":"2024-09-21T06:34:55.723696+0000","last_active":"2024-09-21T06:34:56.896947+0000","last_peered":"2024-09-21T06:34:56.896947+0000","last_clean":"2024-09-21T06:34:56.896947+0000","last_became_active":"2024-09-21T06:34:55.723259+0000","last_became_peered":"2024-09-21T06:34:55.723259+0000","last_unstale":"2024-09-21T06:34:56.896947+0000","last_undegraded":"2024-09-21T06:34:56.896947+0000","last_fullsized":"2024-09-21T06:34:56.896947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:07.748611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691935+0000","last_change":"2024-09-21T06:34:55.723651+0000","last_active":"2024-09-21T06:34:56.691935+0000","last_peered":"2024-09-21T06:34:56.691935+0000","last_clean":"2024-09-21T06:34:56.691935+0000","last_became_active":"2024-09-21T06:34:55.723288+0000","last_became_peered":"2024-09-21T06:34:55.723288+0000","last_unstale":"2024-09-21T06:34:56.691935+0000","last_undegraded":"2024-09-21T06:34:56.691935+0000","last_fullsized":"2024-09-21T06:34:56.691935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:30.694885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691167+0000","last_change":"2024-09-21T06:34:55.704172+0000","last_active":"2024-09-21T06:34:56.691167+0000","last_peered":"2024-09-21T06:34:56.691167+0000","last_clean":"2024-09-21T06:34:56.691167+0000","last_became_active":"2024-09-21T06:34:55.703829+0000","last_became_peered":"2024-09-21T06:34:55.703829+0000","last_unstale":"2024-09-21T06:34:56.691167+0000","last_undegraded":"2024-09-21T06:34:56.691167+0000","last_fullsized":"2024-09-21T06:34:56.691167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:10.923582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693214+0000","last_change":"2024-09-21T06:34:55.714365+0000","last_active":"2024-09-21T06:34:56.693214+0000","last_peered":"2024-09-21T06:34:56.693214+0000","last_clean":"2024-09-21T06:34:56.693214+0000","last_became_active":"2024-09-21T06:34:55.711411+0000","last_became_peered":"2024-09-21T06:34:55.711411+0000","last_unstale":"2024-09-21T06:34:56.693214+0000","last_undegraded":"2024-09-21T06:34:56.693214+0000","last_fullsized":"2024-09-21T06:34:56.693214+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:55.929081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721189+0000","last_change":"2024-09-21T06:34:55.712794+0000","last_active":"2024-09-21T06:34:55.721189+0000","last_peered":"2024-09-21T06:34:55.721189+0000","last_clean":"2024-09-21T06:34:55.721189+0000","last_became_active":"2024-09-21T06:34:55.712487+0000","last_became_peered":"2024-09-21T06:34:55.712487+0000","last_unstale":"2024-09-21T06:34:55.721189+0000","last_undegraded":"2024-09-21T06:34:55.721189+0000","last_fullsized":"2024-09-21T06:34:55.721189+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:48.749080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897088+0000","last_change":"2024-09-21T06:34:55.720101+0000","last_active":"2024-09-21T06:34:56.897088+0000","last_peered":"2024-09-21T06:34:56.897088+0000","last_clean":"2024-09-21T06:34:56.897088+0000","last_became_active":"2024-09-21T06:34:55.719743+0000","last_became_peered":"2024-09-21T06:34:55.719743+0000","last_unstale":"2024-09-21T06:34:56.897088+0000","last_undegraded":"2024-09-21T06:34:56.897088+0000","last_fullsized":"2024-09-21T06:34:56.897088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:52.305232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491183+0000","last_change":"2024-09-21T06:34:55.712044+0000","last_active":"2024-09-21T06:34:57.491183+0000","last_peered":"2024-09-21T06:34:57.491183+0000","last_clean":"2024-09-21T06:34:57.491183+0000","last_became_active":"2024-09-21T06:34:55.710142+0000","last_became_peered":"2024-09-21T06:34:55.710142+0000","last_unstale":"2024-09-21T06:34:57.491183+0000","last_undegraded":"2024-09-21T06:34:57.491183+0000","last_fullsized":"2024-09-21T06:34:57.491183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:42.402252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897146+0000","last_change":"2024-09-21T06:34:55.722424+0000","last_active":"2024-09-21T06:34:56.897146+0000","last_peered":"2024-09-21T06:34:56.897146+0000","last_clean":"2024-09-21T06:34:56.897146+0000","last_became_active":"2024-09-21T06:34:55.722091+0000","last_became_peered":"2024-09-21T06:34:55.722091+0000","last_unstale":"2024-09-21T06:34:56.897146+0000","last_undegraded":"2024-09-21T06:34:56.897146+0000","last_fullsized":"2024-09-21T06:34:56.897146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:23.664483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690751+0000","last_change":"2024-09-21T06:34:55.727747+0000","last_active":"2024-09-21T06:34:56.690751+0000","last_peered":"2024-09-21T06:34:56.690751+0000","last_clean":"2024-09-21T06:34:56.690751+0000","last_became_active":"2024-09-21T06:34:55.727447+0000","last_became_peered":"2024-09-21T06:34:55.727447+0000","last_unstale":"2024-09-21T06:34:56.690751+0000","last_undegraded":"2024-09-21T06:34:56.690751+0000","last_fullsized":"2024-09-21T06:34:56.690751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.086183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683012+0000","last_change":"2024-09-21T06:34:55.710554+0000","last_active":"2024-09-21T06:34:57.683012+0000","last_peered":"2024-09-21T06:34:57.683012+0000","last_clean":"2024-09-21T06:34:57.683012+0000","last_became_active":"2024-09-21T06:34:55.710216+0000","last_became_peered":"2024-09-21T06:34:55.710216+0000","last_unstale":"2024-09-21T06:34:57.683012+0000","last_undegraded":"2024-09-21T06:34:57.683012+0000","last_fullsized":"2024-09-21T06:34:57.683012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.693106+0000","last_change":"2024-09-21T06:34:54.693106+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.693106+0000","last_undegraded":"2024-09-21T06:34:54.693106+0000","last_fullsized":"2024-09-21T06:34:54.693106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690915+0000","last_change":"2024-09-21T06:34:55.702753+0000","last_active":"2024-09-21T06:34:56.690915+0000","last_peered":"2024-09-21T06:34:56.690915+0000","last_clean":"2024-09-21T06:34:56.690915+0000","last_became_active":"2024-09-21T06:34:55.702431+0000","last_became_peered":"2024-09-21T06:34:55.702431+0000","last_unstale":"2024-09-21T06:34:56.690915+0000","last_undegraded":"2024-09-21T06:34:56.690915+0000","last_fullsized":"2024-09-21T06:34:56.690915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:07.262224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491255+0000","last_change":"2024-09-21T06:34:55.731627+0000","last_active":"2024-09-21T06:34:57.491255+0000","last_peered":"2024-09-21T06:34:57.491255+0000","last_clean":"2024-09-21T06:34:57.491255+0000","last_became_active":"2024-09-21T06:34:55.731309+0000","last_became_peered":"2024-09-21T06:34:55.731309+0000","last_unstale":"2024-09-21T06:34:57.491255+0000","last_undegraded":"2024-09-21T06:34:57.491255+0000","last_fullsized":"2024-09-21T06:34:57.491255+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:24.150794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:55.710593+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:55.710307+0000","last_became_peered":"2024-09-21T06:34:55.710307+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:54.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.689114+0000","last_change":"2024-09-21T06:34:54.689114+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.689114+0000","last_undegraded":"2024-09-21T06:34:54.689114+0000","last_fullsized":"2024-09-21T06:34:54.689114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731832+0000","last_change":"2024-09-21T06:34:55.728815+0000","last_active":"2024-09-21T06:34:55.731832+0000","last_peered":"2024-09-21T06:34:55.731832+0000","last_clean":"2024-09-21T06:34:55.731832+0000","last_became_active":"2024-09-21T06:34:55.728227+0000","last_became_peered":"2024-09-21T06:34:55.728227+0000","last_unstale":"2024-09-21T06:34:55.731832+0000","last_undegraded":"2024-09-21T06:34:55.731832+0000","last_fullsized":"2024-09-21T06:34:55.731832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:32.261965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692011+0000","last_change":"2024-09-21T06:34:55.731414+0000","last_active":"2024-09-21T06:34:56.692011+0000","last_peered":"2024-09-21T06:34:56.692011+0000","last_clean":"2024-09-21T06:34:56.692011+0000","last_became_active":"2024-09-21T06:34:55.731103+0000","last_became_peered":"2024-09-21T06:34:55.731103+0000","last_unstale":"2024-09-21T06:34:56.692011+0000","last_undegraded":"2024-09-21T06:34:56.692011+0000","last_fullsized":"2024-09-21T06:34:56.692011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691979+0000","last_change":"2024-09-21T06:34:55.715389+0000","last_active":"2024-09-21T06:34:56.691979+0000","last_peered":"2024-09-21T06:34:56.691979+0000","last_clean":"2024-09-21T06:34:56.691979+0000","last_became_active":"2024-09-21T06:34:55.715071+0000","last_became_peered":"2024-09-21T06:34:55.715071+0000","last_unstale":"2024-09-21T06:34:56.691979+0000","last_undegraded":"2024-09-21T06:34:56.691979+0000","last_fullsized":"2024-09-21T06:34:56.691979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682949+0000","last_change":"2024-09-21T06:34:55.717327+0000","last_active":"2024-09-21T06:34:57.682949+0000","last_peered":"2024-09-21T06:34:57.682949+0000","last_clean":"2024-09-21T06:34:57.682949+0000","last_became_active":"2024-09-21T06:34:55.716998+0000","last_became_peered":"2024-09-21T06:34:55.716998+0000","last_unstale":"2024-09-21T06:34:57.682949+0000","last_undegraded":"2024-09-21T06:34:57.682949+0000","last_fullsized":"2024-09-21T06:34:57.682949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683071+0000","last_change":"2024-09-21T06:34:55.717181+0000","last_active":"2024-09-21T06:34:57.683071+0000","last_peered":"2024-09-21T06:34:57.683071+0000","last_clean":"2024-09-21T06:34:57.683071+0000","last_became_active":"2024-09-21T06:34:55.716922+0000","last_became_peered":"2024-09-21T06:34:55.716922+0000","last_unstale":"2024-09-21T06:34:57.683071+0000","last_undegraded":"2024-09-21T06:34:57.683071+0000","last_fullsized":"2024-09-21T06:34:57.683071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683004+0000","last_change":"2024-09-21T06:34:55.715886+0000","last_active":"2024-09-21T06:34:57.683004+0000","last_peered":"2024-09-21T06:34:57.683004+0000","last_clean":"2024-09-21T06:34:57.683004+0000","last_became_active":"2024-09-21T06:34:55.715533+0000","last_became_peered":"2024-09-21T06:34:55.715533+0000","last_unstale":"2024-09-21T06:34:57.683004+0000","last_undegraded":"2024-09-21T06:34:57.683004+0000","last_fullsized":"2024-09-21T06:34:57.683004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897335+0000","last_change":"2024-09-21T06:34:55.730908+0000","last_active":"2024-09-21T06:34:56.897335+0000","last_peered":"2024-09-21T06:34:56.897335+0000","last_clean":"2024-09-21T06:34:56.897335+0000","last_became_active":"2024-09-21T06:34:55.730593+0000","last_became_peered":"2024-09-21T06:34:55.730593+0000","last_unstale":"2024-09-21T06:34:56.897335+0000","last_undegraded":"2024-09-21T06:34:56.897335+0000","last_fullsized":"2024-09-21T06:34:56.897335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692651+0000","last_change":"2024-09-21T06:34:55.728164+0000","last_active":"2024-09-21T06:34:56.692651+0000","last_peered":"2024-09-21T06:34:56.692651+0000","last_clean":"2024-09-21T06:34:56.692651+0000","last_became_active":"2024-09-21T06:34:55.727836+0000","last_became_peered":"2024-09-21T06:34:55.727836+0000","last_unstale":"2024-09-21T06:34:56.692651+0000","last_undegraded":"2024-09-21T06:34:56.692651+0000","last_fullsized":"2024-09-21T06:34:56.692651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.918656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721753+0000","last_change":"2024-09-21T06:34:55.714924+0000","last_active":"2024-09-21T06:34:55.721753+0000","last_peered":"2024-09-21T06:34:55.721753+0000","last_clean":"2024-09-21T06:34:55.721753+0000","last_became_active":"2024-09-21T06:34:55.713658+0000","last_became_peered":"2024-09-21T06:34:55.713658+0000","last_unstale":"2024-09-21T06:34:55.721753+0000","last_undegraded":"2024-09-21T06:34:55.721753+0000","last_fullsized":"2024-09-21T06:34:55.721753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:07.310461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897057+0000","last_change":"2024-09-21T06:34:55.718692+0000","last_active":"2024-09-21T06:34:56.897057+0000","last_peered":"2024-09-21T06:34:56.897057+0000","last_clean":"2024-09-21T06:34:56.897057+0000","last_became_active":"2024-09-21T06:34:55.718323+0000","last_became_peered":"2024-09-21T06:34:55.718323+0000","last_unstale":"2024-09-21T06:34:56.897057+0000","last_undegraded":"2024-09-21T06:34:56.897057+0000","last_fullsized":"2024-09-21T06:34:56.897057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:21.216504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.715947+0000","last_change":"2024-09-21T06:34:55.711836+0000","last_active":"2024-09-21T06:34:55.715947+0000","last_peered":"2024-09-21T06:34:55.715947+0000","last_clean":"2024-09-21T06:34:55.715947+0000","last_became_active":"2024-09-21T06:34:55.711556+0000","last_became_peered":"2024-09-21T06:34:55.711556+0000","last_unstale":"2024-09-21T06:34:55.715947+0000","last_undegraded":"2024-09-21T06:34:55.715947+0000","last_fullsized":"2024-09-21T06:34:55.715947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:24.575657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.690579+0000","last_change":"2024-09-21T06:34:54.690579+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.690579+0000","last_undegraded":"2024-09-21T06:34:54.690579+0000","last_fullsized":"2024-09-21T06:34:54.690579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.730527+0000","last_change":"2024-09-21T06:34:54.714493+0000","last_active":"2024-09-21T06:34:55.730527+0000","last_peered":"2024-09-21T06:34:55.730527+0000","last_clean":"2024-09-21T06:34:55.730527+0000","last_became_active":"2024-09-21T06:34:54.714069+0000","last_became_peered":"2024-09-21T06:34:54.714069+0000","last_unstale":"2024-09-21T06:34:55.730527+0000","last_undegraded":"2024-09-21T06:34:55.730527+0000","last_fullsized":"2024-09-21T06:34:55.730527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:49.070679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692607+0000","last_change":"2024-09-21T06:34:55.731634+0000","last_active":"2024-09-21T06:34:56.692607+0000","last_peered":"2024-09-21T06:34:56.692607+0000","last_clean":"2024-09-21T06:34:56.692607+0000","last_became_active":"2024-09-21T06:34:55.731317+0000","last_became_peered":"2024-09-21T06:34:55.731317+0000","last_unstale":"2024-09-21T06:34:56.692607+0000","last_undegraded":"2024-09-21T06:34:56.692607+0000","last_fullsized":"2024-09-21T06:34:56.692607+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.714077+0000","last_change":"2024-09-21T06:34:54.710899+0000","last_active":"2024-09-21T06:34:54.714077+0000","last_peered":"2024-09-21T06:34:54.714077+0000","last_clean":"2024-09-21T06:34:54.714077+0000","last_became_active":"2024-09-21T06:34:54.710535+0000","last_became_peered":"2024-09-21T06:34:54.710535+0000","last_unstale":"2024-09-21T06:34:54.714077+0000","last_undegraded":"2024-09-21T06:34:54.714077+0000","last_fullsized":"2024-09-21T06:34:54.714077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:53.792219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691077+0000","last_change":"2024-09-21T06:34:55.707231+0000","last_active":"2024-09-21T06:34:56.691077+0000","last_peered":"2024-09-21T06:34:56.691077+0000","last_clean":"2024-09-21T06:34:56.691077+0000","last_became_active":"2024-09-21T06:34:55.706910+0000","last_became_peered":"2024-09-21T06:34:55.706910+0000","last_unstale":"2024-09-21T06:34:56.691077+0000","last_undegraded":"2024-09-21T06:34:56.691077+0000","last_fullsized":"2024-09-21T06:34:56.691077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:31.422885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691219+0000","last_change":"2024-09-21T06:34:54.704122+0000","last_active":"2024-09-21T06:34:56.691219+0000","last_peered":"2024-09-21T06:34:56.691219+0000","last_clean":"2024-09-21T06:34:56.691219+0000","last_became_active":"2024-09-21T06:34:54.703803+0000","last_became_peered":"2024-09-21T06:34:54.703803+0000","last_unstale":"2024-09-21T06:34:56.691219+0000","last_undegraded":"2024-09-21T06:34:56.691219+0000","last_fullsized":"2024-09-21T06:34:56.691219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:49.261246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.737546+0000","last_change":"2024-09-21T06:34:55.731772+0000","last_active":"2024-09-21T06:34:55.737546+0000","last_peered":"2024-09-21T06:34:55.737546+0000","last_clean":"2024-09-21T06:34:55.737546+0000","last_became_active":"2024-09-21T06:34:55.731464+0000","last_became_peered":"2024-09-21T06:34:55.731464+0000","last_unstale":"2024-09-21T06:34:55.737546+0000","last_undegraded":"2024-09-21T06:34:55.737546+0000","last_fullsized":"2024-09-21T06:34:55.737546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:41.639926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692910+0000","last_change":"2024-09-21T06:34:54.701225+0000","last_active":"2024-09-21T06:34:56.692910+0000","last_peered":"2024-09-21T06:34:56.692910+0000","last_clean":"2024-09-21T06:34:56.692910+0000","last_became_active":"2024-09-21T06:34:54.700905+0000","last_became_peered":"2024-09-21T06:34:54.700905+0000","last_unstale":"2024-09-21T06:34:56.692910+0000","last_undegraded":"2024-09-21T06:34:56.692910+0000","last_fullsized":"2024-09-21T06:34:56.692910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491690+0000","last_change":"2024-09-21T06:34:55.698335+0000","last_active":"2024-09-21T06:34:57.491690+0000","last_peered":"2024-09-21T06:34:57.491690+0000","last_clean":"2024-09-21T06:34:57.491690+0000","last_became_active":"2024-09-21T06:34:55.697982+0000","last_became_peered":"2024-09-21T06:34:55.697982+0000","last_unstale":"2024-09-21T06:34:57.491690+0000","last_undegraded":"2024-09-21T06:34:57.491690+0000","last_fullsized":"2024-09-21T06:34:57.491690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:33.941883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.732028+0000","last_change":"2024-09-21T06:34:54.714529+0000","last_active":"2024-09-21T06:34:55.732028+0000","last_peered":"2024-09-21T06:34:55.732028+0000","last_clean":"2024-09-21T06:34:55.732028+0000","last_became_active":"2024-09-21T06:34:54.714159+0000","last_became_peered":"2024-09-21T06:34:54.714159+0000","last_unstale":"2024-09-21T06:34:55.732028+0000","last_undegraded":"2024-09-21T06:34:55.732028+0000","last_fullsized":"2024-09-21T06:34:55.732028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.164452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491696+0000","last_change":"2024-09-21T06:34:55.708931+0000","last_active":"2024-09-21T06:34:57.491696+0000","last_peered":"2024-09-21T06:34:57.491696+0000","last_clean":"2024-09-21T06:34:57.491696+0000","last_became_active":"2024-09-21T06:34:55.708602+0000","last_became_peered":"2024-09-21T06:34:55.708602+0000","last_unstale":"2024-09-21T06:34:57.491696+0000","last_undegraded":"2024-09-21T06:34:57.491696+0000","last_fullsized":"2024-09-21T06:34:57.491696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:53.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:54.709657+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:54.709345+0000","last_became_peered":"2024-09-21T06:34:54.709345+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.714451+0000","last_change":"2024-09-21T06:34:55.709527+0000","last_active":"2024-09-21T06:34:55.714451+0000","last_peered":"2024-09-21T06:34:55.714451+0000","last_clean":"2024-09-21T06:34:55.714451+0000","last_became_active":"2024-09-21T06:34:55.709199+0000","last_became_peered":"2024-09-21T06:34:55.709199+0000","last_unstale":"2024-09-21T06:34:55.714451+0000","last_undegraded":"2024-09-21T06:34:55.714451+0000","last_fullsized":"2024-09-21T06:34:55.714451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:07.792379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682816+0000","last_change":"2024-09-21T06:34:54.708914+0000","last_active":"2024-09-21T06:34:57.682816+0000","last_peered":"2024-09-21T06:34:57.682816+0000","last_clean":"2024-09-21T06:34:57.682816+0000","last_became_active":"2024-09-21T06:34:54.706923+0000","last_became_peered":"2024-09-21T06:34:54.706923+0000","last_unstale":"2024-09-21T06:34:57.682816+0000","last_undegraded":"2024-09-21T06:34:57.682816+0000","last_fullsized":"2024-09-21T06:34:57.682816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491887+0000","last_change":"2024-09-21T06:34:55.710127+0000","last_active":"2024-09-21T06:34:57.491887+0000","last_peered":"2024-09-21T06:34:57.491887+0000","last_clean":"2024-09-21T06:34:57.491887+0000","last_became_active":"2024-09-21T06:34:55.709577+0000","last_became_peered":"2024-09-21T06:34:55.709577+0000","last_unstale":"2024-09-21T06:34:57.491887+0000","last_undegraded":"2024-09-21T06:34:57.491887+0000","last_fullsized":"2024-09-21T06:34:57.491887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.912010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.702796+0000","last_change":"2024-09-21T06:34:54.711684+0000","last_active":"2024-09-21T06:34:55.702796+0000","last_peered":"2024-09-21T06:34:55.702796+0000","last_clean":"2024-09-21T06:34:55.702796+0000","last_became_active":"2024-09-21T06:34:54.711307+0000","last_became_peered":"2024-09-21T06:34:54.711307+0000","last_unstale":"2024-09-21T06:34:55.702796+0000","last_undegraded":"2024-09-21T06:34:55.702796+0000","last_fullsized":"2024-09-21T06:34:55.702796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:08.787586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490933+0000","last_change":"2024-09-21T06:34:55.718040+0000","last_active":"2024-09-21T06:34:57.490933+0000","last_peered":"2024-09-21T06:34:57.490933+0000","last_clean":"2024-09-21T06:34:57.490933+0000","last_became_active":"2024-09-21T06:34:55.717355+0000","last_became_peered":"2024-09-21T06:34:55.717355+0000","last_unstale":"2024-09-21T06:34:57.490933+0000","last_undegraded":"2024-09-21T06:34:57.490933+0000","last_fullsized":"2024-09-21T06:34:57.490933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897542+0000","last_change":"2024-09-21T06:34:54.709082+0000","last_active":"2024-09-21T06:34:56.897542+0000","last_peered":"2024-09-21T06:34:56.897542+0000","last_clean":"2024-09-21T06:34:56.897542+0000","last_became_active":"2024-09-21T06:34:54.708646+0000","last_became_peered":"2024-09-21T06:34:54.708646+0000","last_unstale":"2024-09-21T06:34:56.897542+0000","last_undegraded":"2024-09-21T06:34:56.897542+0000","last_fullsized":"2024-09-21T06:34:56.897542+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:59.547312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490987+0000","last_change":"2024-09-21T06:34:55.706722+0000","last_active":"2024-09-21T06:34:57.490987+0000","last_peered":"2024-09-21T06:34:57.490987+0000","last_clean":"2024-09-21T06:34:57.490987+0000","last_became_active":"2024-09-21T06:34:55.706428+0000","last_became_peered":"2024-09-21T06:34:55.706428+0000","last_unstale":"2024-09-21T06:34:57.490987+0000","last_undegraded":"2024-09-21T06:34:57.490987+0000","last_fullsized":"2024-09-21T06:34:57.490987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:17.159683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693179+0000","last_change":"2024-09-21T06:34:54.700087+0000","last_active":"2024-09-21T06:34:56.693179+0000","last_peered":"2024-09-21T06:34:56.693179+0000","last_clean":"2024-09-21T06:34:56.693179+0000","last_became_active":"2024-09-21T06:34:54.699715+0000","last_became_peered":"2024-09-21T06:34:54.699715+0000","last_unstale":"2024-09-21T06:34:56.693179+0000","last_undegraded":"2024-09-21T06:34:56.693179+0000","last_fullsized":"2024-09-21T06:34:56.693179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693783+0000","last_change":"2024-09-21T06:34:55.714419+0000","last_active":"2024-09-21T06:34:56.693783+0000","last_peered":"2024-09-21T06:34:56.693783+0000","last_clean":"2024-09-21T06:34:56.693783+0000","last_became_active":"2024-09-21T06:34:55.714040+0000","last_became_peered":"2024-09-21T06:34:55.714040+0000","last_unstale":"2024-09-21T06:34:56.693783+0000","last_undegraded":"2024-09-21T06:34:56.693783+0000","last_fullsized":"2024-09-21T06:34:56.693783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.757308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897468+0000","last_change":"2024-09-21T06:34:54.708697+0000","last_active":"2024-09-21T06:34:56.897468+0000","last_peered":"2024-09-21T06:34:56.897468+0000","last_clean":"2024-09-21T06:34:56.897468+0000","last_became_active":"2024-09-21T06:34:54.708338+0000","last_became_peered":"2024-09-21T06:34:54.708338+0000","last_unstale":"2024-09-21T06:34:56.897468+0000","last_undegraded":"2024-09-21T06:34:56.897468+0000","last_fullsized":"2024-09-21T06:34:56.897468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:29.524412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.733897+0000","last_change":"2024-09-21T06:34:55.732505+0000","last_active":"2024-09-21T06:34:55.733897+0000","last_peered":"2024-09-21T06:34:55.733897+0000","last_clean":"2024-09-21T06:34:55.733897+0000","last_became_active":"2024-09-21T06:34:55.732209+0000","last_became_peered":"2024-09-21T06:34:55.732209+0000","last_unstale":"2024-09-21T06:34:55.733897+0000","last_undegraded":"2024-09-21T06:34:55.733897+0000","last_fullsized":"2024-09-21T06:34:55.733897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:36.296181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.740620+0000","last_change":"2024-09-21T06:34:54.704500+0000","last_active":"2024-09-21T06:34:55.740620+0000","last_peered":"2024-09-21T06:34:55.740620+0000","last_clean":"2024-09-21T06:34:55.740620+0000","last_became_active":"2024-09-21T06:34:54.704165+0000","last_became_peered":"2024-09-21T06:34:54.704165+0000","last_unstale":"2024-09-21T06:34:55.740620+0000","last_undegraded":"2024-09-21T06:34:55.740620+0000","last_fullsized":"2024-09-21T06:34:55.740620+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.649075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690939+0000","last_change":"2024-09-21T06:34:55.705237+0000","last_active":"2024-09-21T06:34:56.690939+0000","last_peered":"2024-09-21T06:34:56.690939+0000","last_clean":"2024-09-21T06:34:56.690939+0000","last_became_active":"2024-09-21T06:34:55.704951+0000","last_became_peered":"2024-09-21T06:34:55.704951+0000","last_unstale":"2024-09-21T06:34:56.690939+0000","last_undegraded":"2024-09-21T06:34:56.690939+0000","last_fullsized":"2024-09-21T06:34:56.690939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491461+0000","last_change":"2024-09-21T06:34:54.709537+0000","last_active":"2024-09-21T06:34:57.491461+0000","last_peered":"2024-09-21T06:34:57.491461+0000","last_clean":"2024-09-21T06:34:57.491461+0000","last_became_active":"2024-09-21T06:34:54.706229+0000","last_became_peered":"2024-09-21T06:34:54.706229+0000","last_unstale":"2024-09-21T06:34:57.491461+0000","last_undegraded":"2024-09-21T06:34:57.491461+0000","last_fullsized":"2024-09-21T06:34:57.491461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:57.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691227+0000","last_change":"2024-09-21T06:34:55.699196+0000","last_active":"2024-09-21T06:34:56.691227+0000","last_peered":"2024-09-21T06:34:56.691227+0000","last_clean":"2024-09-21T06:34:56.691227+0000","last_became_active":"2024-09-21T06:34:55.698813+0000","last_became_peered":"2024-09-21T06:34:55.698813+0000","last_unstale":"2024-09-21T06:34:56.691227+0000","last_undegraded":"2024-09-21T06:34:56.691227+0000","last_fullsized":"2024-09-21T06:34:56.691227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:46.110588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693829+0000","last_change":"2024-09-21T06:34:54.701370+0000","last_active":"2024-09-21T06:34:56.693829+0000","last_peered":"2024-09-21T06:34:56.693829+0000","last_clean":"2024-09-21T06:34:56.693829+0000","last_became_active":"2024-09-21T06:34:54.700999+0000","last_became_peered":"2024-09-21T06:34:54.700999+0000","last_unstale":"2024-09-21T06:34:56.693829+0000","last_undegraded":"2024-09-21T06:34:56.693829+0000","last_fullsized":"2024-09-21T06:34:56.693829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731581+0000","last_change":"2024-09-21T06:34:55.728639+0000","last_active":"2024-09-21T06:34:55.731581+0000","last_peered":"2024-09-21T06:34:55.731581+0000","last_clean":"2024-09-21T06:34:55.731581+0000","last_became_active":"2024-09-21T06:34:55.728220+0000","last_became_peered":"2024-09-21T06:34:55.728220+0000","last_unstale":"2024-09-21T06:34:55.731581+0000","last_undegraded":"2024-09-21T06:34:55.731581+0000","last_fullsized":"2024-09-21T06:34:55.731581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.666244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490873+0000","last_change":"2024-09-21T06:34:54.708770+0000","last_active":"2024-09-21T06:34:57.490873+0000","last_peered":"2024-09-21T06:34:57.490873+0000","last_clean":"2024-09-21T06:34:57.490873+0000","last_became_active":"2024-09-21T06:34:54.708363+0000","last_became_peered":"2024-09-21T06:34:54.708363+0000","last_unstale":"2024-09-21T06:34:57.490873+0000","last_undegraded":"2024-09-21T06:34:57.490873+0000","last_fullsized":"2024-09-21T06:34:57.490873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:08.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-21T06:34:54.692465+0000","last_change":"2024-09-21T06:34:54.692465+0000","last_active":"2024-09-21T06:34:54.676087+0000","last_peered":"2024-09-21T06:34:54.676087+0000","last_clean":"2024-09-21T06:34:54.676087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:34:54.692465+0000","last_undegraded":"2024-09-21T06:34:54.692465+0000","last_fullsized":"2024-09-21T06:34:54.692465+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.734554+0000","last_change":"2024-09-21T06:34:54.713801+0000","last_active":"2024-09-21T06:34:55.734554+0000","last_peered":"2024-09-21T06:34:55.734554+0000","last_clean":"2024-09-21T06:34:55.734554+0000","last_became_active":"2024-09-21T06:34:54.712759+0000","last_became_peered":"2024-09-21T06:34:54.712759+0000","last_unstale":"2024-09-21T06:34:55.734554+0000","last_undegraded":"2024-09-21T06:34:55.734554+0000","last_fullsized":"2024-09-21T06:34:55.734554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:47.339290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692225+0000","last_change":"2024-09-21T06:34:55.732582+0000","last_active":"2024-09-21T06:34:56.692225+0000","last_peered":"2024-09-21T06:34:56.692225+0000","last_clean":"2024-09-21T06:34:56.692225+0000","last_became_active":"2024-09-21T06:34:55.732328+0000","last_became_peered":"2024-09-21T06:34:55.732328+0000","last_unstale":"2024-09-21T06:34:56.692225+0000","last_undegraded":"2024-09-21T06:34:56.692225+0000","last_fullsized":"2024-09-21T06:34:56.692225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897574+0000","last_change":"2024-09-21T06:34:54.710621+0000","last_active":"2024-09-21T06:34:56.897574+0000","last_peered":"2024-09-21T06:34:56.897574+0000","last_clean":"2024-09-21T06:34:56.897574+0000","last_became_active":"2024-09-21T06:34:54.710305+0000","last_became_peered":"2024-09-21T06:34:54.710305+0000","last_unstale":"2024-09-21T06:34:56.897574+0000","last_undegraded":"2024-09-21T06:34:56.897574+0000","last_fullsized":"2024-09-21T06:34:56.897574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:04.681733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683086+0000","last_change":"2024-09-21T06:34:55.716269+0000","last_active":"2024-09-21T06:34:57.683086+0000","last_peered":"2024-09-21T06:34:57.683086+0000","last_clean":"2024-09-21T06:34:57.683086+0000","last_became_active":"2024-09-21T06:34:55.706408+0000","last_became_peered":"2024-09-21T06:34:55.706408+0000","last_unstale":"2024-09-21T06:34:57.683086+0000","last_undegraded":"2024-09-21T06:34:57.683086+0000","last_fullsized":"2024-09-21T06:34:57.683086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683117+0000","last_change":"2024-09-21T06:34:54.712959+0000","last_active":"2024-09-21T06:34:57.683117+0000","last_peered":"2024-09-21T06:34:57.683117+0000","last_clean":"2024-09-21T06:34:57.683117+0000","last_became_active":"2024-09-21T06:34:54.711866+0000","last_became_peered":"2024-09-21T06:34:54.711866+0000","last_unstale":"2024-09-21T06:34:57.683117+0000","last_undegraded":"2024-09-21T06:34:57.683117+0000","last_fullsized":"2024-09-21T06:34:57.683117+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731728+0000","last_change":"2024-09-21T06:34:55.728437+0000","last_active":"2024-09-21T06:34:55.731728+0000","last_peered":"2024-09-21T06:34:55.731728+0000","last_clean":"2024-09-21T06:34:55.731728+0000","last_became_active":"2024-09-21T06:34:55.728111+0000","last_became_peered":"2024-09-21T06:34:55.728111+0000","last_unstale":"2024-09-21T06:34:55.731728+0000","last_undegraded":"2024-09-21T06:34:55.731728+0000","last_fullsized":"2024-09-21T06:34:55.731728+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:21.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897340+0000","last_change":"2024-09-21T06:34:54.710224+0000","last_active":"2024-09-21T06:34:56.897340+0000","last_peered":"2024-09-21T06:34:56.897340+0000","last_clean":"2024-09-21T06:34:56.897340+0000","last_became_active":"2024-09-21T06:34:54.709896+0000","last_became_peered":"2024-09-21T06:34:54.709896+0000","last_unstale":"2024-09-21T06:34:56.897340+0000","last_undegraded":"2024-09-21T06:34:56.897340+0000","last_fullsized":"2024-09-21T06:34:56.897340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:16.511663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491563+0000","last_change":"2024-09-21T06:34:55.708497+0000","last_active":"2024-09-21T06:34:57.491563+0000","last_peered":"2024-09-21T06:34:57.491563+0000","last_clean":"2024-09-21T06:34:57.491563+0000","last_became_active":"2024-09-21T06:34:55.706136+0000","last_became_peered":"2024-09-21T06:34:55.706136+0000","last_unstale":"2024-09-21T06:34:57.491563+0000","last_undegraded":"2024-09-21T06:34:57.491563+0000","last_fullsized":"2024-09-21T06:34:57.491563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:06.652871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713997+0000","last_change":"2024-09-21T06:34:54.712196+0000","last_active":"2024-09-21T06:34:54.713997+0000","last_peered":"2024-09-21T06:34:54.713997+0000","last_clean":"2024-09-21T06:34:54.713997+0000","last_became_active":"2024-09-21T06:34:54.711507+0000","last_became_peered":"2024-09-21T06:34:54.711507+0000","last_unstale":"2024-09-21T06:34:54.713997+0000","last_undegraded":"2024-09-21T06:34:54.713997+0000","last_fullsized":"2024-09-21T06:34:54.713997+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:41.618052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.725352+0000","last_change":"2024-09-21T06:34:55.718899+0000","last_active":"2024-09-21T06:34:55.725352+0000","last_peered":"2024-09-21T06:34:55.725352+0000","last_clean":"2024-09-21T06:34:55.725352+0000","last_became_active":"2024-09-21T06:34:55.718610+0000","last_became_peered":"2024-09-21T06:34:55.718610+0000","last_unstale":"2024-09-21T06:34:55.725352+0000","last_undegraded":"2024-09-21T06:34:55.725352+0000","last_fullsized":"2024-09-21T06:34:55.725352+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:14.255062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707453+0000","last_change":"2024-09-21T06:34:54.704910+0000","last_active":"2024-09-21T06:34:54.707453+0000","last_peered":"2024-09-21T06:34:54.707453+0000","last_clean":"2024-09-21T06:34:54.707453+0000","last_became_active":"2024-09-21T06:34:54.703410+0000","last_became_peered":"2024-09-21T06:34:54.703410+0000","last_unstale":"2024-09-21T06:34:54.707453+0000","last_undegraded":"2024-09-21T06:34:54.707453+0000","last_fullsized":"2024-09-21T06:34:54.707453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:42.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731552+0000","last_change":"2024-09-21T06:34:55.726247+0000","last_active":"2024-09-21T06:34:55.731552+0000","last_peered":"2024-09-21T06:34:55.731552+0000","last_clean":"2024-09-21T06:34:55.731552+0000","last_became_active":"2024-09-21T06:34:55.725850+0000","last_became_peered":"2024-09-21T06:34:55.725850+0000","last_unstale":"2024-09-21T06:34:55.731552+0000","last_undegraded":"2024-09-21T06:34:55.731552+0000","last_fullsized":"2024-09-21T06:34:55.731552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:40.564061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897179+0000","last_change":"2024-09-21T06:34:54.712284+0000","last_active":"2024-09-21T06:34:56.897179+0000","last_peered":"2024-09-21T06:34:56.897179+0000","last_clean":"2024-09-21T06:34:56.897179+0000","last_became_active":"2024-09-21T06:34:54.711828+0000","last_became_peered":"2024-09-21T06:34:54.711828+0000","last_unstale":"2024-09-21T06:34:56.897179+0000","last_undegraded":"2024-09-21T06:34:56.897179+0000","last_fullsized":"2024-09-21T06:34:56.897179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:29.075613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491685+0000","last_change":"2024-09-21T06:34:55.713268+0000","last_active":"2024-09-21T06:34:57.491685+0000","last_peered":"2024-09-21T06:34:57.491685+0000","last_clean":"2024-09-21T06:34:57.491685+0000","last_became_active":"2024-09-21T06:34:55.712940+0000","last_became_peered":"2024-09-21T06:34:55.712940+0000","last_unstale":"2024-09-21T06:34:57.491685+0000","last_undegraded":"2024-09-21T06:34:57.491685+0000","last_fullsized":"2024-09-21T06:34:57.491685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:20.633620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692250+0000","last_change":"2024-09-21T06:34:54.706724+0000","last_active":"2024-09-21T06:34:56.692250+0000","last_peered":"2024-09-21T06:34:56.692250+0000","last_clean":"2024-09-21T06:34:56.692250+0000","last_became_active":"2024-09-21T06:34:54.706045+0000","last_became_peered":"2024-09-21T06:34:54.706045+0000","last_unstale":"2024-09-21T06:34:56.692250+0000","last_undegraded":"2024-09-21T06:34:56.692250+0000","last_fullsized":"2024-09-21T06:34:56.692250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.717407+0000","last_change":"2024-09-21T06:34:55.716497+0000","last_active":"2024-09-21T06:34:55.717407+0000","last_peered":"2024-09-21T06:34:55.717407+0000","last_clean":"2024-09-21T06:34:55.717407+0000","last_became_active":"2024-09-21T06:34:55.716182+0000","last_became_peered":"2024-09-21T06:34:55.716182+0000","last_unstale":"2024-09-21T06:34:55.717407+0000","last_undegraded":"2024-09-21T06:34:55.717407+0000","last_fullsized":"2024-09-21T06:34:55.717407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:13.835969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.704018+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.703663+0000","last_became_peered":"2024-09-21T06:34:54.703663+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:45.237205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721388+0000","last_change":"2024-09-21T06:34:55.712873+0000","last_active":"2024-09-21T06:34:55.721388+0000","last_peered":"2024-09-21T06:34:55.721388+0000","last_clean":"2024-09-21T06:34:55.721388+0000","last_became_active":"2024-09-21T06:34:55.711695+0000","last_became_peered":"2024-09-21T06:34:55.711695+0000","last_unstale":"2024-09-21T06:34:55.721388+0000","last_undegraded":"2024-09-21T06:34:55.721388+0000","last_fullsized":"2024-09-21T06:34:55.721388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:31.987545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.713460+0000","last_change":"2024-09-21T06:34:54.709868+0000","last_active":"2024-09-21T06:34:54.713460+0000","last_peered":"2024-09-21T06:34:54.713460+0000","last_clean":"2024-09-21T06:34:54.713460+0000","last_became_active":"2024-09-21T06:34:54.708447+0000","last_became_peered":"2024-09-21T06:34:54.708447+0000","last_unstale":"2024-09-21T06:34:54.713460+0000","last_undegraded":"2024-09-21T06:34:54.713460+0000","last_fullsized":"2024-09-21T06:34:54.713460+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:13.936837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693744+0000","last_change":"2024-09-21T06:34:55.729064+0000","last_active":"2024-09-21T06:34:56.693744+0000","last_peered":"2024-09-21T06:34:56.693744+0000","last_clean":"2024-09-21T06:34:56.693744+0000","last_became_active":"2024-09-21T06:34:55.728795+0000","last_became_peered":"2024-09-21T06:34:55.728795+0000","last_unstale":"2024-09-21T06:34:56.693744+0000","last_undegraded":"2024-09-21T06:34:56.693744+0000","last_fullsized":"2024-09-21T06:34:56.693744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:38.265711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.703782+0000","last_change":"2024-09-21T06:34:54.702424+0000","last_active":"2024-09-21T06:34:55.703782+0000","last_peered":"2024-09-21T06:34:55.703782+0000","last_clean":"2024-09-21T06:34:55.703782+0000","last_became_active":"2024-09-21T06:34:54.702065+0000","last_became_peered":"2024-09-21T06:34:54.702065+0000","last_unstale":"2024-09-21T06:34:55.703782+0000","last_undegraded":"2024-09-21T06:34:55.703782+0000","last_fullsized":"2024-09-21T06:34:55.703782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:51.535195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683128+0000","last_change":"2024-09-21T06:34:55.718743+0000","last_active":"2024-09-21T06:34:57.683128+0000","last_peered":"2024-09-21T06:34:57.683128+0000","last_clean":"2024-09-21T06:34:57.683128+0000","last_became_active":"2024-09-21T06:34:55.703205+0000","last_became_peered":"2024-09-21T06:34:55.703205+0000","last_unstale":"2024-09-21T06:34:57.683128+0000","last_undegraded":"2024-09-21T06:34:57.683128+0000","last_fullsized":"2024-09-21T06:34:57.683128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491730+0000","last_change":"2024-09-21T06:34:54.711548+0000","last_active":"2024-09-21T06:34:57.491730+0000","last_peered":"2024-09-21T06:34:57.491730+0000","last_clean":"2024-09-21T06:34:57.491730+0000","last_became_active":"2024-09-21T06:34:54.711141+0000","last_became_peered":"2024-09-21T06:34:54.711141+0000","last_unstale":"2024-09-21T06:34:57.491730+0000","last_undegraded":"2024-09-21T06:34:57.491730+0000","last_fullsized":"2024-09-21T06:34:57.491730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:27.210366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491185+0000","last_change":"2024-09-21T06:34:55.712582+0000","last_active":"2024-09-21T06:34:57.491185+0000","last_peered":"2024-09-21T06:34:57.491185+0000","last_clean":"2024-09-21T06:34:57.491185+0000","last_became_active":"2024-09-21T06:34:55.710833+0000","last_became_peered":"2024-09-21T06:34:55.710833+0000","last_unstale":"2024-09-21T06:34:57.491185+0000","last_undegraded":"2024-09-21T06:34:57.491185+0000","last_fullsized":"2024-09-21T06:34:57.491185+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.706570+0000","last_change":"2024-09-21T06:34:54.713168+0000","last_active":"2024-09-21T06:34:55.706570+0000","last_peered":"2024-09-21T06:34:55.706570+0000","last_clean":"2024-09-21T06:34:55.706570+0000","last_became_active":"2024-09-21T06:34:54.712837+0000","last_became_peered":"2024-09-21T06:34:54.712837+0000","last_unstale":"2024-09-21T06:34:55.706570+0000","last_undegraded":"2024-09-21T06:34:55.706570+0000","last_fullsized":"2024-09-21T06:34:55.706570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:29.731794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491694+0000","last_change":"2024-09-21T06:34:55.711645+0000","last_active":"2024-09-21T06:34:57.491694+0000","last_peered":"2024-09-21T06:34:57.491694+0000","last_clean":"2024-09-21T06:34:57.491694+0000","last_became_active":"2024-09-21T06:34:55.711312+0000","last_became_peered":"2024-09-21T06:34:55.711312+0000","last_unstale":"2024-09-21T06:34:57.491694+0000","last_undegraded":"2024-09-21T06:34:57.491694+0000","last_fullsized":"2024-09-21T06:34:57.491694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707480+0000","last_change":"2024-09-21T06:34:54.705515+0000","last_active":"2024-09-21T06:34:54.707480+0000","last_peered":"2024-09-21T06:34:54.707480+0000","last_clean":"2024-09-21T06:34:54.707480+0000","last_became_active":"2024-09-21T06:34:54.705153+0000","last_became_peered":"2024-09-21T06:34:54.705153+0000","last_unstale":"2024-09-21T06:34:54.707480+0000","last_undegraded":"2024-09-21T06:34:54.707480+0000","last_fullsized":"2024-09-21T06:34:54.707480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:38.546287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897683+0000","last_change":"2024-09-21T06:34:55.729077+0000","last_active":"2024-09-21T06:34:56.897683+0000","last_peered":"2024-09-21T06:34:56.897683+0000","last_clean":"2024-09-21T06:34:56.897683+0000","last_became_active":"2024-09-21T06:34:55.728776+0000","last_became_peered":"2024-09-21T06:34:55.728776+0000","last_unstale":"2024-09-21T06:34:56.897683+0000","last_undegraded":"2024-09-21T06:34:56.897683+0000","last_fullsized":"2024-09-21T06:34:56.897683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683192+0000","last_change":"2024-09-21T06:34:54.703341+0000","last_active":"2024-09-21T06:34:57.683192+0000","last_peered":"2024-09-21T06:34:57.683192+0000","last_clean":"2024-09-21T06:34:57.683192+0000","last_became_active":"2024-09-21T06:34:54.703002+0000","last_became_peered":"2024-09-21T06:34:54.703002+0000","last_unstale":"2024-09-21T06:34:57.683192+0000","last_undegraded":"2024-09-21T06:34:57.683192+0000","last_fullsized":"2024-09-21T06:34:57.683192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691146+0000","last_change":"2024-09-21T06:34:55.737366+0000","last_active":"2024-09-21T06:34:56.691146+0000","last_peered":"2024-09-21T06:34:56.691146+0000","last_clean":"2024-09-21T06:34:56.691146+0000","last_became_active":"2024-09-21T06:34:55.734444+0000","last_became_peered":"2024-09-21T06:34:55.734444+0000","last_unstale":"2024-09-21T06:34:56.691146+0000","last_undegraded":"2024-09-21T06:34:56.691146+0000","last_fullsized":"2024-09-21T06:34:56.691146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:47.619903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T06:34:54.707339+0000","last_change":"2024-09-21T06:34:54.702747+0000","last_active":"2024-09-21T06:34:54.707339+0000","last_peered":"2024-09-21T06:34:54.707339+0000","last_clean":"2024-09-21T06:34:54.707339+0000","last_became_active":"2024-09-21T06:34:54.702221+0000","last_became_peered":"2024-09-21T06:34:54.702221+0000","last_unstale":"2024-09-21T06:34:54.707339+0000","last_undegraded":"2024-09-21T06:34:54.707339+0000","last_fullsized":"2024-09-21T06:34:54.707339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:50.889270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897188+0000","last_change":"2024-09-21T06:34:55.724899+0000","last_active":"2024-09-21T06:34:56.897188+0000","last_peered":"2024-09-21T06:34:56.897188+0000","last_clean":"2024-09-21T06:34:56.897188+0000","last_became_active":"2024-09-21T06:34:55.724167+0000","last_became_peered":"2024-09-21T06:34:55.724167+0000","last_unstale":"2024-09-21T06:34:56.897188+0000","last_undegraded":"2024-09-21T06:34:56.897188+0000","last_fullsized":"2024-09-21T06:34:56.897188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491794+0000","last_change":"2024-09-21T06:34:54.707367+0000","last_active":"2024-09-21T06:34:57.491794+0000","last_peered":"2024-09-21T06:34:57.491794+0000","last_clean":"2024-09-21T06:34:57.491794+0000","last_became_active":"2024-09-21T06:34:54.705863+0000","last_became_peered":"2024-09-21T06:34:54.705863+0000","last_unstale":"2024-09-21T06:34:57.491794+0000","last_undegraded":"2024-09-21T06:34:57.491794+0000","last_fullsized":"2024-09-21T06:34:57.491794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.687054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682836+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:34:57.682836+0000","last_peered":"2024-09-21T06:34:57.682836+0000","last_clean":"2024-09-21T06:34:57.682836+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:34:57.682836+0000","last_undegraded":"2024-09-21T06:34:57.682836+0000","last_fullsized":"2024-09-21T06:34:57.682836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1277952,"data_stored":589850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509529,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":595924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1282048,"data_stored":590810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":1179658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":596472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1839104,"data_stored":1165732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:01.199 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3d is not active+clean 2024-09-21T06:35:01.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:34:54.689595+0000', 'last_change': '2024-09-21T06:34:54.689595+0000', 'last_active': '2024-09-21T06:34:54.676087+0000', 'last_peered': '2024-09-21T06:34:54.676087+0000', 'last_clean': '2024-09-21T06:34:54.676087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:34:54.689595+0000', 'last_undegraded': '2024-09-21T06:34:54.689595+0000', 'last_fullsized': '2024-09-21T06:34:54.689595+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T06:35:01.199 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2f is not active+clean 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:34:54.693106+0000', 'last_change': '2024-09-21T06:34:54.693106+0000', 'last_active': '2024-09-21T06:34:54.676087+0000', 'last_peered': '2024-09-21T06:34:54.676087+0000', 'last_clean': '2024-09-21T06:34:54.676087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:34:54.693106+0000', 'last_undegraded': '2024-09-21T06:34:54.693106+0000', 'last_fullsized': '2024-09-21T06:34:54.693106+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2b is not active+clean 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:34:54.689114+0000', 'last_change': '2024-09-21T06:34:54.689114+0000', 'last_active': '2024-09-21T06:34:54.676087+0000', 'last_peered': '2024-09-21T06:34:54.676087+0000', 'last_clean': '2024-09-21T06:34:54.676087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:34:54.689114+0000', 'last_undegraded': '2024-09-21T06:34:54.689114+0000', 'last_fullsized': '2024-09-21T06:34:54.689114+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:PG 16.0 is not active+clean 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:34:54.690579+0000', 'last_change': '2024-09-21T06:34:54.690579+0000', 'last_active': '2024-09-21T06:34:54.676087+0000', 'last_peered': '2024-09-21T06:34:54.676087+0000', 'last_clean': '2024-09-21T06:34:54.676087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:34:54.690579+0000', 'last_undegraded': '2024-09-21T06:34:54.690579+0000', 'last_fullsized': '2024-09-21T06:34:54.690579+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:PG 16.f is not active+clean 2024-09-21T06:35:01.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:34:54.692465+0000', 'last_change': '2024-09-21T06:34:54.692465+0000', 'last_active': '2024-09-21T06:34:54.676087+0000', 'last_peered': '2024-09-21T06:34:54.676087+0000', 'last_clean': '2024-09-21T06:34:54.676087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:34:54.692465+0000', 'last_undegraded': '2024-09-21T06:34:54.692465+0000', 'last_fullsized': '2024-09-21T06:34:54.692465+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:54.676087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T06:35:01.201 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 -- 172.21.15.37:0/1079475014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a415cb20 msgr2=0x7f31a40cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 --2- 172.21.15.37:0/1079475014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a415cb20 0x7f31a40cddc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f3194009830 tx=0x7f319402a510 comp rx=0 tx=0).stop 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 -- 172.21.15.37:0/1079475014 shutdown_connections 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 --2- 172.21.15.37:0/1079475014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a40cc8d0 0x7f31a40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 --2- 172.21.15.37:0/1079475014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a415cb20 0x7f31a40cddc0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 -- 172.21.15.37:0/1079475014 >> 172.21.15.37:0/1079475014 conn(0x7f31a406f940 msgr2=0x7f31a40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 -- 172.21.15.37:0/1079475014 shutdown_connections 2024-09-21T06:35:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.459+0000 7f31a3e00640 1 -- 172.21.15.37:0/1079475014 wait complete. 2024-09-21T06:35:01.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.462+0000 7f31a3e00640 1 Processor -- start 2024-09-21T06:35:01.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.462+0000 7f31a3e00640 1 -- start start 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a40b4990 0x7f31a4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a415cb20 0x7f31a415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31a406d600 con 0x7f31a40cc8d0 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31a406d480 con 0x7f31a40b4990 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f31a3e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f31a406d780 con 0x7f31a415cb20 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.463+0000 7f319a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38314/0 (socket says 172.21.15.37:38314) 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 -- 172.21.15.37:0/4149129138 learned_addr learned my addr 172.21.15.37:0/4149129138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:01.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f31a2a00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a40b4990 0x7f31a4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f31a3400640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a415cb20 0x7f31a415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 -- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a415cb20 msgr2=0x7f31a415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a415cb20 0x7f31a415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 -- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a40b4990 msgr2=0x7f31a4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a40b4990 0x7f31a4159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 -- 172.21.15.37:0/4149129138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a4143f00 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319a000640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ea20 tx=0x7f318c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00cde0 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.464+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f318c011560 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.465+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c011990 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.465+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a4158ce0 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.465+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41592a0 con 0x7f31a40cc8d0 2024-09-21T06:35:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.466+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a406dc30 con 0x7f31a40cc8d0 2024-09-21T06:35:01.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.467+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f318c011b30 con 0x7f31a40cc8d0 2024-09-21T06:35:01.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.468+0000 7f319b400640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 0x7f318007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:01.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.468+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f318c09bb30 con 0x7f31a40cc8d0 2024-09-21T06:35:01.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.468+0000 7f31a2a00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 0x7f318007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:01.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.469+0000 7f31a2a00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 0x7f318007b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3194034000 tx=0x7f3194004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:01.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.473+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f318c016030 con 0x7f31a40cc8d0 2024-09-21T06:35:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.746+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f31a415f9a0 con 0x7f31a40cc8d0 2024-09-21T06:35:01.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.747+0000 7f319b400640 1 -- 172.21.15.37:0/4149129138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1477 (secure 0 0 0) 0x7f318c011e20 con 0x7f31a40cc8d0 2024-09-21T06:35:01.748 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:01.748 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":266993664,"bytes_avail":772827119616,"bytes_total":773094113280,"write_bytes_sec":2198,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":108,"btime":"2024-09-21T06:34:58:717858+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-09-21T06:34:45.201307+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 msgr2=0x7f318007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 0x7f318007b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3194034000 tx=0x7f3194004730 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 msgr2=0x7f31a4143060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ea20 tx=0x7f318c00eee0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 shutdown_connections 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3180078b90 0x7f318007b0a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31a40cc8d0 0x7f31a4143060 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a415cb20 0x7f31a415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.749+0000 7f31a3e00640 1 --2- 172.21.15.37:0/4149129138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a40b4990 0x7f31a4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.750+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 >> 172.21.15.37:0/4149129138 conn(0x7f31a406f940 msgr2=0x7f31a40b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:01.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.750+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 shutdown_connections 2024-09-21T06:35:01.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:01.750+0000 7f31a3e00640 1 -- 172.21.15.37:0/4149129138 wait complete. 2024-09-21T06:35:01.775 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T06:35:01.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:35:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 -- 172.21.15.37:0/2758151577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594152530 msgr2=0x7f85940b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 --2- 172.21.15.37:0/2758151577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594152530 0x7f85940b3490 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f85940cef10 tx=0x7f858002a5b0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 -- 172.21.15.37:0/2758151577 shutdown_connections 2024-09-21T06:35:02.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 --2- 172.21.15.37:0/2758151577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594152530 0x7f85940b3490 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 -- 172.21.15.37:0/2758151577 >> 172.21.15.37:0/2758151577 conn(0x7f859406f940 msgr2=0x7f85940b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:02.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 -- 172.21.15.37:0/2758151577 shutdown_connections 2024-09-21T06:35:02.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.001+0000 7f8599800640 1 -- 172.21.15.37:0/2758151577 wait complete. 2024-09-21T06:35:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.003+0000 7f8599800640 1 Processor -- start 2024-09-21T06:35:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.003+0000 7f8599800640 1 -- start start 2024-09-21T06:35:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 0x7f85941427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 0x7f85940b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cf700 con 0x7f8594073dd0 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cf580 con 0x7f85940b39d0 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8599800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f85940cf880 con 0x7f8594152530 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38322/0 (socket says 172.21.15.37:38322) 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 -- 172.21.15.37:0/1433969676 learned_addr learned my addr 172.21.15.37:0/1433969676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8598e00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 0x7f85940b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593400640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 0x7f85941427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 -- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 msgr2=0x7f85940b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 0x7f85940b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 -- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 msgr2=0x7f85941427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 0x7f85941427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593e00640 1 -- 172.21.15.37:0/1433969676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859415b9a0 con 0x7f8594073dd0 2024-09-21T06:35:02.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8598e00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 0x7f85940b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:02.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.005+0000 7f8593400640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 0x7f85941427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:02.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f8593e00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f858400c400 tx=0x7f858400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584017070 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85840053c0 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859415be10 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584003850 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.006+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859415c3d0 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.007+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859406e040 con 0x7f8594073dd0 2024-09-21T06:35:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.008+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f858401a070 con 0x7f8594073dd0 2024-09-21T06:35:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.009+0000 7f858be00640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 0x7f857407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.009+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f858409a2e0 con 0x7f8594073dd0 2024-09-21T06:35:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.010+0000 7f8593400640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 0x7f857407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.011+0000 7f8593400640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 0x7f857407b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8580034000 tx=0x7f8580004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.014+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f859406e040 con 0x7f8594073dd0 2024-09-21T06:35:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.212+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85940b3490 con 0x7f8574078b90 2024-09-21T06:35:02.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.217+0000 7f858be00640 1 -- 172.21.15.37:0/1433969676 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f85940b3490 con 0x7f8574078b90 2024-09-21T06:35:02.220 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:02.222 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:35:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 msgr2=0x7f857407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 0x7f857407b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8580034000 tx=0x7f8580004730 comp rx=0 tx=0).stop 2024-09-21T06:35:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 msgr2=0x7f859414f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f858400c400 tx=0x7f858400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 shutdown_connections 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8574078b90 0x7f857407b0a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8594152530 0x7f85940b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85940b39d0 0x7f85941427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 --2- 172.21.15.37:0/1433969676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8594073dd0 0x7f859414f290 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 >> 172.21.15.37:0/1433969676 conn(0x7f859406f940 msgr2=0x7f85940b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 shutdown_connections 2024-09-21T06:35:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:02.221+0000 7f8599800640 1 -- 172.21.15.37:0/1433969676 wait complete. 2024-09-21T06:35:02.247 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-21T06:35:01.205867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260736,"kb_used_data":11204,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713984,"statfs":{"total":773094113280,"available":772827119616,"internally_reserved":0,"allocated":11472896,"data_stored":5932600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.520635"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491654+0000","last_change":"2024-09-21T06:34:55.702355+0000","last_active":"2024-09-21T06:34:57.491654+0000","last_peered":"2024-09-21T06:34:57.491654+0000","last_clean":"2024-09-21T06:34:57.491654+0000","last_became_active":"2024-09-21T06:34:55.701180+0000","last_became_peered":"2024-09-21T06:34:55.701180+0000","last_unstale":"2024-09-21T06:34:57.491654+0000","last_undegraded":"2024-09-21T06:34:57.491654+0000","last_fullsized":"2024-09-21T06:34:57.491654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:40.085758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690863+0000","last_change":"2024-09-21T06:34:55.734713+0000","last_active":"2024-09-21T06:34:56.690863+0000","last_peered":"2024-09-21T06:34:56.690863+0000","last_clean":"2024-09-21T06:34:56.690863+0000","last_became_active":"2024-09-21T06:34:55.734367+0000","last_became_peered":"2024-09-21T06:34:55.734367+0000","last_unstale":"2024-09-21T06:34:56.690863+0000","last_undegraded":"2024-09-21T06:34:56.690863+0000","last_fullsized":"2024-09-21T06:34:56.690863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.630265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692852+0000","last_change":"2024-09-21T06:34:55.730432+0000","last_active":"2024-09-21T06:34:56.692852+0000","last_peered":"2024-09-21T06:34:56.692852+0000","last_clean":"2024-09-21T06:34:56.692852+0000","last_became_active":"2024-09-21T06:34:55.730085+0000","last_became_peered":"2024-09-21T06:34:55.730085+0000","last_unstale":"2024-09-21T06:34:56.692852+0000","last_undegraded":"2024-09-21T06:34:56.692852+0000","last_fullsized":"2024-09-21T06:34:56.692852+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:13.769344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491281+0000","last_change":"2024-09-21T06:34:55.713592+0000","last_active":"2024-09-21T06:34:57.491281+0000","last_peered":"2024-09-21T06:34:57.491281+0000","last_clean":"2024-09-21T06:34:57.491281+0000","last_became_active":"2024-09-21T06:34:55.711240+0000","last_became_peered":"2024-09-21T06:34:55.711240+0000","last_unstale":"2024-09-21T06:34:57.491281+0000","last_undegraded":"2024-09-21T06:34:57.491281+0000","last_fullsized":"2024-09-21T06:34:57.491281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.722733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491268+0000","last_change":"2024-09-21T06:34:55.712441+0000","last_active":"2024-09-21T06:34:57.491268+0000","last_peered":"2024-09-21T06:34:57.491268+0000","last_clean":"2024-09-21T06:34:57.491268+0000","last_became_active":"2024-09-21T06:34:55.710543+0000","last_became_peered":"2024-09-21T06:34:55.710543+0000","last_unstale":"2024-09-21T06:34:57.491268+0000","last_undegraded":"2024-09-21T06:34:57.491268+0000","last_fullsized":"2024-09-21T06:34:57.491268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:02.131613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897116+0000","last_change":"2024-09-21T06:34:55.727939+0000","last_active":"2024-09-21T06:34:56.897116+0000","last_peered":"2024-09-21T06:34:56.897116+0000","last_clean":"2024-09-21T06:34:56.897116+0000","last_became_active":"2024-09-21T06:34:55.727032+0000","last_became_peered":"2024-09-21T06:34:55.727032+0000","last_unstale":"2024-09-21T06:34:56.897116+0000","last_undegraded":"2024-09-21T06:34:56.897116+0000","last_fullsized":"2024-09-21T06:34:56.897116+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.403443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.896947+0000","last_change":"2024-09-21T06:34:55.723696+0000","last_active":"2024-09-21T06:34:56.896947+0000","last_peered":"2024-09-21T06:34:56.896947+0000","last_clean":"2024-09-21T06:34:56.896947+0000","last_became_active":"2024-09-21T06:34:55.723259+0000","last_became_peered":"2024-09-21T06:34:55.723259+0000","last_unstale":"2024-09-21T06:34:56.896947+0000","last_undegraded":"2024-09-21T06:34:56.896947+0000","last_fullsized":"2024-09-21T06:34:56.896947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:07.748611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691935+0000","last_change":"2024-09-21T06:34:55.723651+0000","last_active":"2024-09-21T06:34:56.691935+0000","last_peered":"2024-09-21T06:34:56.691935+0000","last_clean":"2024-09-21T06:34:56.691935+0000","last_became_active":"2024-09-21T06:34:55.723288+0000","last_became_peered":"2024-09-21T06:34:55.723288+0000","last_unstale":"2024-09-21T06:34:56.691935+0000","last_undegraded":"2024-09-21T06:34:56.691935+0000","last_fullsized":"2024-09-21T06:34:56.691935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:30.694885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691167+0000","last_change":"2024-09-21T06:34:55.704172+0000","last_active":"2024-09-21T06:34:56.691167+0000","last_peered":"2024-09-21T06:34:56.691167+0000","last_clean":"2024-09-21T06:34:56.691167+0000","last_became_active":"2024-09-21T06:34:55.703829+0000","last_became_peered":"2024-09-21T06:34:55.703829+0000","last_unstale":"2024-09-21T06:34:56.691167+0000","last_undegraded":"2024-09-21T06:34:56.691167+0000","last_fullsized":"2024-09-21T06:34:56.691167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:10.923582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693214+0000","last_change":"2024-09-21T06:34:55.714365+0000","last_active":"2024-09-21T06:34:56.693214+0000","last_peered":"2024-09-21T06:34:56.693214+0000","last_clean":"2024-09-21T06:34:56.693214+0000","last_became_active":"2024-09-21T06:34:55.711411+0000","last_became_peered":"2024-09-21T06:34:55.711411+0000","last_unstale":"2024-09-21T06:34:56.693214+0000","last_undegraded":"2024-09-21T06:34:56.693214+0000","last_fullsized":"2024-09-21T06:34:56.693214+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:55.929081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721189+0000","last_change":"2024-09-21T06:34:55.712794+0000","last_active":"2024-09-21T06:34:55.721189+0000","last_peered":"2024-09-21T06:34:55.721189+0000","last_clean":"2024-09-21T06:34:55.721189+0000","last_became_active":"2024-09-21T06:34:55.712487+0000","last_became_peered":"2024-09-21T06:34:55.712487+0000","last_unstale":"2024-09-21T06:34:55.721189+0000","last_undegraded":"2024-09-21T06:34:55.721189+0000","last_fullsized":"2024-09-21T06:34:55.721189+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:48.749080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897088+0000","last_change":"2024-09-21T06:34:55.720101+0000","last_active":"2024-09-21T06:34:56.897088+0000","last_peered":"2024-09-21T06:34:56.897088+0000","last_clean":"2024-09-21T06:34:56.897088+0000","last_became_active":"2024-09-21T06:34:55.719743+0000","last_became_peered":"2024-09-21T06:34:55.719743+0000","last_unstale":"2024-09-21T06:34:56.897088+0000","last_undegraded":"2024-09-21T06:34:56.897088+0000","last_fullsized":"2024-09-21T06:34:56.897088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:52.305232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491183+0000","last_change":"2024-09-21T06:34:55.712044+0000","last_active":"2024-09-21T06:34:57.491183+0000","last_peered":"2024-09-21T06:34:57.491183+0000","last_clean":"2024-09-21T06:34:57.491183+0000","last_became_active":"2024-09-21T06:34:55.710142+0000","last_became_peered":"2024-09-21T06:34:55.710142+0000","last_unstale":"2024-09-21T06:34:57.491183+0000","last_undegraded":"2024-09-21T06:34:57.491183+0000","last_fullsized":"2024-09-21T06:34:57.491183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:42.402252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897146+0000","last_change":"2024-09-21T06:34:55.722424+0000","last_active":"2024-09-21T06:34:56.897146+0000","last_peered":"2024-09-21T06:34:56.897146+0000","last_clean":"2024-09-21T06:34:56.897146+0000","last_became_active":"2024-09-21T06:34:55.722091+0000","last_became_peered":"2024-09-21T06:34:55.722091+0000","last_unstale":"2024-09-21T06:34:56.897146+0000","last_undegraded":"2024-09-21T06:34:56.897146+0000","last_fullsized":"2024-09-21T06:34:56.897146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:23.664483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690751+0000","last_change":"2024-09-21T06:34:55.727747+0000","last_active":"2024-09-21T06:34:56.690751+0000","last_peered":"2024-09-21T06:34:56.690751+0000","last_clean":"2024-09-21T06:34:56.690751+0000","last_became_active":"2024-09-21T06:34:55.727447+0000","last_became_peered":"2024-09-21T06:34:55.727447+0000","last_unstale":"2024-09-21T06:34:56.690751+0000","last_undegraded":"2024-09-21T06:34:56.690751+0000","last_fullsized":"2024-09-21T06:34:56.690751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.086183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683012+0000","last_change":"2024-09-21T06:34:55.710554+0000","last_active":"2024-09-21T06:34:57.683012+0000","last_peered":"2024-09-21T06:34:57.683012+0000","last_clean":"2024-09-21T06:34:57.683012+0000","last_became_active":"2024-09-21T06:34:55.710216+0000","last_became_peered":"2024-09-21T06:34:55.710216+0000","last_unstale":"2024-09-21T06:34:57.683012+0000","last_undegraded":"2024-09-21T06:34:57.683012+0000","last_fullsized":"2024-09-21T06:34:57.683012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692663+0000","last_change":"2024-09-21T06:34:55.732203+0000","last_active":"2024-09-21T06:34:56.692663+0000","last_peered":"2024-09-21T06:34:56.692663+0000","last_clean":"2024-09-21T06:34:56.692663+0000","last_became_active":"2024-09-21T06:34:55.729943+0000","last_became_peered":"2024-09-21T06:34:55.729943+0000","last_unstale":"2024-09-21T06:34:56.692663+0000","last_undegraded":"2024-09-21T06:34:56.692663+0000","last_fullsized":"2024-09-21T06:34:56.692663+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:15.629949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690915+0000","last_change":"2024-09-21T06:34:55.702753+0000","last_active":"2024-09-21T06:34:56.690915+0000","last_peered":"2024-09-21T06:34:56.690915+0000","last_clean":"2024-09-21T06:34:56.690915+0000","last_became_active":"2024-09-21T06:34:55.702431+0000","last_became_peered":"2024-09-21T06:34:55.702431+0000","last_unstale":"2024-09-21T06:34:56.690915+0000","last_undegraded":"2024-09-21T06:34:56.690915+0000","last_fullsized":"2024-09-21T06:34:56.690915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:07.262224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491255+0000","last_change":"2024-09-21T06:34:55.731627+0000","last_active":"2024-09-21T06:34:57.491255+0000","last_peered":"2024-09-21T06:34:57.491255+0000","last_clean":"2024-09-21T06:34:57.491255+0000","last_became_active":"2024-09-21T06:34:55.731309+0000","last_became_peered":"2024-09-21T06:34:55.731309+0000","last_unstale":"2024-09-21T06:34:57.491255+0000","last_undegraded":"2024-09-21T06:34:57.491255+0000","last_fullsized":"2024-09-21T06:34:57.491255+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:24.150794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:55.710593+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:55.710307+0000","last_became_peered":"2024-09-21T06:34:55.710307+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:54.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693341+0000","last_change":"2024-09-21T06:34:55.707636+0000","last_active":"2024-09-21T06:34:56.693341+0000","last_peered":"2024-09-21T06:34:56.693341+0000","last_clean":"2024-09-21T06:34:56.693341+0000","last_became_active":"2024-09-21T06:34:55.707223+0000","last_became_peered":"2024-09-21T06:34:55.707223+0000","last_unstale":"2024-09-21T06:34:56.693341+0000","last_undegraded":"2024-09-21T06:34:56.693341+0000","last_fullsized":"2024-09-21T06:34:56.693341+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:41.577299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731832+0000","last_change":"2024-09-21T06:34:55.728815+0000","last_active":"2024-09-21T06:34:55.731832+0000","last_peered":"2024-09-21T06:34:55.731832+0000","last_clean":"2024-09-21T06:34:55.731832+0000","last_became_active":"2024-09-21T06:34:55.728227+0000","last_became_peered":"2024-09-21T06:34:55.728227+0000","last_unstale":"2024-09-21T06:34:55.731832+0000","last_undegraded":"2024-09-21T06:34:55.731832+0000","last_fullsized":"2024-09-21T06:34:55.731832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:32.261965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692011+0000","last_change":"2024-09-21T06:34:55.731414+0000","last_active":"2024-09-21T06:34:56.692011+0000","last_peered":"2024-09-21T06:34:56.692011+0000","last_clean":"2024-09-21T06:34:56.692011+0000","last_became_active":"2024-09-21T06:34:55.731103+0000","last_became_peered":"2024-09-21T06:34:55.731103+0000","last_unstale":"2024-09-21T06:34:56.692011+0000","last_undegraded":"2024-09-21T06:34:56.692011+0000","last_fullsized":"2024-09-21T06:34:56.692011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691979+0000","last_change":"2024-09-21T06:34:55.715389+0000","last_active":"2024-09-21T06:34:56.691979+0000","last_peered":"2024-09-21T06:34:56.691979+0000","last_clean":"2024-09-21T06:34:56.691979+0000","last_became_active":"2024-09-21T06:34:55.715071+0000","last_became_peered":"2024-09-21T06:34:55.715071+0000","last_unstale":"2024-09-21T06:34:56.691979+0000","last_undegraded":"2024-09-21T06:34:56.691979+0000","last_fullsized":"2024-09-21T06:34:56.691979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682949+0000","last_change":"2024-09-21T06:34:55.717327+0000","last_active":"2024-09-21T06:34:57.682949+0000","last_peered":"2024-09-21T06:34:57.682949+0000","last_clean":"2024-09-21T06:34:57.682949+0000","last_became_active":"2024-09-21T06:34:55.716998+0000","last_became_peered":"2024-09-21T06:34:55.716998+0000","last_unstale":"2024-09-21T06:34:57.682949+0000","last_undegraded":"2024-09-21T06:34:57.682949+0000","last_fullsized":"2024-09-21T06:34:57.682949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683071+0000","last_change":"2024-09-21T06:34:55.717181+0000","last_active":"2024-09-21T06:34:57.683071+0000","last_peered":"2024-09-21T06:34:57.683071+0000","last_clean":"2024-09-21T06:34:57.683071+0000","last_became_active":"2024-09-21T06:34:55.716922+0000","last_became_peered":"2024-09-21T06:34:55.716922+0000","last_unstale":"2024-09-21T06:34:57.683071+0000","last_undegraded":"2024-09-21T06:34:57.683071+0000","last_fullsized":"2024-09-21T06:34:57.683071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683004+0000","last_change":"2024-09-21T06:34:55.715886+0000","last_active":"2024-09-21T06:34:57.683004+0000","last_peered":"2024-09-21T06:34:57.683004+0000","last_clean":"2024-09-21T06:34:57.683004+0000","last_became_active":"2024-09-21T06:34:55.715533+0000","last_became_peered":"2024-09-21T06:34:55.715533+0000","last_unstale":"2024-09-21T06:34:57.683004+0000","last_undegraded":"2024-09-21T06:34:57.683004+0000","last_fullsized":"2024-09-21T06:34:57.683004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897335+0000","last_change":"2024-09-21T06:34:55.730908+0000","last_active":"2024-09-21T06:34:56.897335+0000","last_peered":"2024-09-21T06:34:56.897335+0000","last_clean":"2024-09-21T06:34:56.897335+0000","last_became_active":"2024-09-21T06:34:55.730593+0000","last_became_peered":"2024-09-21T06:34:55.730593+0000","last_unstale":"2024-09-21T06:34:56.897335+0000","last_undegraded":"2024-09-21T06:34:56.897335+0000","last_fullsized":"2024-09-21T06:34:56.897335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692651+0000","last_change":"2024-09-21T06:34:55.728164+0000","last_active":"2024-09-21T06:34:56.692651+0000","last_peered":"2024-09-21T06:34:56.692651+0000","last_clean":"2024-09-21T06:34:56.692651+0000","last_became_active":"2024-09-21T06:34:55.727836+0000","last_became_peered":"2024-09-21T06:34:55.727836+0000","last_unstale":"2024-09-21T06:34:56.692651+0000","last_undegraded":"2024-09-21T06:34:56.692651+0000","last_fullsized":"2024-09-21T06:34:56.692651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.918656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721753+0000","last_change":"2024-09-21T06:34:55.714924+0000","last_active":"2024-09-21T06:34:55.721753+0000","last_peered":"2024-09-21T06:34:55.721753+0000","last_clean":"2024-09-21T06:34:55.721753+0000","last_became_active":"2024-09-21T06:34:55.713658+0000","last_became_peered":"2024-09-21T06:34:55.713658+0000","last_unstale":"2024-09-21T06:34:55.721753+0000","last_undegraded":"2024-09-21T06:34:55.721753+0000","last_fullsized":"2024-09-21T06:34:55.721753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:07.310461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897057+0000","last_change":"2024-09-21T06:34:55.718692+0000","last_active":"2024-09-21T06:34:56.897057+0000","last_peered":"2024-09-21T06:34:56.897057+0000","last_clean":"2024-09-21T06:34:56.897057+0000","last_became_active":"2024-09-21T06:34:55.718323+0000","last_became_peered":"2024-09-21T06:34:55.718323+0000","last_unstale":"2024-09-21T06:34:56.897057+0000","last_undegraded":"2024-09-21T06:34:56.897057+0000","last_fullsized":"2024-09-21T06:34:56.897057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:21.216504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.715947+0000","last_change":"2024-09-21T06:34:55.711836+0000","last_active":"2024-09-21T06:34:55.715947+0000","last_peered":"2024-09-21T06:34:55.715947+0000","last_clean":"2024-09-21T06:34:55.715947+0000","last_became_active":"2024-09-21T06:34:55.711556+0000","last_became_peered":"2024-09-21T06:34:55.711556+0000","last_unstale":"2024-09-21T06:34:55.715947+0000","last_undegraded":"2024-09-21T06:34:55.715947+0000","last_fullsized":"2024-09-21T06:34:55.715947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:24.575657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691960+0000","last_change":"2024-09-21T06:34:55.710652+0000","last_active":"2024-09-21T06:34:56.691960+0000","last_peered":"2024-09-21T06:34:56.691960+0000","last_clean":"2024-09-21T06:34:56.691960+0000","last_became_active":"2024-09-21T06:34:55.709352+0000","last_became_peered":"2024-09-21T06:34:55.709352+0000","last_unstale":"2024-09-21T06:34:56.691960+0000","last_undegraded":"2024-09-21T06:34:56.691960+0000","last_fullsized":"2024-09-21T06:34:56.691960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:32.405217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.730527+0000","last_change":"2024-09-21T06:34:54.714493+0000","last_active":"2024-09-21T06:34:55.730527+0000","last_peered":"2024-09-21T06:34:55.730527+0000","last_clean":"2024-09-21T06:34:55.730527+0000","last_became_active":"2024-09-21T06:34:54.714069+0000","last_became_peered":"2024-09-21T06:34:54.714069+0000","last_unstale":"2024-09-21T06:34:55.730527+0000","last_undegraded":"2024-09-21T06:34:55.730527+0000","last_fullsized":"2024-09-21T06:34:55.730527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:49.070679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692607+0000","last_change":"2024-09-21T06:34:55.731634+0000","last_active":"2024-09-21T06:34:56.692607+0000","last_peered":"2024-09-21T06:34:56.692607+0000","last_clean":"2024-09-21T06:34:56.692607+0000","last_became_active":"2024-09-21T06:34:55.731317+0000","last_became_peered":"2024-09-21T06:34:55.731317+0000","last_unstale":"2024-09-21T06:34:56.692607+0000","last_undegraded":"2024-09-21T06:34:56.692607+0000","last_fullsized":"2024-09-21T06:34:56.692607+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692575+0000","last_change":"2024-09-21T06:34:54.710899+0000","last_active":"2024-09-21T06:34:56.692575+0000","last_peered":"2024-09-21T06:34:56.692575+0000","last_clean":"2024-09-21T06:34:56.692575+0000","last_became_active":"2024-09-21T06:34:54.710535+0000","last_became_peered":"2024-09-21T06:34:54.710535+0000","last_unstale":"2024-09-21T06:34:56.692575+0000","last_undegraded":"2024-09-21T06:34:56.692575+0000","last_fullsized":"2024-09-21T06:34:56.692575+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:21.778699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691077+0000","last_change":"2024-09-21T06:34:55.707231+0000","last_active":"2024-09-21T06:34:56.691077+0000","last_peered":"2024-09-21T06:34:56.691077+0000","last_clean":"2024-09-21T06:34:56.691077+0000","last_became_active":"2024-09-21T06:34:55.706910+0000","last_became_peered":"2024-09-21T06:34:55.706910+0000","last_unstale":"2024-09-21T06:34:56.691077+0000","last_undegraded":"2024-09-21T06:34:56.691077+0000","last_fullsized":"2024-09-21T06:34:56.691077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:31.422885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691219+0000","last_change":"2024-09-21T06:34:54.704122+0000","last_active":"2024-09-21T06:34:56.691219+0000","last_peered":"2024-09-21T06:34:56.691219+0000","last_clean":"2024-09-21T06:34:56.691219+0000","last_became_active":"2024-09-21T06:34:54.703803+0000","last_became_peered":"2024-09-21T06:34:54.703803+0000","last_unstale":"2024-09-21T06:34:56.691219+0000","last_undegraded":"2024-09-21T06:34:56.691219+0000","last_fullsized":"2024-09-21T06:34:56.691219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:49.261246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.737546+0000","last_change":"2024-09-21T06:34:55.731772+0000","last_active":"2024-09-21T06:34:55.737546+0000","last_peered":"2024-09-21T06:34:55.737546+0000","last_clean":"2024-09-21T06:34:55.737546+0000","last_became_active":"2024-09-21T06:34:55.731464+0000","last_became_peered":"2024-09-21T06:34:55.731464+0000","last_unstale":"2024-09-21T06:34:55.737546+0000","last_undegraded":"2024-09-21T06:34:55.737546+0000","last_fullsized":"2024-09-21T06:34:55.737546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:41.639926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692910+0000","last_change":"2024-09-21T06:34:54.701225+0000","last_active":"2024-09-21T06:34:56.692910+0000","last_peered":"2024-09-21T06:34:56.692910+0000","last_clean":"2024-09-21T06:34:56.692910+0000","last_became_active":"2024-09-21T06:34:54.700905+0000","last_became_peered":"2024-09-21T06:34:54.700905+0000","last_unstale":"2024-09-21T06:34:56.692910+0000","last_undegraded":"2024-09-21T06:34:56.692910+0000","last_fullsized":"2024-09-21T06:34:56.692910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491690+0000","last_change":"2024-09-21T06:34:55.698335+0000","last_active":"2024-09-21T06:34:57.491690+0000","last_peered":"2024-09-21T06:34:57.491690+0000","last_clean":"2024-09-21T06:34:57.491690+0000","last_became_active":"2024-09-21T06:34:55.697982+0000","last_became_peered":"2024-09-21T06:34:55.697982+0000","last_unstale":"2024-09-21T06:34:57.491690+0000","last_undegraded":"2024-09-21T06:34:57.491690+0000","last_fullsized":"2024-09-21T06:34:57.491690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:33.941883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.732028+0000","last_change":"2024-09-21T06:34:54.714529+0000","last_active":"2024-09-21T06:34:55.732028+0000","last_peered":"2024-09-21T06:34:55.732028+0000","last_clean":"2024-09-21T06:34:55.732028+0000","last_became_active":"2024-09-21T06:34:54.714159+0000","last_became_peered":"2024-09-21T06:34:54.714159+0000","last_unstale":"2024-09-21T06:34:55.732028+0000","last_undegraded":"2024-09-21T06:34:55.732028+0000","last_fullsized":"2024-09-21T06:34:55.732028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.164452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491696+0000","last_change":"2024-09-21T06:34:55.708931+0000","last_active":"2024-09-21T06:34:57.491696+0000","last_peered":"2024-09-21T06:34:57.491696+0000","last_clean":"2024-09-21T06:34:57.491696+0000","last_became_active":"2024-09-21T06:34:55.708602+0000","last_became_peered":"2024-09-21T06:34:55.708602+0000","last_unstale":"2024-09-21T06:34:57.491696+0000","last_undegraded":"2024-09-21T06:34:57.491696+0000","last_fullsized":"2024-09-21T06:34:57.491696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:53.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:54.709657+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:54.709345+0000","last_became_peered":"2024-09-21T06:34:54.709345+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.714451+0000","last_change":"2024-09-21T06:34:55.709527+0000","last_active":"2024-09-21T06:34:55.714451+0000","last_peered":"2024-09-21T06:34:55.714451+0000","last_clean":"2024-09-21T06:34:55.714451+0000","last_became_active":"2024-09-21T06:34:55.709199+0000","last_became_peered":"2024-09-21T06:34:55.709199+0000","last_unstale":"2024-09-21T06:34:55.714451+0000","last_undegraded":"2024-09-21T06:34:55.714451+0000","last_fullsized":"2024-09-21T06:34:55.714451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:07.792379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682816+0000","last_change":"2024-09-21T06:34:54.708914+0000","last_active":"2024-09-21T06:34:57.682816+0000","last_peered":"2024-09-21T06:34:57.682816+0000","last_clean":"2024-09-21T06:34:57.682816+0000","last_became_active":"2024-09-21T06:34:54.706923+0000","last_became_peered":"2024-09-21T06:34:54.706923+0000","last_unstale":"2024-09-21T06:34:57.682816+0000","last_undegraded":"2024-09-21T06:34:57.682816+0000","last_fullsized":"2024-09-21T06:34:57.682816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491887+0000","last_change":"2024-09-21T06:34:55.710127+0000","last_active":"2024-09-21T06:34:57.491887+0000","last_peered":"2024-09-21T06:34:57.491887+0000","last_clean":"2024-09-21T06:34:57.491887+0000","last_became_active":"2024-09-21T06:34:55.709577+0000","last_became_peered":"2024-09-21T06:34:55.709577+0000","last_unstale":"2024-09-21T06:34:57.491887+0000","last_undegraded":"2024-09-21T06:34:57.491887+0000","last_fullsized":"2024-09-21T06:34:57.491887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.912010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.702796+0000","last_change":"2024-09-21T06:34:54.711684+0000","last_active":"2024-09-21T06:34:55.702796+0000","last_peered":"2024-09-21T06:34:55.702796+0000","last_clean":"2024-09-21T06:34:55.702796+0000","last_became_active":"2024-09-21T06:34:54.711307+0000","last_became_peered":"2024-09-21T06:34:54.711307+0000","last_unstale":"2024-09-21T06:34:55.702796+0000","last_undegraded":"2024-09-21T06:34:55.702796+0000","last_fullsized":"2024-09-21T06:34:55.702796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:08.787586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490933+0000","last_change":"2024-09-21T06:34:55.718040+0000","last_active":"2024-09-21T06:34:57.490933+0000","last_peered":"2024-09-21T06:34:57.490933+0000","last_clean":"2024-09-21T06:34:57.490933+0000","last_became_active":"2024-09-21T06:34:55.717355+0000","last_became_peered":"2024-09-21T06:34:55.717355+0000","last_unstale":"2024-09-21T06:34:57.490933+0000","last_undegraded":"2024-09-21T06:34:57.490933+0000","last_fullsized":"2024-09-21T06:34:57.490933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897542+0000","last_change":"2024-09-21T06:34:54.709082+0000","last_active":"2024-09-21T06:34:56.897542+0000","last_peered":"2024-09-21T06:34:56.897542+0000","last_clean":"2024-09-21T06:34:56.897542+0000","last_became_active":"2024-09-21T06:34:54.708646+0000","last_became_peered":"2024-09-21T06:34:54.708646+0000","last_unstale":"2024-09-21T06:34:56.897542+0000","last_undegraded":"2024-09-21T06:34:56.897542+0000","last_fullsized":"2024-09-21T06:34:56.897542+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:59.547312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490987+0000","last_change":"2024-09-21T06:34:55.706722+0000","last_active":"2024-09-21T06:34:57.490987+0000","last_peered":"2024-09-21T06:34:57.490987+0000","last_clean":"2024-09-21T06:34:57.490987+0000","last_became_active":"2024-09-21T06:34:55.706428+0000","last_became_peered":"2024-09-21T06:34:55.706428+0000","last_unstale":"2024-09-21T06:34:57.490987+0000","last_undegraded":"2024-09-21T06:34:57.490987+0000","last_fullsized":"2024-09-21T06:34:57.490987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:17.159683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693179+0000","last_change":"2024-09-21T06:34:54.700087+0000","last_active":"2024-09-21T06:34:56.693179+0000","last_peered":"2024-09-21T06:34:56.693179+0000","last_clean":"2024-09-21T06:34:56.693179+0000","last_became_active":"2024-09-21T06:34:54.699715+0000","last_became_peered":"2024-09-21T06:34:54.699715+0000","last_unstale":"2024-09-21T06:34:56.693179+0000","last_undegraded":"2024-09-21T06:34:56.693179+0000","last_fullsized":"2024-09-21T06:34:56.693179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693783+0000","last_change":"2024-09-21T06:34:55.714419+0000","last_active":"2024-09-21T06:34:56.693783+0000","last_peered":"2024-09-21T06:34:56.693783+0000","last_clean":"2024-09-21T06:34:56.693783+0000","last_became_active":"2024-09-21T06:34:55.714040+0000","last_became_peered":"2024-09-21T06:34:55.714040+0000","last_unstale":"2024-09-21T06:34:56.693783+0000","last_undegraded":"2024-09-21T06:34:56.693783+0000","last_fullsized":"2024-09-21T06:34:56.693783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.757308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897468+0000","last_change":"2024-09-21T06:34:54.708697+0000","last_active":"2024-09-21T06:34:56.897468+0000","last_peered":"2024-09-21T06:34:56.897468+0000","last_clean":"2024-09-21T06:34:56.897468+0000","last_became_active":"2024-09-21T06:34:54.708338+0000","last_became_peered":"2024-09-21T06:34:54.708338+0000","last_unstale":"2024-09-21T06:34:56.897468+0000","last_undegraded":"2024-09-21T06:34:56.897468+0000","last_fullsized":"2024-09-21T06:34:56.897468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:29.524412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.733897+0000","last_change":"2024-09-21T06:34:55.732505+0000","last_active":"2024-09-21T06:34:55.733897+0000","last_peered":"2024-09-21T06:34:55.733897+0000","last_clean":"2024-09-21T06:34:55.733897+0000","last_became_active":"2024-09-21T06:34:55.732209+0000","last_became_peered":"2024-09-21T06:34:55.732209+0000","last_unstale":"2024-09-21T06:34:55.733897+0000","last_undegraded":"2024-09-21T06:34:55.733897+0000","last_fullsized":"2024-09-21T06:34:55.733897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:36.296181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.740620+0000","last_change":"2024-09-21T06:34:54.704500+0000","last_active":"2024-09-21T06:34:55.740620+0000","last_peered":"2024-09-21T06:34:55.740620+0000","last_clean":"2024-09-21T06:34:55.740620+0000","last_became_active":"2024-09-21T06:34:54.704165+0000","last_became_peered":"2024-09-21T06:34:54.704165+0000","last_unstale":"2024-09-21T06:34:55.740620+0000","last_undegraded":"2024-09-21T06:34:55.740620+0000","last_fullsized":"2024-09-21T06:34:55.740620+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.649075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690939+0000","last_change":"2024-09-21T06:34:55.705237+0000","last_active":"2024-09-21T06:34:56.690939+0000","last_peered":"2024-09-21T06:34:56.690939+0000","last_clean":"2024-09-21T06:34:56.690939+0000","last_became_active":"2024-09-21T06:34:55.704951+0000","last_became_peered":"2024-09-21T06:34:55.704951+0000","last_unstale":"2024-09-21T06:34:56.690939+0000","last_undegraded":"2024-09-21T06:34:56.690939+0000","last_fullsized":"2024-09-21T06:34:56.690939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491461+0000","last_change":"2024-09-21T06:34:54.709537+0000","last_active":"2024-09-21T06:34:57.491461+0000","last_peered":"2024-09-21T06:34:57.491461+0000","last_clean":"2024-09-21T06:34:57.491461+0000","last_became_active":"2024-09-21T06:34:54.706229+0000","last_became_peered":"2024-09-21T06:34:54.706229+0000","last_unstale":"2024-09-21T06:34:57.491461+0000","last_undegraded":"2024-09-21T06:34:57.491461+0000","last_fullsized":"2024-09-21T06:34:57.491461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:57.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691227+0000","last_change":"2024-09-21T06:34:55.699196+0000","last_active":"2024-09-21T06:34:56.691227+0000","last_peered":"2024-09-21T06:34:56.691227+0000","last_clean":"2024-09-21T06:34:56.691227+0000","last_became_active":"2024-09-21T06:34:55.698813+0000","last_became_peered":"2024-09-21T06:34:55.698813+0000","last_unstale":"2024-09-21T06:34:56.691227+0000","last_undegraded":"2024-09-21T06:34:56.691227+0000","last_fullsized":"2024-09-21T06:34:56.691227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:46.110588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693829+0000","last_change":"2024-09-21T06:34:54.701370+0000","last_active":"2024-09-21T06:34:56.693829+0000","last_peered":"2024-09-21T06:34:56.693829+0000","last_clean":"2024-09-21T06:34:56.693829+0000","last_became_active":"2024-09-21T06:34:54.700999+0000","last_became_peered":"2024-09-21T06:34:54.700999+0000","last_unstale":"2024-09-21T06:34:56.693829+0000","last_undegraded":"2024-09-21T06:34:56.693829+0000","last_fullsized":"2024-09-21T06:34:56.693829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731581+0000","last_change":"2024-09-21T06:34:55.728639+0000","last_active":"2024-09-21T06:34:55.731581+0000","last_peered":"2024-09-21T06:34:55.731581+0000","last_clean":"2024-09-21T06:34:55.731581+0000","last_became_active":"2024-09-21T06:34:55.728220+0000","last_became_peered":"2024-09-21T06:34:55.728220+0000","last_unstale":"2024-09-21T06:34:55.731581+0000","last_undegraded":"2024-09-21T06:34:55.731581+0000","last_fullsized":"2024-09-21T06:34:55.731581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.666244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490873+0000","last_change":"2024-09-21T06:34:54.708770+0000","last_active":"2024-09-21T06:34:57.490873+0000","last_peered":"2024-09-21T06:34:57.490873+0000","last_clean":"2024-09-21T06:34:57.490873+0000","last_became_active":"2024-09-21T06:34:54.708363+0000","last_became_peered":"2024-09-21T06:34:54.708363+0000","last_unstale":"2024-09-21T06:34:57.490873+0000","last_undegraded":"2024-09-21T06:34:57.490873+0000","last_fullsized":"2024-09-21T06:34:57.490873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:08.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692930+0000","last_change":"2024-09-21T06:34:55.730868+0000","last_active":"2024-09-21T06:34:56.692930+0000","last_peered":"2024-09-21T06:34:56.692930+0000","last_clean":"2024-09-21T06:34:56.692930+0000","last_became_active":"2024-09-21T06:34:55.730509+0000","last_became_peered":"2024-09-21T06:34:55.730509+0000","last_unstale":"2024-09-21T06:34:56.692930+0000","last_undegraded":"2024-09-21T06:34:56.692930+0000","last_fullsized":"2024-09-21T06:34:56.692930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.519311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.734554+0000","last_change":"2024-09-21T06:34:54.713801+0000","last_active":"2024-09-21T06:34:55.734554+0000","last_peered":"2024-09-21T06:34:55.734554+0000","last_clean":"2024-09-21T06:34:55.734554+0000","last_became_active":"2024-09-21T06:34:54.712759+0000","last_became_peered":"2024-09-21T06:34:54.712759+0000","last_unstale":"2024-09-21T06:34:55.734554+0000","last_undegraded":"2024-09-21T06:34:55.734554+0000","last_fullsized":"2024-09-21T06:34:55.734554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:47.339290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692225+0000","last_change":"2024-09-21T06:34:55.732582+0000","last_active":"2024-09-21T06:34:56.692225+0000","last_peered":"2024-09-21T06:34:56.692225+0000","last_clean":"2024-09-21T06:34:56.692225+0000","last_became_active":"2024-09-21T06:34:55.732328+0000","last_became_peered":"2024-09-21T06:34:55.732328+0000","last_unstale":"2024-09-21T06:34:56.692225+0000","last_undegraded":"2024-09-21T06:34:56.692225+0000","last_fullsized":"2024-09-21T06:34:56.692225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897574+0000","last_change":"2024-09-21T06:34:54.710621+0000","last_active":"2024-09-21T06:34:56.897574+0000","last_peered":"2024-09-21T06:34:56.897574+0000","last_clean":"2024-09-21T06:34:56.897574+0000","last_became_active":"2024-09-21T06:34:54.710305+0000","last_became_peered":"2024-09-21T06:34:54.710305+0000","last_unstale":"2024-09-21T06:34:56.897574+0000","last_undegraded":"2024-09-21T06:34:56.897574+0000","last_fullsized":"2024-09-21T06:34:56.897574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:04.681733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683086+0000","last_change":"2024-09-21T06:34:55.716269+0000","last_active":"2024-09-21T06:34:57.683086+0000","last_peered":"2024-09-21T06:34:57.683086+0000","last_clean":"2024-09-21T06:34:57.683086+0000","last_became_active":"2024-09-21T06:34:55.706408+0000","last_became_peered":"2024-09-21T06:34:55.706408+0000","last_unstale":"2024-09-21T06:34:57.683086+0000","last_undegraded":"2024-09-21T06:34:57.683086+0000","last_fullsized":"2024-09-21T06:34:57.683086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683117+0000","last_change":"2024-09-21T06:34:54.712959+0000","last_active":"2024-09-21T06:34:57.683117+0000","last_peered":"2024-09-21T06:34:57.683117+0000","last_clean":"2024-09-21T06:34:57.683117+0000","last_became_active":"2024-09-21T06:34:54.711866+0000","last_became_peered":"2024-09-21T06:34:54.711866+0000","last_unstale":"2024-09-21T06:34:57.683117+0000","last_undegraded":"2024-09-21T06:34:57.683117+0000","last_fullsized":"2024-09-21T06:34:57.683117+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731728+0000","last_change":"2024-09-21T06:34:55.728437+0000","last_active":"2024-09-21T06:34:55.731728+0000","last_peered":"2024-09-21T06:34:55.731728+0000","last_clean":"2024-09-21T06:34:55.731728+0000","last_became_active":"2024-09-21T06:34:55.728111+0000","last_became_peered":"2024-09-21T06:34:55.728111+0000","last_unstale":"2024-09-21T06:34:55.731728+0000","last_undegraded":"2024-09-21T06:34:55.731728+0000","last_fullsized":"2024-09-21T06:34:55.731728+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:21.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897340+0000","last_change":"2024-09-21T06:34:54.710224+0000","last_active":"2024-09-21T06:34:56.897340+0000","last_peered":"2024-09-21T06:34:56.897340+0000","last_clean":"2024-09-21T06:34:56.897340+0000","last_became_active":"2024-09-21T06:34:54.709896+0000","last_became_peered":"2024-09-21T06:34:54.709896+0000","last_unstale":"2024-09-21T06:34:56.897340+0000","last_undegraded":"2024-09-21T06:34:56.897340+0000","last_fullsized":"2024-09-21T06:34:56.897340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:16.511663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491563+0000","last_change":"2024-09-21T06:34:55.708497+0000","last_active":"2024-09-21T06:34:57.491563+0000","last_peered":"2024-09-21T06:34:57.491563+0000","last_clean":"2024-09-21T06:34:57.491563+0000","last_became_active":"2024-09-21T06:34:55.706136+0000","last_became_peered":"2024-09-21T06:34:55.706136+0000","last_unstale":"2024-09-21T06:34:57.491563+0000","last_undegraded":"2024-09-21T06:34:57.491563+0000","last_fullsized":"2024-09-21T06:34:57.491563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:06.652871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693236+0000","last_change":"2024-09-21T06:34:54.712196+0000","last_active":"2024-09-21T06:34:56.693236+0000","last_peered":"2024-09-21T06:34:56.693236+0000","last_clean":"2024-09-21T06:34:56.693236+0000","last_became_active":"2024-09-21T06:34:54.711507+0000","last_became_peered":"2024-09-21T06:34:54.711507+0000","last_unstale":"2024-09-21T06:34:56.693236+0000","last_undegraded":"2024-09-21T06:34:56.693236+0000","last_fullsized":"2024-09-21T06:34:56.693236+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:07.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.725352+0000","last_change":"2024-09-21T06:34:55.718899+0000","last_active":"2024-09-21T06:34:55.725352+0000","last_peered":"2024-09-21T06:34:55.725352+0000","last_clean":"2024-09-21T06:34:55.725352+0000","last_became_active":"2024-09-21T06:34:55.718610+0000","last_became_peered":"2024-09-21T06:34:55.718610+0000","last_unstale":"2024-09-21T06:34:55.725352+0000","last_undegraded":"2024-09-21T06:34:55.725352+0000","last_fullsized":"2024-09-21T06:34:55.725352+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:14.255062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692044+0000","last_change":"2024-09-21T06:34:54.704910+0000","last_active":"2024-09-21T06:34:56.692044+0000","last_peered":"2024-09-21T06:34:56.692044+0000","last_clean":"2024-09-21T06:34:56.692044+0000","last_became_active":"2024-09-21T06:34:54.703410+0000","last_became_peered":"2024-09-21T06:34:54.703410+0000","last_unstale":"2024-09-21T06:34:56.692044+0000","last_undegraded":"2024-09-21T06:34:56.692044+0000","last_fullsized":"2024-09-21T06:34:56.692044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:36.483455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.731552+0000","last_change":"2024-09-21T06:34:55.726247+0000","last_active":"2024-09-21T06:34:55.731552+0000","last_peered":"2024-09-21T06:34:55.731552+0000","last_clean":"2024-09-21T06:34:55.731552+0000","last_became_active":"2024-09-21T06:34:55.725850+0000","last_became_peered":"2024-09-21T06:34:55.725850+0000","last_unstale":"2024-09-21T06:34:55.731552+0000","last_undegraded":"2024-09-21T06:34:55.731552+0000","last_fullsized":"2024-09-21T06:34:55.731552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:40.564061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897179+0000","last_change":"2024-09-21T06:34:54.712284+0000","last_active":"2024-09-21T06:34:56.897179+0000","last_peered":"2024-09-21T06:34:56.897179+0000","last_clean":"2024-09-21T06:34:56.897179+0000","last_became_active":"2024-09-21T06:34:54.711828+0000","last_became_peered":"2024-09-21T06:34:54.711828+0000","last_unstale":"2024-09-21T06:34:56.897179+0000","last_undegraded":"2024-09-21T06:34:56.897179+0000","last_fullsized":"2024-09-21T06:34:56.897179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:29.075613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491685+0000","last_change":"2024-09-21T06:34:55.713268+0000","last_active":"2024-09-21T06:34:57.491685+0000","last_peered":"2024-09-21T06:34:57.491685+0000","last_clean":"2024-09-21T06:34:57.491685+0000","last_became_active":"2024-09-21T06:34:55.712940+0000","last_became_peered":"2024-09-21T06:34:55.712940+0000","last_unstale":"2024-09-21T06:34:57.491685+0000","last_undegraded":"2024-09-21T06:34:57.491685+0000","last_fullsized":"2024-09-21T06:34:57.491685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:20.633620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692250+0000","last_change":"2024-09-21T06:34:54.706724+0000","last_active":"2024-09-21T06:34:56.692250+0000","last_peered":"2024-09-21T06:34:56.692250+0000","last_clean":"2024-09-21T06:34:56.692250+0000","last_became_active":"2024-09-21T06:34:54.706045+0000","last_became_peered":"2024-09-21T06:34:54.706045+0000","last_unstale":"2024-09-21T06:34:56.692250+0000","last_undegraded":"2024-09-21T06:34:56.692250+0000","last_fullsized":"2024-09-21T06:34:56.692250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.717407+0000","last_change":"2024-09-21T06:34:55.716497+0000","last_active":"2024-09-21T06:34:55.717407+0000","last_peered":"2024-09-21T06:34:55.717407+0000","last_clean":"2024-09-21T06:34:55.717407+0000","last_became_active":"2024-09-21T06:34:55.716182+0000","last_became_peered":"2024-09-21T06:34:55.716182+0000","last_unstale":"2024-09-21T06:34:55.717407+0000","last_undegraded":"2024-09-21T06:34:55.717407+0000","last_fullsized":"2024-09-21T06:34:55.717407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:13.835969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693187+0000","last_change":"2024-09-21T06:34:54.704018+0000","last_active":"2024-09-21T06:34:56.693187+0000","last_peered":"2024-09-21T06:34:56.693187+0000","last_clean":"2024-09-21T06:34:56.693187+0000","last_became_active":"2024-09-21T06:34:54.703663+0000","last_became_peered":"2024-09-21T06:34:54.703663+0000","last_unstale":"2024-09-21T06:34:56.693187+0000","last_undegraded":"2024-09-21T06:34:56.693187+0000","last_fullsized":"2024-09-21T06:34:56.693187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:06.524348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.721388+0000","last_change":"2024-09-21T06:34:55.712873+0000","last_active":"2024-09-21T06:34:55.721388+0000","last_peered":"2024-09-21T06:34:55.721388+0000","last_clean":"2024-09-21T06:34:55.721388+0000","last_became_active":"2024-09-21T06:34:55.711695+0000","last_became_peered":"2024-09-21T06:34:55.711695+0000","last_unstale":"2024-09-21T06:34:55.721388+0000","last_undegraded":"2024-09-21T06:34:55.721388+0000","last_fullsized":"2024-09-21T06:34:55.721388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:31.987545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692851+0000","last_change":"2024-09-21T06:34:54.709868+0000","last_active":"2024-09-21T06:34:56.692851+0000","last_peered":"2024-09-21T06:34:56.692851+0000","last_clean":"2024-09-21T06:34:56.692851+0000","last_became_active":"2024-09-21T06:34:54.708447+0000","last_became_peered":"2024-09-21T06:34:54.708447+0000","last_unstale":"2024-09-21T06:34:56.692851+0000","last_undegraded":"2024-09-21T06:34:56.692851+0000","last_fullsized":"2024-09-21T06:34:56.692851+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:50.904913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693744+0000","last_change":"2024-09-21T06:34:55.729064+0000","last_active":"2024-09-21T06:34:56.693744+0000","last_peered":"2024-09-21T06:34:56.693744+0000","last_clean":"2024-09-21T06:34:56.693744+0000","last_became_active":"2024-09-21T06:34:55.728795+0000","last_became_peered":"2024-09-21T06:34:55.728795+0000","last_unstale":"2024-09-21T06:34:56.693744+0000","last_undegraded":"2024-09-21T06:34:56.693744+0000","last_fullsized":"2024-09-21T06:34:56.693744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:38.265711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.703782+0000","last_change":"2024-09-21T06:34:54.702424+0000","last_active":"2024-09-21T06:34:55.703782+0000","last_peered":"2024-09-21T06:34:55.703782+0000","last_clean":"2024-09-21T06:34:55.703782+0000","last_became_active":"2024-09-21T06:34:54.702065+0000","last_became_peered":"2024-09-21T06:34:54.702065+0000","last_unstale":"2024-09-21T06:34:55.703782+0000","last_undegraded":"2024-09-21T06:34:55.703782+0000","last_fullsized":"2024-09-21T06:34:55.703782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:51.535195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683128+0000","last_change":"2024-09-21T06:34:55.718743+0000","last_active":"2024-09-21T06:34:57.683128+0000","last_peered":"2024-09-21T06:34:57.683128+0000","last_clean":"2024-09-21T06:34:57.683128+0000","last_became_active":"2024-09-21T06:34:55.703205+0000","last_became_peered":"2024-09-21T06:34:55.703205+0000","last_unstale":"2024-09-21T06:34:57.683128+0000","last_undegraded":"2024-09-21T06:34:57.683128+0000","last_fullsized":"2024-09-21T06:34:57.683128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491730+0000","last_change":"2024-09-21T06:34:54.711548+0000","last_active":"2024-09-21T06:34:57.491730+0000","last_peered":"2024-09-21T06:34:57.491730+0000","last_clean":"2024-09-21T06:34:57.491730+0000","last_became_active":"2024-09-21T06:34:54.711141+0000","last_became_peered":"2024-09-21T06:34:54.711141+0000","last_unstale":"2024-09-21T06:34:57.491730+0000","last_undegraded":"2024-09-21T06:34:57.491730+0000","last_fullsized":"2024-09-21T06:34:57.491730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:27.210366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491185+0000","last_change":"2024-09-21T06:34:55.712582+0000","last_active":"2024-09-21T06:34:57.491185+0000","last_peered":"2024-09-21T06:34:57.491185+0000","last_clean":"2024-09-21T06:34:57.491185+0000","last_became_active":"2024-09-21T06:34:55.710833+0000","last_became_peered":"2024-09-21T06:34:55.710833+0000","last_unstale":"2024-09-21T06:34:57.491185+0000","last_undegraded":"2024-09-21T06:34:57.491185+0000","last_fullsized":"2024-09-21T06:34:57.491185+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T06:34:55.706570+0000","last_change":"2024-09-21T06:34:54.713168+0000","last_active":"2024-09-21T06:34:55.706570+0000","last_peered":"2024-09-21T06:34:55.706570+0000","last_clean":"2024-09-21T06:34:55.706570+0000","last_became_active":"2024-09-21T06:34:54.712837+0000","last_became_peered":"2024-09-21T06:34:54.712837+0000","last_unstale":"2024-09-21T06:34:55.706570+0000","last_undegraded":"2024-09-21T06:34:55.706570+0000","last_fullsized":"2024-09-21T06:34:55.706570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:29.731794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491694+0000","last_change":"2024-09-21T06:34:55.711645+0000","last_active":"2024-09-21T06:34:57.491694+0000","last_peered":"2024-09-21T06:34:57.491694+0000","last_clean":"2024-09-21T06:34:57.491694+0000","last_became_active":"2024-09-21T06:34:55.711312+0000","last_became_peered":"2024-09-21T06:34:55.711312+0000","last_unstale":"2024-09-21T06:34:57.491694+0000","last_undegraded":"2024-09-21T06:34:57.491694+0000","last_fullsized":"2024-09-21T06:34:57.491694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692192+0000","last_change":"2024-09-21T06:34:54.705515+0000","last_active":"2024-09-21T06:34:56.692192+0000","last_peered":"2024-09-21T06:34:56.692192+0000","last_clean":"2024-09-21T06:34:56.692192+0000","last_became_active":"2024-09-21T06:34:54.705153+0000","last_became_peered":"2024-09-21T06:34:54.705153+0000","last_unstale":"2024-09-21T06:34:56.692192+0000","last_undegraded":"2024-09-21T06:34:56.692192+0000","last_fullsized":"2024-09-21T06:34:56.692192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:35.586391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897683+0000","last_change":"2024-09-21T06:34:55.729077+0000","last_active":"2024-09-21T06:34:56.897683+0000","last_peered":"2024-09-21T06:34:56.897683+0000","last_clean":"2024-09-21T06:34:56.897683+0000","last_became_active":"2024-09-21T06:34:55.728776+0000","last_became_peered":"2024-09-21T06:34:55.728776+0000","last_unstale":"2024-09-21T06:34:56.897683+0000","last_undegraded":"2024-09-21T06:34:56.897683+0000","last_fullsized":"2024-09-21T06:34:56.897683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683192+0000","last_change":"2024-09-21T06:34:54.703341+0000","last_active":"2024-09-21T06:34:57.683192+0000","last_peered":"2024-09-21T06:34:57.683192+0000","last_clean":"2024-09-21T06:34:57.683192+0000","last_became_active":"2024-09-21T06:34:54.703002+0000","last_became_peered":"2024-09-21T06:34:54.703002+0000","last_unstale":"2024-09-21T06:34:57.683192+0000","last_undegraded":"2024-09-21T06:34:57.683192+0000","last_fullsized":"2024-09-21T06:34:57.683192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691146+0000","last_change":"2024-09-21T06:34:55.737366+0000","last_active":"2024-09-21T06:34:56.691146+0000","last_peered":"2024-09-21T06:34:56.691146+0000","last_clean":"2024-09-21T06:34:56.691146+0000","last_became_active":"2024-09-21T06:34:55.734444+0000","last_became_peered":"2024-09-21T06:34:55.734444+0000","last_unstale":"2024-09-21T06:34:56.691146+0000","last_undegraded":"2024-09-21T06:34:56.691146+0000","last_fullsized":"2024-09-21T06:34:56.691146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:47.619903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692878+0000","last_change":"2024-09-21T06:34:54.702747+0000","last_active":"2024-09-21T06:34:56.692878+0000","last_peered":"2024-09-21T06:34:56.692878+0000","last_clean":"2024-09-21T06:34:56.692878+0000","last_became_active":"2024-09-21T06:34:54.702221+0000","last_became_peered":"2024-09-21T06:34:54.702221+0000","last_unstale":"2024-09-21T06:34:56.692878+0000","last_undegraded":"2024-09-21T06:34:56.692878+0000","last_fullsized":"2024-09-21T06:34:56.692878+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:26.752919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897188+0000","last_change":"2024-09-21T06:34:55.724899+0000","last_active":"2024-09-21T06:34:56.897188+0000","last_peered":"2024-09-21T06:34:56.897188+0000","last_clean":"2024-09-21T06:34:56.897188+0000","last_became_active":"2024-09-21T06:34:55.724167+0000","last_became_peered":"2024-09-21T06:34:55.724167+0000","last_unstale":"2024-09-21T06:34:56.897188+0000","last_undegraded":"2024-09-21T06:34:56.897188+0000","last_fullsized":"2024-09-21T06:34:56.897188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491794+0000","last_change":"2024-09-21T06:34:54.707367+0000","last_active":"2024-09-21T06:34:57.491794+0000","last_peered":"2024-09-21T06:34:57.491794+0000","last_clean":"2024-09-21T06:34:57.491794+0000","last_became_active":"2024-09-21T06:34:54.705863+0000","last_became_peered":"2024-09-21T06:34:54.705863+0000","last_unstale":"2024-09-21T06:34:57.491794+0000","last_undegraded":"2024-09-21T06:34:57.491794+0000","last_fullsized":"2024-09-21T06:34:57.491794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.687054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682836+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:34:57.682836+0000","last_peered":"2024-09-21T06:34:57.682836+0000","last_clean":"2024-09-21T06:34:57.682836+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:34:57.682836+0000","last_undegraded":"2024-09-21T06:34:57.682836+0000","last_fullsized":"2024-09-21T06:34:57.682836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1277952,"data_stored":589850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509529,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":595924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1282048,"data_stored":590810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":1179658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":596472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1884160,"data_stored":1186382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:05.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 -- 172.21.15.37:0/1127839119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571006e500 msgr2=0x7f571006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 --2- 172.21.15.37:0/1127839119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571006e500 0x7f571006ede0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f5708005fd0 tx=0x7f570802a840 comp rx=0 tx=0).stop 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 -- 172.21.15.37:0/1127839119 shutdown_connections 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 --2- 172.21.15.37:0/1127839119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571006e500 0x7f571006ede0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 -- 172.21.15.37:0/1127839119 >> 172.21.15.37:0/1127839119 conn(0x7f57100cca40 msgr2=0x7f57100cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 -- 172.21.15.37:0/1127839119 shutdown_connections 2024-09-21T06:35:05.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.489+0000 7f5716800640 1 -- 172.21.15.37:0/1127839119 wait complete. 2024-09-21T06:35:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.492+0000 7f5716800640 1 Processor -- start 2024-09-21T06:35:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.492+0000 7f5716800640 1 -- start start 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5710074cb0 0x7f57101e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 0x7f571015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f571006f6b0 con 0x7f5710073dd0 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f571006f530 con 0x7f5710157f60 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f5716800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f571006f830 con 0x7f5710074cb0 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f570f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f570f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58368/0 (socket says 172.21.15.37:58368) 2024-09-21T06:35:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.493+0000 7f570f400640 1 -- 172.21.15.37:0/2829702510 learned_addr learned my addr 172.21.15.37:0/2829702510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570fe00640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 0x7f571015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 -- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5710074cb0 msgr2=0x7f57101e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5710074cb0 0x7f57101e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 -- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 msgr2=0x7f571015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 0x7f571015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 -- 172.21.15.37:0/2829702510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101f6a90 con 0x7f5710073dd0 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570fe00640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 0x7f571015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f570f400640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f570000c730 tx=0x7f570000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700017070 con 0x7f5710073dd0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5700005360 con 0x7f5710073dd0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101e71a0 con 0x7f5710073dd0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.494+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570000a8a0 con 0x7f5710073dd0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.495+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101e7440 con 0x7f5710073dd0 2024-09-21T06:35:05.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.496+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57101e7880 con 0x7f5710073dd0 2024-09-21T06:35:05.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.497+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f570001a070 con 0x7f5710073dd0 2024-09-21T06:35:05.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.498+0000 7f5707400640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.498+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f570009a360 con 0x7f5710073dd0 2024-09-21T06:35:05.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.499+0000 7f5707e00640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.499+0000 7f5707e00640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 0x7f56ec07b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000f60 tx=0x7f56f80079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.503+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5700062750 con 0x7f5710073dd0 2024-09-21T06:35:05.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.697+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f571015edd0 con 0x7f56ec078b90 2024-09-21T06:35:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.702+0000 7f5707400640 1 -- 172.21.15.37:0/2829702510 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252405 (secure 0 0 0) 0x7f56f8005080 con 0x7f56ec078b90 2024-09-21T06:35:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:05.707 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:35:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 msgr2=0x7f56ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 0x7f56ec07b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000f60 tx=0x7f56f80079b0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 msgr2=0x7f57101f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f570000c730 tx=0x7f570000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 shutdown_connections 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5710157f60 0x7f571015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5710074cb0 0x7f57101e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 --2- 172.21.15.37:0/2829702510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5710073dd0 0x7f57101f6550 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 >> 172.21.15.37:0/2829702510 conn(0x7f57100cca40 msgr2=0x7f57100ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 shutdown_connections 2024-09-21T06:35:05.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:05.706+0000 7f5716800640 1 -- 172.21.15.37:0/2829702510 wait complete. 2024-09-21T06:35:05.731 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-21T06:35:05.206978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260752,"kb_used_data":11240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713968,"statfs":{"total":773094113280,"available":772827103232,"internally_reserved":0,"allocated":11509760,"data_stored":5952676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.518152"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491654+0000","last_change":"2024-09-21T06:34:55.702355+0000","last_active":"2024-09-21T06:34:57.491654+0000","last_peered":"2024-09-21T06:34:57.491654+0000","last_clean":"2024-09-21T06:34:57.491654+0000","last_became_active":"2024-09-21T06:34:55.701180+0000","last_became_peered":"2024-09-21T06:34:55.701180+0000","last_unstale":"2024-09-21T06:34:57.491654+0000","last_undegraded":"2024-09-21T06:34:57.491654+0000","last_fullsized":"2024-09-21T06:34:57.491654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:40.085758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690863+0000","last_change":"2024-09-21T06:34:55.734713+0000","last_active":"2024-09-21T06:34:56.690863+0000","last_peered":"2024-09-21T06:34:56.690863+0000","last_clean":"2024-09-21T06:34:56.690863+0000","last_became_active":"2024-09-21T06:34:55.734367+0000","last_became_peered":"2024-09-21T06:34:55.734367+0000","last_unstale":"2024-09-21T06:34:56.690863+0000","last_undegraded":"2024-09-21T06:34:56.690863+0000","last_fullsized":"2024-09-21T06:34:56.690863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.630265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692852+0000","last_change":"2024-09-21T06:34:55.730432+0000","last_active":"2024-09-21T06:34:56.692852+0000","last_peered":"2024-09-21T06:34:56.692852+0000","last_clean":"2024-09-21T06:34:56.692852+0000","last_became_active":"2024-09-21T06:34:55.730085+0000","last_became_peered":"2024-09-21T06:34:55.730085+0000","last_unstale":"2024-09-21T06:34:56.692852+0000","last_undegraded":"2024-09-21T06:34:56.692852+0000","last_fullsized":"2024-09-21T06:34:56.692852+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:13.769344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491281+0000","last_change":"2024-09-21T06:34:55.713592+0000","last_active":"2024-09-21T06:34:57.491281+0000","last_peered":"2024-09-21T06:34:57.491281+0000","last_clean":"2024-09-21T06:34:57.491281+0000","last_became_active":"2024-09-21T06:34:55.711240+0000","last_became_peered":"2024-09-21T06:34:55.711240+0000","last_unstale":"2024-09-21T06:34:57.491281+0000","last_undegraded":"2024-09-21T06:34:57.491281+0000","last_fullsized":"2024-09-21T06:34:57.491281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.722733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491268+0000","last_change":"2024-09-21T06:34:55.712441+0000","last_active":"2024-09-21T06:34:57.491268+0000","last_peered":"2024-09-21T06:34:57.491268+0000","last_clean":"2024-09-21T06:34:57.491268+0000","last_became_active":"2024-09-21T06:34:55.710543+0000","last_became_peered":"2024-09-21T06:34:55.710543+0000","last_unstale":"2024-09-21T06:34:57.491268+0000","last_undegraded":"2024-09-21T06:34:57.491268+0000","last_fullsized":"2024-09-21T06:34:57.491268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:02.131613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897116+0000","last_change":"2024-09-21T06:34:55.727939+0000","last_active":"2024-09-21T06:34:56.897116+0000","last_peered":"2024-09-21T06:34:56.897116+0000","last_clean":"2024-09-21T06:34:56.897116+0000","last_became_active":"2024-09-21T06:34:55.727032+0000","last_became_peered":"2024-09-21T06:34:55.727032+0000","last_unstale":"2024-09-21T06:34:56.897116+0000","last_undegraded":"2024-09-21T06:34:56.897116+0000","last_fullsized":"2024-09-21T06:34:56.897116+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.403443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.896947+0000","last_change":"2024-09-21T06:34:55.723696+0000","last_active":"2024-09-21T06:34:56.896947+0000","last_peered":"2024-09-21T06:34:56.896947+0000","last_clean":"2024-09-21T06:34:56.896947+0000","last_became_active":"2024-09-21T06:34:55.723259+0000","last_became_peered":"2024-09-21T06:34:55.723259+0000","last_unstale":"2024-09-21T06:34:56.896947+0000","last_undegraded":"2024-09-21T06:34:56.896947+0000","last_fullsized":"2024-09-21T06:34:56.896947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:07.748611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691935+0000","last_change":"2024-09-21T06:34:55.723651+0000","last_active":"2024-09-21T06:34:56.691935+0000","last_peered":"2024-09-21T06:34:56.691935+0000","last_clean":"2024-09-21T06:34:56.691935+0000","last_became_active":"2024-09-21T06:34:55.723288+0000","last_became_peered":"2024-09-21T06:34:55.723288+0000","last_unstale":"2024-09-21T06:34:56.691935+0000","last_undegraded":"2024-09-21T06:34:56.691935+0000","last_fullsized":"2024-09-21T06:34:56.691935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:30.694885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691167+0000","last_change":"2024-09-21T06:34:55.704172+0000","last_active":"2024-09-21T06:34:56.691167+0000","last_peered":"2024-09-21T06:34:56.691167+0000","last_clean":"2024-09-21T06:34:56.691167+0000","last_became_active":"2024-09-21T06:34:55.703829+0000","last_became_peered":"2024-09-21T06:34:55.703829+0000","last_unstale":"2024-09-21T06:34:56.691167+0000","last_undegraded":"2024-09-21T06:34:56.691167+0000","last_fullsized":"2024-09-21T06:34:56.691167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:10.923582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693214+0000","last_change":"2024-09-21T06:34:55.714365+0000","last_active":"2024-09-21T06:34:56.693214+0000","last_peered":"2024-09-21T06:34:56.693214+0000","last_clean":"2024-09-21T06:34:56.693214+0000","last_became_active":"2024-09-21T06:34:55.711411+0000","last_became_peered":"2024-09-21T06:34:55.711411+0000","last_unstale":"2024-09-21T06:34:56.693214+0000","last_undegraded":"2024-09-21T06:34:56.693214+0000","last_fullsized":"2024-09-21T06:34:56.693214+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:55.929081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183276+0000","last_change":"2024-09-21T06:34:55.712794+0000","last_active":"2024-09-21T06:34:58.183276+0000","last_peered":"2024-09-21T06:34:58.183276+0000","last_clean":"2024-09-21T06:34:58.183276+0000","last_became_active":"2024-09-21T06:34:55.712487+0000","last_became_peered":"2024-09-21T06:34:55.712487+0000","last_unstale":"2024-09-21T06:34:58.183276+0000","last_undegraded":"2024-09-21T06:34:58.183276+0000","last_fullsized":"2024-09-21T06:34:58.183276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:41.626948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897088+0000","last_change":"2024-09-21T06:34:55.720101+0000","last_active":"2024-09-21T06:34:56.897088+0000","last_peered":"2024-09-21T06:34:56.897088+0000","last_clean":"2024-09-21T06:34:56.897088+0000","last_became_active":"2024-09-21T06:34:55.719743+0000","last_became_peered":"2024-09-21T06:34:55.719743+0000","last_unstale":"2024-09-21T06:34:56.897088+0000","last_undegraded":"2024-09-21T06:34:56.897088+0000","last_fullsized":"2024-09-21T06:34:56.897088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:52.305232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491183+0000","last_change":"2024-09-21T06:34:55.712044+0000","last_active":"2024-09-21T06:34:57.491183+0000","last_peered":"2024-09-21T06:34:57.491183+0000","last_clean":"2024-09-21T06:34:57.491183+0000","last_became_active":"2024-09-21T06:34:55.710142+0000","last_became_peered":"2024-09-21T06:34:55.710142+0000","last_unstale":"2024-09-21T06:34:57.491183+0000","last_undegraded":"2024-09-21T06:34:57.491183+0000","last_fullsized":"2024-09-21T06:34:57.491183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:42.402252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897146+0000","last_change":"2024-09-21T06:34:55.722424+0000","last_active":"2024-09-21T06:34:56.897146+0000","last_peered":"2024-09-21T06:34:56.897146+0000","last_clean":"2024-09-21T06:34:56.897146+0000","last_became_active":"2024-09-21T06:34:55.722091+0000","last_became_peered":"2024-09-21T06:34:55.722091+0000","last_unstale":"2024-09-21T06:34:56.897146+0000","last_undegraded":"2024-09-21T06:34:56.897146+0000","last_fullsized":"2024-09-21T06:34:56.897146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:23.664483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690751+0000","last_change":"2024-09-21T06:34:55.727747+0000","last_active":"2024-09-21T06:34:56.690751+0000","last_peered":"2024-09-21T06:34:56.690751+0000","last_clean":"2024-09-21T06:34:56.690751+0000","last_became_active":"2024-09-21T06:34:55.727447+0000","last_became_peered":"2024-09-21T06:34:55.727447+0000","last_unstale":"2024-09-21T06:34:56.690751+0000","last_undegraded":"2024-09-21T06:34:56.690751+0000","last_fullsized":"2024-09-21T06:34:56.690751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.086183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683012+0000","last_change":"2024-09-21T06:34:55.710554+0000","last_active":"2024-09-21T06:34:57.683012+0000","last_peered":"2024-09-21T06:34:57.683012+0000","last_clean":"2024-09-21T06:34:57.683012+0000","last_became_active":"2024-09-21T06:34:55.710216+0000","last_became_peered":"2024-09-21T06:34:55.710216+0000","last_unstale":"2024-09-21T06:34:57.683012+0000","last_undegraded":"2024-09-21T06:34:57.683012+0000","last_fullsized":"2024-09-21T06:34:57.683012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692663+0000","last_change":"2024-09-21T06:34:55.732203+0000","last_active":"2024-09-21T06:34:56.692663+0000","last_peered":"2024-09-21T06:34:56.692663+0000","last_clean":"2024-09-21T06:34:56.692663+0000","last_became_active":"2024-09-21T06:34:55.729943+0000","last_became_peered":"2024-09-21T06:34:55.729943+0000","last_unstale":"2024-09-21T06:34:56.692663+0000","last_undegraded":"2024-09-21T06:34:56.692663+0000","last_fullsized":"2024-09-21T06:34:56.692663+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:15.629949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690915+0000","last_change":"2024-09-21T06:34:55.702753+0000","last_active":"2024-09-21T06:34:56.690915+0000","last_peered":"2024-09-21T06:34:56.690915+0000","last_clean":"2024-09-21T06:34:56.690915+0000","last_became_active":"2024-09-21T06:34:55.702431+0000","last_became_peered":"2024-09-21T06:34:55.702431+0000","last_unstale":"2024-09-21T06:34:56.690915+0000","last_undegraded":"2024-09-21T06:34:56.690915+0000","last_fullsized":"2024-09-21T06:34:56.690915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:07.262224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491255+0000","last_change":"2024-09-21T06:34:55.731627+0000","last_active":"2024-09-21T06:34:57.491255+0000","last_peered":"2024-09-21T06:34:57.491255+0000","last_clean":"2024-09-21T06:34:57.491255+0000","last_became_active":"2024-09-21T06:34:55.731309+0000","last_became_peered":"2024-09-21T06:34:55.731309+0000","last_unstale":"2024-09-21T06:34:57.491255+0000","last_undegraded":"2024-09-21T06:34:57.491255+0000","last_fullsized":"2024-09-21T06:34:57.491255+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:24.150794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:55.710593+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:55.710307+0000","last_became_peered":"2024-09-21T06:34:55.710307+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:54.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693341+0000","last_change":"2024-09-21T06:34:55.707636+0000","last_active":"2024-09-21T06:34:56.693341+0000","last_peered":"2024-09-21T06:34:56.693341+0000","last_clean":"2024-09-21T06:34:56.693341+0000","last_became_active":"2024-09-21T06:34:55.707223+0000","last_became_peered":"2024-09-21T06:34:55.707223+0000","last_unstale":"2024-09-21T06:34:56.693341+0000","last_undegraded":"2024-09-21T06:34:56.693341+0000","last_fullsized":"2024-09-21T06:34:56.693341+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:41.577299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184343+0000","last_change":"2024-09-21T06:34:55.728815+0000","last_active":"2024-09-21T06:34:58.184343+0000","last_peered":"2024-09-21T06:34:58.184343+0000","last_clean":"2024-09-21T06:34:58.184343+0000","last_became_active":"2024-09-21T06:34:55.728227+0000","last_became_peered":"2024-09-21T06:34:55.728227+0000","last_unstale":"2024-09-21T06:34:58.184343+0000","last_undegraded":"2024-09-21T06:34:58.184343+0000","last_fullsized":"2024-09-21T06:34:58.184343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:19.364312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692011+0000","last_change":"2024-09-21T06:34:55.731414+0000","last_active":"2024-09-21T06:34:56.692011+0000","last_peered":"2024-09-21T06:34:56.692011+0000","last_clean":"2024-09-21T06:34:56.692011+0000","last_became_active":"2024-09-21T06:34:55.731103+0000","last_became_peered":"2024-09-21T06:34:55.731103+0000","last_unstale":"2024-09-21T06:34:56.692011+0000","last_undegraded":"2024-09-21T06:34:56.692011+0000","last_fullsized":"2024-09-21T06:34:56.692011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691979+0000","last_change":"2024-09-21T06:34:55.715389+0000","last_active":"2024-09-21T06:34:56.691979+0000","last_peered":"2024-09-21T06:34:56.691979+0000","last_clean":"2024-09-21T06:34:56.691979+0000","last_became_active":"2024-09-21T06:34:55.715071+0000","last_became_peered":"2024-09-21T06:34:55.715071+0000","last_unstale":"2024-09-21T06:34:56.691979+0000","last_undegraded":"2024-09-21T06:34:56.691979+0000","last_fullsized":"2024-09-21T06:34:56.691979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682949+0000","last_change":"2024-09-21T06:34:55.717327+0000","last_active":"2024-09-21T06:34:57.682949+0000","last_peered":"2024-09-21T06:34:57.682949+0000","last_clean":"2024-09-21T06:34:57.682949+0000","last_became_active":"2024-09-21T06:34:55.716998+0000","last_became_peered":"2024-09-21T06:34:55.716998+0000","last_unstale":"2024-09-21T06:34:57.682949+0000","last_undegraded":"2024-09-21T06:34:57.682949+0000","last_fullsized":"2024-09-21T06:34:57.682949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683071+0000","last_change":"2024-09-21T06:34:55.717181+0000","last_active":"2024-09-21T06:34:57.683071+0000","last_peered":"2024-09-21T06:34:57.683071+0000","last_clean":"2024-09-21T06:34:57.683071+0000","last_became_active":"2024-09-21T06:34:55.716922+0000","last_became_peered":"2024-09-21T06:34:55.716922+0000","last_unstale":"2024-09-21T06:34:57.683071+0000","last_undegraded":"2024-09-21T06:34:57.683071+0000","last_fullsized":"2024-09-21T06:34:57.683071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683004+0000","last_change":"2024-09-21T06:34:55.715886+0000","last_active":"2024-09-21T06:34:57.683004+0000","last_peered":"2024-09-21T06:34:57.683004+0000","last_clean":"2024-09-21T06:34:57.683004+0000","last_became_active":"2024-09-21T06:34:55.715533+0000","last_became_peered":"2024-09-21T06:34:55.715533+0000","last_unstale":"2024-09-21T06:34:57.683004+0000","last_undegraded":"2024-09-21T06:34:57.683004+0000","last_fullsized":"2024-09-21T06:34:57.683004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:58.428678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897335+0000","last_change":"2024-09-21T06:34:55.730908+0000","last_active":"2024-09-21T06:34:56.897335+0000","last_peered":"2024-09-21T06:34:56.897335+0000","last_clean":"2024-09-21T06:34:56.897335+0000","last_became_active":"2024-09-21T06:34:55.730593+0000","last_became_peered":"2024-09-21T06:34:55.730593+0000","last_unstale":"2024-09-21T06:34:56.897335+0000","last_undegraded":"2024-09-21T06:34:56.897335+0000","last_fullsized":"2024-09-21T06:34:56.897335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:43.814342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692651+0000","last_change":"2024-09-21T06:34:55.728164+0000","last_active":"2024-09-21T06:34:56.692651+0000","last_peered":"2024-09-21T06:34:56.692651+0000","last_clean":"2024-09-21T06:34:56.692651+0000","last_became_active":"2024-09-21T06:34:55.727836+0000","last_became_peered":"2024-09-21T06:34:55.727836+0000","last_unstale":"2024-09-21T06:34:56.692651+0000","last_undegraded":"2024-09-21T06:34:56.692651+0000","last_fullsized":"2024-09-21T06:34:56.692651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.918656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183388+0000","last_change":"2024-09-21T06:34:55.714924+0000","last_active":"2024-09-21T06:34:58.183388+0000","last_peered":"2024-09-21T06:34:58.183388+0000","last_clean":"2024-09-21T06:34:58.183388+0000","last_became_active":"2024-09-21T06:34:55.713658+0000","last_became_peered":"2024-09-21T06:34:55.713658+0000","last_unstale":"2024-09-21T06:34:58.183388+0000","last_undegraded":"2024-09-21T06:34:58.183388+0000","last_fullsized":"2024-09-21T06:34:58.183388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:21.346890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897057+0000","last_change":"2024-09-21T06:34:55.718692+0000","last_active":"2024-09-21T06:34:56.897057+0000","last_peered":"2024-09-21T06:34:56.897057+0000","last_clean":"2024-09-21T06:34:56.897057+0000","last_became_active":"2024-09-21T06:34:55.718323+0000","last_became_peered":"2024-09-21T06:34:55.718323+0000","last_unstale":"2024-09-21T06:34:56.897057+0000","last_undegraded":"2024-09-21T06:34:56.897057+0000","last_fullsized":"2024-09-21T06:34:56.897057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:21.216504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184147+0000","last_change":"2024-09-21T06:34:55.711836+0000","last_active":"2024-09-21T06:34:58.184147+0000","last_peered":"2024-09-21T06:34:58.184147+0000","last_clean":"2024-09-21T06:34:58.184147+0000","last_became_active":"2024-09-21T06:34:55.711556+0000","last_became_peered":"2024-09-21T06:34:55.711556+0000","last_unstale":"2024-09-21T06:34:58.184147+0000","last_undegraded":"2024-09-21T06:34:58.184147+0000","last_fullsized":"2024-09-21T06:34:58.184147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:31.443690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691960+0000","last_change":"2024-09-21T06:34:55.710652+0000","last_active":"2024-09-21T06:34:56.691960+0000","last_peered":"2024-09-21T06:34:56.691960+0000","last_clean":"2024-09-21T06:34:56.691960+0000","last_became_active":"2024-09-21T06:34:55.709352+0000","last_became_peered":"2024-09-21T06:34:55.709352+0000","last_unstale":"2024-09-21T06:34:56.691960+0000","last_undegraded":"2024-09-21T06:34:56.691960+0000","last_fullsized":"2024-09-21T06:34:56.691960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:32.405217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184756+0000","last_change":"2024-09-21T06:34:54.714493+0000","last_active":"2024-09-21T06:34:58.184756+0000","last_peered":"2024-09-21T06:34:58.184756+0000","last_clean":"2024-09-21T06:34:58.184756+0000","last_became_active":"2024-09-21T06:34:54.714069+0000","last_became_peered":"2024-09-21T06:34:54.714069+0000","last_unstale":"2024-09-21T06:34:58.184756+0000","last_undegraded":"2024-09-21T06:34:58.184756+0000","last_fullsized":"2024-09-21T06:34:58.184756+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:49.070679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692607+0000","last_change":"2024-09-21T06:34:55.731634+0000","last_active":"2024-09-21T06:34:56.692607+0000","last_peered":"2024-09-21T06:34:56.692607+0000","last_clean":"2024-09-21T06:34:56.692607+0000","last_became_active":"2024-09-21T06:34:55.731317+0000","last_became_peered":"2024-09-21T06:34:55.731317+0000","last_unstale":"2024-09-21T06:34:56.692607+0000","last_undegraded":"2024-09-21T06:34:56.692607+0000","last_fullsized":"2024-09-21T06:34:56.692607+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692575+0000","last_change":"2024-09-21T06:34:54.710899+0000","last_active":"2024-09-21T06:34:56.692575+0000","last_peered":"2024-09-21T06:34:56.692575+0000","last_clean":"2024-09-21T06:34:56.692575+0000","last_became_active":"2024-09-21T06:34:54.710535+0000","last_became_peered":"2024-09-21T06:34:54.710535+0000","last_unstale":"2024-09-21T06:34:56.692575+0000","last_undegraded":"2024-09-21T06:34:56.692575+0000","last_fullsized":"2024-09-21T06:34:56.692575+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:21.778699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691077+0000","last_change":"2024-09-21T06:34:55.707231+0000","last_active":"2024-09-21T06:34:56.691077+0000","last_peered":"2024-09-21T06:34:56.691077+0000","last_clean":"2024-09-21T06:34:56.691077+0000","last_became_active":"2024-09-21T06:34:55.706910+0000","last_became_peered":"2024-09-21T06:34:55.706910+0000","last_unstale":"2024-09-21T06:34:56.691077+0000","last_undegraded":"2024-09-21T06:34:56.691077+0000","last_fullsized":"2024-09-21T06:34:56.691077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:31.422885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691219+0000","last_change":"2024-09-21T06:34:54.704122+0000","last_active":"2024-09-21T06:34:56.691219+0000","last_peered":"2024-09-21T06:34:56.691219+0000","last_clean":"2024-09-21T06:34:56.691219+0000","last_became_active":"2024-09-21T06:34:54.703803+0000","last_became_peered":"2024-09-21T06:34:54.703803+0000","last_unstale":"2024-09-21T06:34:56.691219+0000","last_undegraded":"2024-09-21T06:34:56.691219+0000","last_fullsized":"2024-09-21T06:34:56.691219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:49.261246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183306+0000","last_change":"2024-09-21T06:34:55.731772+0000","last_active":"2024-09-21T06:34:58.183306+0000","last_peered":"2024-09-21T06:34:58.183306+0000","last_clean":"2024-09-21T06:34:58.183306+0000","last_became_active":"2024-09-21T06:34:55.731464+0000","last_became_peered":"2024-09-21T06:34:55.731464+0000","last_unstale":"2024-09-21T06:34:58.183306+0000","last_undegraded":"2024-09-21T06:34:58.183306+0000","last_fullsized":"2024-09-21T06:34:58.183306+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692910+0000","last_change":"2024-09-21T06:34:54.701225+0000","last_active":"2024-09-21T06:34:56.692910+0000","last_peered":"2024-09-21T06:34:56.692910+0000","last_clean":"2024-09-21T06:34:56.692910+0000","last_became_active":"2024-09-21T06:34:54.700905+0000","last_became_peered":"2024-09-21T06:34:54.700905+0000","last_unstale":"2024-09-21T06:34:56.692910+0000","last_undegraded":"2024-09-21T06:34:56.692910+0000","last_fullsized":"2024-09-21T06:34:56.692910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491690+0000","last_change":"2024-09-21T06:34:55.698335+0000","last_active":"2024-09-21T06:34:57.491690+0000","last_peered":"2024-09-21T06:34:57.491690+0000","last_clean":"2024-09-21T06:34:57.491690+0000","last_became_active":"2024-09-21T06:34:55.697982+0000","last_became_peered":"2024-09-21T06:34:55.697982+0000","last_unstale":"2024-09-21T06:34:57.491690+0000","last_undegraded":"2024-09-21T06:34:57.491690+0000","last_fullsized":"2024-09-21T06:34:57.491690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:33.941883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183322+0000","last_change":"2024-09-21T06:34:54.714529+0000","last_active":"2024-09-21T06:34:58.183322+0000","last_peered":"2024-09-21T06:34:58.183322+0000","last_clean":"2024-09-21T06:34:58.183322+0000","last_became_active":"2024-09-21T06:34:54.714159+0000","last_became_peered":"2024-09-21T06:34:54.714159+0000","last_unstale":"2024-09-21T06:34:58.183322+0000","last_undegraded":"2024-09-21T06:34:58.183322+0000","last_fullsized":"2024-09-21T06:34:58.183322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.164452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491696+0000","last_change":"2024-09-21T06:34:55.708931+0000","last_active":"2024-09-21T06:34:57.491696+0000","last_peered":"2024-09-21T06:34:57.491696+0000","last_clean":"2024-09-21T06:34:57.491696+0000","last_became_active":"2024-09-21T06:34:55.708602+0000","last_became_peered":"2024-09-21T06:34:55.708602+0000","last_unstale":"2024-09-21T06:34:57.491696+0000","last_undegraded":"2024-09-21T06:34:57.491696+0000","last_fullsized":"2024-09-21T06:34:57.491696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:53.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683133+0000","last_change":"2024-09-21T06:34:54.709657+0000","last_active":"2024-09-21T06:34:57.683133+0000","last_peered":"2024-09-21T06:34:57.683133+0000","last_clean":"2024-09-21T06:34:57.683133+0000","last_became_active":"2024-09-21T06:34:54.709345+0000","last_became_peered":"2024-09-21T06:34:54.709345+0000","last_unstale":"2024-09-21T06:34:57.683133+0000","last_undegraded":"2024-09-21T06:34:57.683133+0000","last_fullsized":"2024-09-21T06:34:57.683133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:59.697451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183643+0000","last_change":"2024-09-21T06:34:55.709527+0000","last_active":"2024-09-21T06:34:58.183643+0000","last_peered":"2024-09-21T06:34:58.183643+0000","last_clean":"2024-09-21T06:34:58.183643+0000","last_became_active":"2024-09-21T06:34:55.709199+0000","last_became_peered":"2024-09-21T06:34:55.709199+0000","last_unstale":"2024-09-21T06:34:58.183643+0000","last_undegraded":"2024-09-21T06:34:58.183643+0000","last_fullsized":"2024-09-21T06:34:58.183643+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.641532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682816+0000","last_change":"2024-09-21T06:34:54.708914+0000","last_active":"2024-09-21T06:34:57.682816+0000","last_peered":"2024-09-21T06:34:57.682816+0000","last_clean":"2024-09-21T06:34:57.682816+0000","last_became_active":"2024-09-21T06:34:54.706923+0000","last_became_peered":"2024-09-21T06:34:54.706923+0000","last_unstale":"2024-09-21T06:34:57.682816+0000","last_undegraded":"2024-09-21T06:34:57.682816+0000","last_fullsized":"2024-09-21T06:34:57.682816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491887+0000","last_change":"2024-09-21T06:34:55.710127+0000","last_active":"2024-09-21T06:34:57.491887+0000","last_peered":"2024-09-21T06:34:57.491887+0000","last_clean":"2024-09-21T06:34:57.491887+0000","last_became_active":"2024-09-21T06:34:55.709577+0000","last_became_peered":"2024-09-21T06:34:55.709577+0000","last_unstale":"2024-09-21T06:34:57.491887+0000","last_undegraded":"2024-09-21T06:34:57.491887+0000","last_fullsized":"2024-09-21T06:34:57.491887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.912010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184485+0000","last_change":"2024-09-21T06:34:54.711684+0000","last_active":"2024-09-21T06:34:58.184485+0000","last_peered":"2024-09-21T06:34:58.184485+0000","last_clean":"2024-09-21T06:34:58.184485+0000","last_became_active":"2024-09-21T06:34:54.711307+0000","last_became_peered":"2024-09-21T06:34:54.711307+0000","last_unstale":"2024-09-21T06:34:58.184485+0000","last_undegraded":"2024-09-21T06:34:58.184485+0000","last_fullsized":"2024-09-21T06:34:58.184485+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:08.787586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490933+0000","last_change":"2024-09-21T06:34:55.718040+0000","last_active":"2024-09-21T06:34:57.490933+0000","last_peered":"2024-09-21T06:34:57.490933+0000","last_clean":"2024-09-21T06:34:57.490933+0000","last_became_active":"2024-09-21T06:34:55.717355+0000","last_became_peered":"2024-09-21T06:34:55.717355+0000","last_unstale":"2024-09-21T06:34:57.490933+0000","last_undegraded":"2024-09-21T06:34:57.490933+0000","last_fullsized":"2024-09-21T06:34:57.490933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:06.976783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897542+0000","last_change":"2024-09-21T06:34:54.709082+0000","last_active":"2024-09-21T06:34:56.897542+0000","last_peered":"2024-09-21T06:34:56.897542+0000","last_clean":"2024-09-21T06:34:56.897542+0000","last_became_active":"2024-09-21T06:34:54.708646+0000","last_became_peered":"2024-09-21T06:34:54.708646+0000","last_unstale":"2024-09-21T06:34:56.897542+0000","last_undegraded":"2024-09-21T06:34:56.897542+0000","last_fullsized":"2024-09-21T06:34:56.897542+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:59.547312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490987+0000","last_change":"2024-09-21T06:34:55.706722+0000","last_active":"2024-09-21T06:34:57.490987+0000","last_peered":"2024-09-21T06:34:57.490987+0000","last_clean":"2024-09-21T06:34:57.490987+0000","last_became_active":"2024-09-21T06:34:55.706428+0000","last_became_peered":"2024-09-21T06:34:55.706428+0000","last_unstale":"2024-09-21T06:34:57.490987+0000","last_undegraded":"2024-09-21T06:34:57.490987+0000","last_fullsized":"2024-09-21T06:34:57.490987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:17.159683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693179+0000","last_change":"2024-09-21T06:34:54.700087+0000","last_active":"2024-09-21T06:34:56.693179+0000","last_peered":"2024-09-21T06:34:56.693179+0000","last_clean":"2024-09-21T06:34:56.693179+0000","last_became_active":"2024-09-21T06:34:54.699715+0000","last_became_peered":"2024-09-21T06:34:54.699715+0000","last_unstale":"2024-09-21T06:34:56.693179+0000","last_undegraded":"2024-09-21T06:34:56.693179+0000","last_fullsized":"2024-09-21T06:34:56.693179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693783+0000","last_change":"2024-09-21T06:34:55.714419+0000","last_active":"2024-09-21T06:34:56.693783+0000","last_peered":"2024-09-21T06:34:56.693783+0000","last_clean":"2024-09-21T06:34:56.693783+0000","last_became_active":"2024-09-21T06:34:55.714040+0000","last_became_peered":"2024-09-21T06:34:55.714040+0000","last_unstale":"2024-09-21T06:34:56.693783+0000","last_undegraded":"2024-09-21T06:34:56.693783+0000","last_fullsized":"2024-09-21T06:34:56.693783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.757308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897468+0000","last_change":"2024-09-21T06:34:54.708697+0000","last_active":"2024-09-21T06:34:56.897468+0000","last_peered":"2024-09-21T06:34:56.897468+0000","last_clean":"2024-09-21T06:34:56.897468+0000","last_became_active":"2024-09-21T06:34:54.708338+0000","last_became_peered":"2024-09-21T06:34:54.708338+0000","last_unstale":"2024-09-21T06:34:56.897468+0000","last_undegraded":"2024-09-21T06:34:56.897468+0000","last_fullsized":"2024-09-21T06:34:56.897468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:29.524412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184158+0000","last_change":"2024-09-21T06:34:55.732505+0000","last_active":"2024-09-21T06:34:58.184158+0000","last_peered":"2024-09-21T06:34:58.184158+0000","last_clean":"2024-09-21T06:34:58.184158+0000","last_became_active":"2024-09-21T06:34:55.732209+0000","last_became_peered":"2024-09-21T06:34:55.732209+0000","last_unstale":"2024-09-21T06:34:58.184158+0000","last_undegraded":"2024-09-21T06:34:58.184158+0000","last_fullsized":"2024-09-21T06:34:58.184158+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:50.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":21,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183248+0000","last_change":"2024-09-21T06:34:54.704500+0000","last_active":"2024-09-21T06:34:58.183248+0000","last_peered":"2024-09-21T06:34:58.183248+0000","last_clean":"2024-09-21T06:34:58.183248+0000","last_became_active":"2024-09-21T06:34:54.704165+0000","last_became_peered":"2024-09-21T06:34:54.704165+0000","last_unstale":"2024-09-21T06:34:58.183248+0000","last_undegraded":"2024-09-21T06:34:58.183248+0000","last_fullsized":"2024-09-21T06:34:58.183248+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.649075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.690939+0000","last_change":"2024-09-21T06:34:55.705237+0000","last_active":"2024-09-21T06:34:56.690939+0000","last_peered":"2024-09-21T06:34:56.690939+0000","last_clean":"2024-09-21T06:34:56.690939+0000","last_became_active":"2024-09-21T06:34:55.704951+0000","last_became_peered":"2024-09-21T06:34:55.704951+0000","last_unstale":"2024-09-21T06:34:56.690939+0000","last_undegraded":"2024-09-21T06:34:56.690939+0000","last_fullsized":"2024-09-21T06:34:56.690939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491461+0000","last_change":"2024-09-21T06:34:54.709537+0000","last_active":"2024-09-21T06:34:57.491461+0000","last_peered":"2024-09-21T06:34:57.491461+0000","last_clean":"2024-09-21T06:34:57.491461+0000","last_became_active":"2024-09-21T06:34:54.706229+0000","last_became_peered":"2024-09-21T06:34:54.706229+0000","last_unstale":"2024-09-21T06:34:57.491461+0000","last_undegraded":"2024-09-21T06:34:57.491461+0000","last_fullsized":"2024-09-21T06:34:57.491461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:57.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691227+0000","last_change":"2024-09-21T06:34:55.699196+0000","last_active":"2024-09-21T06:34:56.691227+0000","last_peered":"2024-09-21T06:34:56.691227+0000","last_clean":"2024-09-21T06:34:56.691227+0000","last_became_active":"2024-09-21T06:34:55.698813+0000","last_became_peered":"2024-09-21T06:34:55.698813+0000","last_unstale":"2024-09-21T06:34:56.691227+0000","last_undegraded":"2024-09-21T06:34:56.691227+0000","last_fullsized":"2024-09-21T06:34:56.691227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:46.110588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693829+0000","last_change":"2024-09-21T06:34:54.701370+0000","last_active":"2024-09-21T06:34:56.693829+0000","last_peered":"2024-09-21T06:34:56.693829+0000","last_clean":"2024-09-21T06:34:56.693829+0000","last_became_active":"2024-09-21T06:34:54.700999+0000","last_became_peered":"2024-09-21T06:34:54.700999+0000","last_unstale":"2024-09-21T06:34:56.693829+0000","last_undegraded":"2024-09-21T06:34:56.693829+0000","last_fullsized":"2024-09-21T06:34:56.693829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:26.670269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184625+0000","last_change":"2024-09-21T06:34:55.728639+0000","last_active":"2024-09-21T06:34:58.184625+0000","last_peered":"2024-09-21T06:34:58.184625+0000","last_clean":"2024-09-21T06:34:58.184625+0000","last_became_active":"2024-09-21T06:34:55.728220+0000","last_became_peered":"2024-09-21T06:34:55.728220+0000","last_unstale":"2024-09-21T06:34:58.184625+0000","last_undegraded":"2024-09-21T06:34:58.184625+0000","last_fullsized":"2024-09-21T06:34:58.184625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:17.519494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.490873+0000","last_change":"2024-09-21T06:34:54.708770+0000","last_active":"2024-09-21T06:34:57.490873+0000","last_peered":"2024-09-21T06:34:57.490873+0000","last_clean":"2024-09-21T06:34:57.490873+0000","last_became_active":"2024-09-21T06:34:54.708363+0000","last_became_peered":"2024-09-21T06:34:54.708363+0000","last_unstale":"2024-09-21T06:34:57.490873+0000","last_undegraded":"2024-09-21T06:34:57.490873+0000","last_fullsized":"2024-09-21T06:34:57.490873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:08.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692930+0000","last_change":"2024-09-21T06:34:55.730868+0000","last_active":"2024-09-21T06:34:56.692930+0000","last_peered":"2024-09-21T06:34:56.692930+0000","last_clean":"2024-09-21T06:34:56.692930+0000","last_became_active":"2024-09-21T06:34:55.730509+0000","last_became_peered":"2024-09-21T06:34:55.730509+0000","last_unstale":"2024-09-21T06:34:56.692930+0000","last_undegraded":"2024-09-21T06:34:56.692930+0000","last_fullsized":"2024-09-21T06:34:56.692930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:18.519311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":23,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183154+0000","last_change":"2024-09-21T06:34:54.713801+0000","last_active":"2024-09-21T06:34:58.183154+0000","last_peered":"2024-09-21T06:34:58.183154+0000","last_clean":"2024-09-21T06:34:58.183154+0000","last_became_active":"2024-09-21T06:34:54.712759+0000","last_became_peered":"2024-09-21T06:34:54.712759+0000","last_unstale":"2024-09-21T06:34:58.183154+0000","last_undegraded":"2024-09-21T06:34:58.183154+0000","last_fullsized":"2024-09-21T06:34:58.183154+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:47.339290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692225+0000","last_change":"2024-09-21T06:34:55.732582+0000","last_active":"2024-09-21T06:34:56.692225+0000","last_peered":"2024-09-21T06:34:56.692225+0000","last_clean":"2024-09-21T06:34:56.692225+0000","last_became_active":"2024-09-21T06:34:55.732328+0000","last_became_peered":"2024-09-21T06:34:55.732328+0000","last_unstale":"2024-09-21T06:34:56.692225+0000","last_undegraded":"2024-09-21T06:34:56.692225+0000","last_fullsized":"2024-09-21T06:34:56.692225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:28.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897574+0000","last_change":"2024-09-21T06:34:54.710621+0000","last_active":"2024-09-21T06:34:56.897574+0000","last_peered":"2024-09-21T06:34:56.897574+0000","last_clean":"2024-09-21T06:34:56.897574+0000","last_became_active":"2024-09-21T06:34:54.710305+0000","last_became_peered":"2024-09-21T06:34:54.710305+0000","last_unstale":"2024-09-21T06:34:56.897574+0000","last_undegraded":"2024-09-21T06:34:56.897574+0000","last_fullsized":"2024-09-21T06:34:56.897574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:04.681733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683086+0000","last_change":"2024-09-21T06:34:55.716269+0000","last_active":"2024-09-21T06:34:57.683086+0000","last_peered":"2024-09-21T06:34:57.683086+0000","last_clean":"2024-09-21T06:34:57.683086+0000","last_became_active":"2024-09-21T06:34:55.706408+0000","last_became_peered":"2024-09-21T06:34:55.706408+0000","last_unstale":"2024-09-21T06:34:57.683086+0000","last_undegraded":"2024-09-21T06:34:57.683086+0000","last_fullsized":"2024-09-21T06:34:57.683086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683117+0000","last_change":"2024-09-21T06:34:54.712959+0000","last_active":"2024-09-21T06:34:57.683117+0000","last_peered":"2024-09-21T06:34:57.683117+0000","last_clean":"2024-09-21T06:34:57.683117+0000","last_became_active":"2024-09-21T06:34:54.711866+0000","last_became_peered":"2024-09-21T06:34:54.711866+0000","last_unstale":"2024-09-21T06:34:57.683117+0000","last_undegraded":"2024-09-21T06:34:57.683117+0000","last_fullsized":"2024-09-21T06:34:57.683117+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:01.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184521+0000","last_change":"2024-09-21T06:34:55.728437+0000","last_active":"2024-09-21T06:34:58.184521+0000","last_peered":"2024-09-21T06:34:58.184521+0000","last_clean":"2024-09-21T06:34:58.184521+0000","last_became_active":"2024-09-21T06:34:55.728111+0000","last_became_peered":"2024-09-21T06:34:55.728111+0000","last_unstale":"2024-09-21T06:34:58.184521+0000","last_undegraded":"2024-09-21T06:34:58.184521+0000","last_fullsized":"2024-09-21T06:34:58.184521+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:15.330591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897340+0000","last_change":"2024-09-21T06:34:54.710224+0000","last_active":"2024-09-21T06:34:56.897340+0000","last_peered":"2024-09-21T06:34:56.897340+0000","last_clean":"2024-09-21T06:34:56.897340+0000","last_became_active":"2024-09-21T06:34:54.709896+0000","last_became_peered":"2024-09-21T06:34:54.709896+0000","last_unstale":"2024-09-21T06:34:56.897340+0000","last_undegraded":"2024-09-21T06:34:56.897340+0000","last_fullsized":"2024-09-21T06:34:56.897340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:16.511663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491563+0000","last_change":"2024-09-21T06:34:55.708497+0000","last_active":"2024-09-21T06:34:57.491563+0000","last_peered":"2024-09-21T06:34:57.491563+0000","last_clean":"2024-09-21T06:34:57.491563+0000","last_became_active":"2024-09-21T06:34:55.706136+0000","last_became_peered":"2024-09-21T06:34:55.706136+0000","last_unstale":"2024-09-21T06:34:57.491563+0000","last_undegraded":"2024-09-21T06:34:57.491563+0000","last_fullsized":"2024-09-21T06:34:57.491563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:06.652871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693236+0000","last_change":"2024-09-21T06:34:54.712196+0000","last_active":"2024-09-21T06:34:56.693236+0000","last_peered":"2024-09-21T06:34:56.693236+0000","last_clean":"2024-09-21T06:34:56.693236+0000","last_became_active":"2024-09-21T06:34:54.711507+0000","last_became_peered":"2024-09-21T06:34:54.711507+0000","last_unstale":"2024-09-21T06:34:56.693236+0000","last_undegraded":"2024-09-21T06:34:56.693236+0000","last_fullsized":"2024-09-21T06:34:56.693236+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:07.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184434+0000","last_change":"2024-09-21T06:34:55.718899+0000","last_active":"2024-09-21T06:34:58.184434+0000","last_peered":"2024-09-21T06:34:58.184434+0000","last_clean":"2024-09-21T06:34:58.184434+0000","last_became_active":"2024-09-21T06:34:55.718610+0000","last_became_peered":"2024-09-21T06:34:55.718610+0000","last_unstale":"2024-09-21T06:34:58.184434+0000","last_undegraded":"2024-09-21T06:34:58.184434+0000","last_fullsized":"2024-09-21T06:34:58.184434+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:00.211260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692044+0000","last_change":"2024-09-21T06:34:54.704910+0000","last_active":"2024-09-21T06:34:56.692044+0000","last_peered":"2024-09-21T06:34:56.692044+0000","last_clean":"2024-09-21T06:34:56.692044+0000","last_became_active":"2024-09-21T06:34:54.703410+0000","last_became_peered":"2024-09-21T06:34:54.703410+0000","last_unstale":"2024-09-21T06:34:56.692044+0000","last_undegraded":"2024-09-21T06:34:56.692044+0000","last_fullsized":"2024-09-21T06:34:56.692044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:36.483455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184330+0000","last_change":"2024-09-21T06:34:55.726247+0000","last_active":"2024-09-21T06:34:58.184330+0000","last_peered":"2024-09-21T06:34:58.184330+0000","last_clean":"2024-09-21T06:34:58.184330+0000","last_became_active":"2024-09-21T06:34:55.725850+0000","last_became_peered":"2024-09-21T06:34:55.725850+0000","last_unstale":"2024-09-21T06:34:58.184330+0000","last_undegraded":"2024-09-21T06:34:58.184330+0000","last_fullsized":"2024-09-21T06:34:58.184330+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:00.730957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897179+0000","last_change":"2024-09-21T06:34:54.712284+0000","last_active":"2024-09-21T06:34:56.897179+0000","last_peered":"2024-09-21T06:34:56.897179+0000","last_clean":"2024-09-21T06:34:56.897179+0000","last_became_active":"2024-09-21T06:34:54.711828+0000","last_became_peered":"2024-09-21T06:34:54.711828+0000","last_unstale":"2024-09-21T06:34:56.897179+0000","last_undegraded":"2024-09-21T06:34:56.897179+0000","last_fullsized":"2024-09-21T06:34:56.897179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:29.075613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491685+0000","last_change":"2024-09-21T06:34:55.713268+0000","last_active":"2024-09-21T06:34:57.491685+0000","last_peered":"2024-09-21T06:34:57.491685+0000","last_clean":"2024-09-21T06:34:57.491685+0000","last_became_active":"2024-09-21T06:34:55.712940+0000","last_became_peered":"2024-09-21T06:34:55.712940+0000","last_unstale":"2024-09-21T06:34:57.491685+0000","last_undegraded":"2024-09-21T06:34:57.491685+0000","last_fullsized":"2024-09-21T06:34:57.491685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:20.633620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692250+0000","last_change":"2024-09-21T06:34:54.706724+0000","last_active":"2024-09-21T06:34:56.692250+0000","last_peered":"2024-09-21T06:34:56.692250+0000","last_clean":"2024-09-21T06:34:56.692250+0000","last_became_active":"2024-09-21T06:34:54.706045+0000","last_became_peered":"2024-09-21T06:34:54.706045+0000","last_unstale":"2024-09-21T06:34:56.692250+0000","last_undegraded":"2024-09-21T06:34:56.692250+0000","last_fullsized":"2024-09-21T06:34:56.692250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.184525+0000","last_change":"2024-09-21T06:34:55.716497+0000","last_active":"2024-09-21T06:34:58.184525+0000","last_peered":"2024-09-21T06:34:58.184525+0000","last_clean":"2024-09-21T06:34:58.184525+0000","last_became_active":"2024-09-21T06:34:55.716182+0000","last_became_peered":"2024-09-21T06:34:55.716182+0000","last_unstale":"2024-09-21T06:34:58.184525+0000","last_undegraded":"2024-09-21T06:34:58.184525+0000","last_fullsized":"2024-09-21T06:34:58.184525+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:26.825309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693187+0000","last_change":"2024-09-21T06:34:54.704018+0000","last_active":"2024-09-21T06:34:56.693187+0000","last_peered":"2024-09-21T06:34:56.693187+0000","last_clean":"2024-09-21T06:34:56.693187+0000","last_became_active":"2024-09-21T06:34:54.703663+0000","last_became_peered":"2024-09-21T06:34:54.703663+0000","last_unstale":"2024-09-21T06:34:56.693187+0000","last_undegraded":"2024-09-21T06:34:56.693187+0000","last_fullsized":"2024-09-21T06:34:56.693187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:06.524348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183247+0000","last_change":"2024-09-21T06:34:55.712873+0000","last_active":"2024-09-21T06:34:58.183247+0000","last_peered":"2024-09-21T06:34:58.183247+0000","last_clean":"2024-09-21T06:34:58.183247+0000","last_became_active":"2024-09-21T06:34:55.711695+0000","last_became_peered":"2024-09-21T06:34:55.711695+0000","last_unstale":"2024-09-21T06:34:58.183247+0000","last_undegraded":"2024-09-21T06:34:58.183247+0000","last_fullsized":"2024-09-21T06:34:58.183247+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692851+0000","last_change":"2024-09-21T06:34:54.709868+0000","last_active":"2024-09-21T06:34:56.692851+0000","last_peered":"2024-09-21T06:34:56.692851+0000","last_clean":"2024-09-21T06:34:56.692851+0000","last_became_active":"2024-09-21T06:34:54.708447+0000","last_became_peered":"2024-09-21T06:34:54.708447+0000","last_unstale":"2024-09-21T06:34:56.692851+0000","last_undegraded":"2024-09-21T06:34:56.692851+0000","last_fullsized":"2024-09-21T06:34:56.692851+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:50.904913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.693744+0000","last_change":"2024-09-21T06:34:55.729064+0000","last_active":"2024-09-21T06:34:56.693744+0000","last_peered":"2024-09-21T06:34:56.693744+0000","last_clean":"2024-09-21T06:34:56.693744+0000","last_became_active":"2024-09-21T06:34:55.728795+0000","last_became_peered":"2024-09-21T06:34:55.728795+0000","last_unstale":"2024-09-21T06:34:56.693744+0000","last_undegraded":"2024-09-21T06:34:56.693744+0000","last_fullsized":"2024-09-21T06:34:56.693744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:38.265711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183569+0000","last_change":"2024-09-21T06:34:54.702424+0000","last_active":"2024-09-21T06:34:58.183569+0000","last_peered":"2024-09-21T06:34:58.183569+0000","last_clean":"2024-09-21T06:34:58.183569+0000","last_became_active":"2024-09-21T06:34:54.702065+0000","last_became_peered":"2024-09-21T06:34:54.702065+0000","last_unstale":"2024-09-21T06:34:58.183569+0000","last_undegraded":"2024-09-21T06:34:58.183569+0000","last_fullsized":"2024-09-21T06:34:58.183569+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:51.535195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683128+0000","last_change":"2024-09-21T06:34:55.718743+0000","last_active":"2024-09-21T06:34:57.683128+0000","last_peered":"2024-09-21T06:34:57.683128+0000","last_clean":"2024-09-21T06:34:57.683128+0000","last_became_active":"2024-09-21T06:34:55.703205+0000","last_became_peered":"2024-09-21T06:34:55.703205+0000","last_unstale":"2024-09-21T06:34:57.683128+0000","last_undegraded":"2024-09-21T06:34:57.683128+0000","last_fullsized":"2024-09-21T06:34:57.683128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:27.665102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491730+0000","last_change":"2024-09-21T06:34:54.711548+0000","last_active":"2024-09-21T06:34:57.491730+0000","last_peered":"2024-09-21T06:34:57.491730+0000","last_clean":"2024-09-21T06:34:57.491730+0000","last_became_active":"2024-09-21T06:34:54.711141+0000","last_became_peered":"2024-09-21T06:34:54.711141+0000","last_unstale":"2024-09-21T06:34:57.491730+0000","last_undegraded":"2024-09-21T06:34:57.491730+0000","last_fullsized":"2024-09-21T06:34:57.491730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:27.210366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491185+0000","last_change":"2024-09-21T06:34:55.712582+0000","last_active":"2024-09-21T06:34:57.491185+0000","last_peered":"2024-09-21T06:34:57.491185+0000","last_clean":"2024-09-21T06:34:57.491185+0000","last_became_active":"2024-09-21T06:34:55.710833+0000","last_became_peered":"2024-09-21T06:34:55.710833+0000","last_unstale":"2024-09-21T06:34:57.491185+0000","last_undegraded":"2024-09-21T06:34:57.491185+0000","last_fullsized":"2024-09-21T06:34:57.491185+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:40.561170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:58.183486+0000","last_change":"2024-09-21T06:34:54.713168+0000","last_active":"2024-09-21T06:34:58.183486+0000","last_peered":"2024-09-21T06:34:58.183486+0000","last_clean":"2024-09-21T06:34:58.183486+0000","last_became_active":"2024-09-21T06:34:54.712837+0000","last_became_peered":"2024-09-21T06:34:54.712837+0000","last_unstale":"2024-09-21T06:34:58.183486+0000","last_undegraded":"2024-09-21T06:34:58.183486+0000","last_fullsized":"2024-09-21T06:34:58.183486+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:29.731794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491694+0000","last_change":"2024-09-21T06:34:55.711645+0000","last_active":"2024-09-21T06:34:57.491694+0000","last_peered":"2024-09-21T06:34:57.491694+0000","last_clean":"2024-09-21T06:34:57.491694+0000","last_became_active":"2024-09-21T06:34:55.711312+0000","last_became_peered":"2024-09-21T06:34:55.711312+0000","last_unstale":"2024-09-21T06:34:57.491694+0000","last_undegraded":"2024-09-21T06:34:57.491694+0000","last_fullsized":"2024-09-21T06:34:57.491694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692192+0000","last_change":"2024-09-21T06:34:54.705515+0000","last_active":"2024-09-21T06:34:56.692192+0000","last_peered":"2024-09-21T06:34:56.692192+0000","last_clean":"2024-09-21T06:34:56.692192+0000","last_became_active":"2024-09-21T06:34:54.705153+0000","last_became_peered":"2024-09-21T06:34:54.705153+0000","last_unstale":"2024-09-21T06:34:56.692192+0000","last_undegraded":"2024-09-21T06:34:56.692192+0000","last_fullsized":"2024-09-21T06:34:56.692192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:35.586391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897683+0000","last_change":"2024-09-21T06:34:55.729077+0000","last_active":"2024-09-21T06:34:56.897683+0000","last_peered":"2024-09-21T06:34:56.897683+0000","last_clean":"2024-09-21T06:34:56.897683+0000","last_became_active":"2024-09-21T06:34:55.728776+0000","last_became_peered":"2024-09-21T06:34:55.728776+0000","last_unstale":"2024-09-21T06:34:56.897683+0000","last_undegraded":"2024-09-21T06:34:56.897683+0000","last_fullsized":"2024-09-21T06:34:56.897683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:03.331560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.683192+0000","last_change":"2024-09-21T06:34:54.703341+0000","last_active":"2024-09-21T06:34:57.683192+0000","last_peered":"2024-09-21T06:34:57.683192+0000","last_clean":"2024-09-21T06:34:57.683192+0000","last_became_active":"2024-09-21T06:34:54.703002+0000","last_became_peered":"2024-09-21T06:34:54.703002+0000","last_unstale":"2024-09-21T06:34:57.683192+0000","last_undegraded":"2024-09-21T06:34:57.683192+0000","last_fullsized":"2024-09-21T06:34:57.683192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:57.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.691146+0000","last_change":"2024-09-21T06:34:55.737366+0000","last_active":"2024-09-21T06:34:56.691146+0000","last_peered":"2024-09-21T06:34:56.691146+0000","last_clean":"2024-09-21T06:34:56.691146+0000","last_became_active":"2024-09-21T06:34:55.734444+0000","last_became_peered":"2024-09-21T06:34:55.734444+0000","last_unstale":"2024-09-21T06:34:56.691146+0000","last_undegraded":"2024-09-21T06:34:56.691146+0000","last_fullsized":"2024-09-21T06:34:56.691146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:47.619903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.692878+0000","last_change":"2024-09-21T06:34:54.702747+0000","last_active":"2024-09-21T06:34:56.692878+0000","last_peered":"2024-09-21T06:34:56.692878+0000","last_clean":"2024-09-21T06:34:56.692878+0000","last_became_active":"2024-09-21T06:34:54.702221+0000","last_became_peered":"2024-09-21T06:34:54.702221+0000","last_unstale":"2024-09-21T06:34:56.692878+0000","last_undegraded":"2024-09-21T06:34:56.692878+0000","last_fullsized":"2024-09-21T06:34:56.692878+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:26.752919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:56.897188+0000","last_change":"2024-09-21T06:34:55.724899+0000","last_active":"2024-09-21T06:34:56.897188+0000","last_peered":"2024-09-21T06:34:56.897188+0000","last_clean":"2024-09-21T06:34:56.897188+0000","last_became_active":"2024-09-21T06:34:55.724167+0000","last_became_peered":"2024-09-21T06:34:55.724167+0000","last_unstale":"2024-09-21T06:34:56.897188+0000","last_undegraded":"2024-09-21T06:34:56.897188+0000","last_fullsized":"2024-09-21T06:34:56.897188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:54.676087+0000","last_clean_scrub_stamp":"2024-09-21T06:34:54.676087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:18.175370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.491794+0000","last_change":"2024-09-21T06:34:54.707367+0000","last_active":"2024-09-21T06:34:57.491794+0000","last_peered":"2024-09-21T06:34:57.491794+0000","last_clean":"2024-09-21T06:34:57.491794+0000","last_became_active":"2024-09-21T06:34:54.705863+0000","last_became_peered":"2024-09-21T06:34:54.705863+0000","last_unstale":"2024-09-21T06:34:57.491794+0000","last_undegraded":"2024-09-21T06:34:57.491794+0000","last_fullsized":"2024-09-21T06:34:57.491794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:53.668255+0000","last_clean_scrub_stamp":"2024-09-21T06:34:53.668255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.687054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T06:34:57.682836+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:34:57.682836+0000","last_peered":"2024-09-21T06:34:57.682836+0000","last_clean":"2024-09-21T06:34:57.682836+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:34:57.682836+0000","last_undegraded":"2024-09-21T06:34:57.682836+0000","last_fullsized":"2024-09-21T06:34:57.682836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":595924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509529,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":596002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1186350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":596472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1884160,"data_stored":1186382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:05.735 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:35:05.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:35:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.004+0000 7fcdf1200640 1 -- 172.21.15.37:0/494214977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec06e330 msgr2=0x7fcdec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.004+0000 7fcdf1200640 1 --2- 172.21.15.37:0/494214977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec06e330 0x7fcdec06ede0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcdec09e110 tx=0x7fcde402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 -- 172.21.15.37:0/494214977 shutdown_connections 2024-09-21T06:35:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 --2- 172.21.15.37:0/494214977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec06e330 0x7fcdec06ede0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 --2- 172.21.15.37:0/494214977 >> v2:172.21.15.150:3300/0 conn(0x7fcdec073dd0 0x7fcdec06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 -- 172.21.15.37:0/494214977 >> 172.21.15.37:0/494214977 conn(0x7fcdec0cca40 msgr2=0x7fcdec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 -- 172.21.15.37:0/494214977 shutdown_connections 2024-09-21T06:35:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.005+0000 7fcdf1200640 1 -- 172.21.15.37:0/494214977 wait complete. 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.007+0000 7fcdf1200640 1 Processor -- start 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.007+0000 7fcdf1200640 1 -- start start 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 0x7fcdec1f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdec074cb0 0x7fcdec15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06f4a0 con 0x7fcdec073dd0 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06f320 con 0x7fcdec06e330 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.008+0000 7fcdf1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec06f620 con 0x7fcdec074cb0 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58388/0 (socket says 172.21.15.37:58388) 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 -- 172.21.15.37:0/373856751 learned_addr learned my addr 172.21.15.37:0/373856751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde9e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 0x7fcdec1f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcdea800640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdec074cb0 0x7fcdec15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 -- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdec074cb0 msgr2=0x7fcdec15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdec074cb0 0x7fcdec15ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 -- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 msgr2=0x7fcdec1f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 0x7fcdec1f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde3e00640 1 -- 172.21.15.37:0/373856751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec09f5a0 con 0x7fcdec073dd0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.009+0000 7fcde9e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 0x7fcdec1f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcde3e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400d6d0 tx=0x7fcdd400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4017070 con 0x7fcdec073dd0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd4009b10 con 0x7fcdec073dd0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4010460 con 0x7fcdec073dd0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec09f880 con 0x7fcdec073dd0 2024-09-21T06:35:06.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.010+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec09fe40 con 0x7fcdec073dd0 2024-09-21T06:35:06.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.011+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec09f310 con 0x7fcdec073dd0 2024-09-21T06:35:06.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.012+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdd401a070 con 0x7fcdec073dd0 2024-09-21T06:35:06.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.013+0000 7fcde2000640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.013+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fcdd409afe0 con 0x7fcdec073dd0 2024-09-21T06:35:06.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.014+0000 7fcde9e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.015+0000 7fcde9e00640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 0x7fcdc807b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000f60 tx=0x7fcddc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.018+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcdd40632a0 con 0x7fcdec073dd0 2024-09-21T06:35:06.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.292+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcdec0a10d0 con 0x7fcdec073dd0 2024-09-21T06:35:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.295+0000 7fcde2000640 1 -- 172.21.15.37:0/373856751 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fcdd4062ba0 con 0x7fcdec073dd0 2024-09-21T06:35:06.296 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-21T06:35:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQAtZ+5mFK97GhAA+lxyzseIntGlbFuGnmoszA== 2024-09-21T06:35:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-21T06:35:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-21T06:35:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:35:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-21T06:35:06.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 msgr2=0x7fcdc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 0x7fcdc807b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000f60 tx=0x7fcddc009040 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 msgr2=0x7fcdec0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400d6d0 tx=0x7fcdd400db90 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 shutdown_connections 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec073dd0 0x7fcdec0b0d40 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdec074cb0 0x7fcdec15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 --2- 172.21.15.37:0/373856751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdec06e330 0x7fcdec1f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 >> 172.21.15.37:0/373856751 conn(0x7fcdec0cca40 msgr2=0x7fcdec0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 shutdown_connections 2024-09-21T06:35:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.298+0000 7fcdf1200640 1 -- 172.21.15.37:0/373856751 wait complete. 2024-09-21T06:35:06.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.550+0000 7fed76600640 1 -- 172.21.15.37:0/1497902421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 msgr2=0x7fed7006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.550+0000 7fed76600640 1 --2- 172.21.15.37:0/1497902421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed7006ddf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009830 tx=0x7fed5c02a500 comp rx=0 tx=0).stop 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 -- 172.21.15.37:0/1497902421 shutdown_connections 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 --2- 172.21.15.37:0/1497902421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed7006ddf0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 --2- 172.21.15.37:0/1497902421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed70074d90 0x7fed7006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 -- 172.21.15.37:0/1497902421 >> 172.21.15.37:0/1497902421 conn(0x7fed700cca40 msgr2=0x7fed700cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:06.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 -- 172.21.15.37:0/1497902421 shutdown_connections 2024-09-21T06:35:06.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.551+0000 7fed76600640 1 -- 172.21.15.37:0/1497902421 wait complete. 2024-09-21T06:35:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.553+0000 7fed76600640 1 Processor -- start 2024-09-21T06:35:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.553+0000 7fed76600640 1 -- start start 2024-09-21T06:35:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed7006e330 0x7fed700c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed70074d90 0x7fed7015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed7015ad50 con 0x7fed70070030 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed7015abd0 con 0x7fed70074d90 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.554+0000 7fed76600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fed70070e80 con 0x7fed7006e330 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58406/0 (socket says 172.21.15.37:58406) 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 -- 172.21.15.37:0/1890584666 learned_addr learned my addr 172.21.15.37:0/1890584666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6f400640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed7006e330 0x7fed700c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6fe00640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed70074d90 0x7fed7015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 -- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed7006e330 msgr2=0x7fed700c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed7006e330 0x7fed700c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 -- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed70074d90 msgr2=0x7fed7015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed70074d90 0x7fed7015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.555+0000 7fed6ea00640 1 -- 172.21.15.37:0/1890584666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed701548e0 con 0x7fed70070030 2024-09-21T06:35:06.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed6ea00640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009950 tx=0x7fed5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0044e0 con 0x7fed70070030 2024-09-21T06:35:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5c005310 con 0x7fed70070030 2024-09-21T06:35:06.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c033e30 con 0x7fed70070030 2024-09-21T06:35:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed70154bc0 con 0x7fed70070030 2024-09-21T06:35:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.556+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed700c3160 con 0x7fed70070030 2024-09-21T06:35:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.557+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7006ddf0 con 0x7fed70070030 2024-09-21T06:35:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.558+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed5c003800 con 0x7fed70070030 2024-09-21T06:35:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.559+0000 7fed67400640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 0x7fed5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.559+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fed5c0bc1c0 con 0x7fed70070030 2024-09-21T06:35:06.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.560+0000 7fed6f400640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 0x7fed5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:06.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.560+0000 7fed6f400640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 0x7fed5007b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fed60005fd0 tx=0x7fed6000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.564+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed5c03f030 con 0x7fed70070030 2024-09-21T06:35:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.800+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed700735d0 con 0x7fed70070030 2024-09-21T06:35:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.800+0000 7fed67400640 1 -- 172.21.15.37:0/1890584666 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5234 (secure 0 0 0) 0x7fed7006e740 con 0x7fed70070030 2024-09-21T06:35:06.801 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":108,"btime":"2024-09-21T06:34:58:717858+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":8172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:34:58.226135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175]},"id":7}]} 2024-09-21T06:35:06.802 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 108 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.802+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 msgr2=0x7fed5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.802+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 0x7fed5007b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fed60005fd0 tx=0x7fed6000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.802+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 msgr2=0x7fed700c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.802+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009950 tx=0x7fed5c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 shutdown_connections 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fed50078b90 0x7fed5007b0a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed70070030 0x7fed700c2c20 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed70074d90 0x7fed7015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 --2- 172.21.15.37:0/1890584666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed7006e330 0x7fed700c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 >> 172.21.15.37:0/1890584666 conn(0x7fed700cca40 msgr2=0x7fed700a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 shutdown_connections 2024-09-21T06:35:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:06.803+0000 7fed76600640 1 -- 172.21.15.37:0/1890584666 wait complete. 2024-09-21T06:35:07.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:34:55.216486+0000', 'modified': '2024-09-21T06:34:58.226135+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/912339030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 912339030}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 912339030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8175, 'qdb_cluster': [8175]} 2024-09-21T06:35:07.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:35:07.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:35:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.463+0000 7fa48b200640 1 -- 172.21.15.37:0/3925275322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 msgr2=0x7fa468005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.463+0000 7fa48b200640 1 --2- 172.21.15.37:0/3925275322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa468005150 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ed0 tx=0x7fa468005830 comp rx=0 tx=0).stop 2024-09-21T06:35:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.463+0000 7fa489200640 1 -- 172.21.15.37:0/3925275322 reap_dead start 2024-09-21T06:35:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.464+0000 7fa48b200640 1 -- 172.21.15.37:0/3925275322 shutdown_connections 2024-09-21T06:35:07.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.464+0000 7fa48b200640 1 -- 172.21.15.37:0/3925275322 >> 172.21.15.37:0/3925275322 conn(0x7fa4840b1b90 msgr2=0x7fa4840b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.464+0000 7fa48b200640 1 -- 172.21.15.37:0/3925275322 shutdown_connections 2024-09-21T06:35:07.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.464+0000 7fa48b200640 1 -- 172.21.15.37:0/3925275322 wait complete. 2024-09-21T06:35:07.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.466+0000 7fa48b200640 1 Processor -- start 2024-09-21T06:35:07.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.466+0000 7fa48b200640 1 -- start start 2024-09-21T06:35:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.467+0000 7fa48b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.467+0000 7fa48b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4840b5200 0x7fa48415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa48b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4840c6190 0x7fa4841f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa48b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840749b0 con 0x7fa468004d40 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa48b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa484074830 con 0x7fa4840b5200 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa48b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa484161060 con 0x7fa4840c6190 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58424/0 (socket says 172.21.15.37:58424) 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 -- 172.21.15.37:0/776533337 learned_addr learned my addr 172.21.15.37:0/776533337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa47b400640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4840b5200 0x7fa48415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa489200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4840c6190 0x7fa4841f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 -- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4840c6190 msgr2=0x7fa4841f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4840c6190 0x7fa4841f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 -- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4840b5200 msgr2=0x7fa48415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4840b5200 0x7fa48415c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.468+0000 7fa483e00640 1 -- 172.21.15.37:0/776533337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4841583e0 con 0x7fa468004d40 2024-09-21T06:35:07.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa483e00640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa474011ae0 tx=0x7fa474009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa480c00640 1 -- 172.21.15.37:0/776533337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47401e070 con 0x7fa468004d40 2024-09-21T06:35:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa480c00640 1 -- 172.21.15.37:0/776533337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47400fe70 con 0x7fa468004d40 2024-09-21T06:35:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa484158680 con 0x7fa468004d40 2024-09-21T06:35:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa480c00640 1 -- 172.21.15.37:0/776533337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474004290 con 0x7fa468004d40 2024-09-21T06:35:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.469+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4840b3580 con 0x7fa468004d40 2024-09-21T06:35:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.471+0000 7fa480c00640 1 -- 172.21.15.37:0/776533337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa474021070 con 0x7fa468004d40 2024-09-21T06:35:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.473+0000 7fa480c00640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.473+0000 7fa480c00640 1 -- 172.21.15.37:0/776533337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa47409b970 con 0x7fa468004d40 2024-09-21T06:35:07.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.474+0000 7fa47b400640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.474+0000 7fa492437740 1 Processor -- start 2024-09-21T06:35:07.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.474+0000 7fa492437740 1 -- start start 2024-09-21T06:35:07.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.474+0000 7fa47b400640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 0x7fa46007b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000fb0 tx=0x7fa46c0067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b019f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac58590 0x56395b01a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56395afa3430 con 0x56395ac79270 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56395af935d0 con 0x56395ac503b0 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56395af82a00 con 0x56395ac58590 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.475+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58432/0 (socket says 172.21.15.37:58432) 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 -- 172.21.15.37:0/1128165258 learned_addr learned my addr 172.21.15.37:0/1128165258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa47b400640 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b019f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa489200640 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac58590 0x56395b01a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 -- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac58590 msgr2=0x56395b01a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac58590 0x56395b01a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 -- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 msgr2=0x56395b019f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b019f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 -- 172.21.15.37:0/1128165258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395af4b520 con 0x56395ac79270 2024-09-21T06:35:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa483e00640 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa47400f790 tx=0x7fa474063680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1db00efdacc8d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa478c00640 1 -- 172.21.15.37:0/1128165258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47405f8f0 con 0x56395ac79270 2024-09-21T06:35:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa478c00640 1 -- 172.21.15.37:0/1128165258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47406cad0 con 0x56395ac79270 2024-09-21T06:35:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.476+0000 7fa478c00640 1 -- 172.21.15.37:0/1128165258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47405fa90 con 0x56395ac79270 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 -- 172.21.15.37:0/1128165258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 msgr2=0x56395b064a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa47400f790 tx=0x7fa474063680 comp rx=0 tx=0).stop 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 -- 172.21.15.37:0/1128165258 shutdown_connections 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac58590 0x56395b01a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b019f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 --2- 172.21.15.37:0/1128165258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac79270 0x56395b064a20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 -- 172.21.15.37:0/1128165258 >> 172.21.15.37:0/1128165258 conn(0x56395ac6d3c0 msgr2=0x56395b01f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 -- 172.21.15.37:0/1128165258 shutdown_connections 2024-09-21T06:35:07.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.477+0000 7fa492437740 1 -- 172.21.15.37:0/1128165258 wait complete. 2024-09-21T06:35:07.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.478+0000 7fa492437740 1 -- start start 2024-09-21T06:35:07.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.479+0000 7fa492437740 1 Processor -- start 2024-09-21T06:35:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 0x56395b0415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56395af29330 con 0x56395ac503b0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56395af86810 con 0x56395ac58590 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56395acf3620 con 0x56395ac79270 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa492437740 10 client.0 _ll_register_callbacks cb 0x56395af2e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.480+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58444/0 (socket says 172.21.15.37:58444) 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 -- 172.21.15.37:0/4145721698 learned_addr learned my addr 172.21.15.37:0/4145721698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa489200640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 0x56395b0415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa47b400640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 -- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 msgr2=0x56395b0415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 0x56395b0415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 -- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 msgr2=0x56395b067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 -- 172.21.15.37:0/4145721698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395ace4850 con 0x56395ac503b0 2024-09-21T06:35:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa47b400640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa489200640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 0x56395b0415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa483e00640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa47400f790 tx=0x7fa474068210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccfd82a8fb3912c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa471600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474089b80 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa474089d20 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.481+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47406cad0 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fa474091050 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 10 client.8331 fetch_fsmap learned FSMap version 108 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x56395ad5b6b0 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47406cad0 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7fa4740629d0 con 0x56395ac503b0 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 10 client.8331 fetch_fsmap finished waiting for FSMap version 108 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 10 client.8331 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 20 client.8331 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac37e40 0x56395b077e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 4 client.8331 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:07.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56395af95520 con 0x56395ac37e40 2024-09-21T06:35:07.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.482+0000 7fa47b400640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac37e40 0x56395b077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.483+0000 7fa471600640 10 client.8331 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.483+0000 7fa47b400640 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac37e40 0x56395b077e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207d97b6a8f0e2e1 server_cookie=bb6371f30e1116a5 in_seq=0 out_seq=0 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa471600640 1 -- 172.21.15.37:0/4145721698 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56395af95520 con 0x56395ac37e40 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa471600640 10 client.8331 handle_command_reply: tid=0 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 client.8331 shutdown 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 msgr2=0x56395b0616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa47400f790 tx=0x7fa474068210 comp rx=0 tx=0).stop 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 shutdown_connections 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac37e40 0x56395b077e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac79270 0x56395b0415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 --2- 172.21.15.37:0/4145721698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0616b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa471600640 0 client.8331 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 >> 172.21.15.37:0/4145721698 conn(0x56395ac6d3c0 msgr2=0x56395b046580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.484+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 shutdown_connections 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.485+0000 7fa492437740 1 -- 172.21.15.37:0/4145721698 wait complete. 2024-09-21T06:35:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.485+0000 7fa492437740 20 client.8331 trim_cache size 0 max 16384 2024-09-21T06:35:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.504+0000 7fa492437740 1 Processor -- start 2024-09-21T06:35:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.504+0000 7fa492437740 1 -- start start 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b0642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841fd480 con 0x56395ac58590 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840cc210 con 0x56395ac503b0 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.505+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa4841fcf00 con 0x56395ac6d3c0 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58460/0 (socket says 172.21.15.37:58460) 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 -- 172.21.15.37:0/282692055 learned_addr learned my addr 172.21.15.37:0/282692055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa483e00640 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa47b400640 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b0642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 -- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 msgr2=0x56395b067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 -- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 msgr2=0x56395b0642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b0642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 -- 172.21.15.37:0/282692055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395b071430 con 0x56395ac58590 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.506+0000 7fa489200640 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02e020 tx=0x7fa47c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77994e00853c9091 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.507+0000 7fa473e00640 1 -- 172.21.15.37:0/282692055 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c045070 con 0x56395ac58590 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.507+0000 7fa473e00640 1 -- 172.21.15.37:0/282692055 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47c003c60 con 0x56395ac58590 2024-09-21T06:35:07.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.507+0000 7fa473e00640 1 -- 172.21.15.37:0/282692055 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c038040 con 0x56395ac58590 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.507+0000 7fa492437740 1 -- 172.21.15.37:0/282692055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 msgr2=0x56395b0647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.507+0000 7fa492437740 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02e020 tx=0x7fa47c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 -- 172.21.15.37:0/282692055 shutdown_connections 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac58590 0x56395b0647f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac503b0 0x56395b0642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 --2- 172.21.15.37:0/282692055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 -- 172.21.15.37:0/282692055 >> 172.21.15.37:0/282692055 conn(0x56395b061920 msgr2=0x56395b060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 -- 172.21.15.37:0/282692055 shutdown_connections 2024-09-21T06:35:07.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.508+0000 7fa492437740 1 -- 172.21.15.37:0/282692055 wait complete. 2024-09-21T06:35:07.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.509+0000 7fa492437740 1 -- start start 2024-09-21T06:35:07.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.509+0000 7fa492437740 1 Processor -- start 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56395af29330 con 0x56395ac503b0 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56395acf3620 con 0x56395ac58590 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56395af86810 con 0x56395ac6d3c0 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.510+0000 7fa492437740 10 client.0 _ll_register_callbacks cb 0x7fa4740044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:07.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58466/0 (socket says 172.21.15.37:58466) 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 -- 172.21.15.37:0/3538949461 learned_addr learned my addr 172.21.15.37:0/3538949461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa47b400640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa489200640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 -- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 msgr2=0x56395b067fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 -- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 msgr2=0x56395b067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 -- 172.21.15.37:0/3538949461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395b0823d0 con 0x56395ac503b0 2024-09-21T06:35:07.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa483e00640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa474099fe0 tx=0x7fa47405f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c80d353a961712c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.511+0000 7fa471600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474089d80 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa4740628e0 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47c003c60 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fa47406cad0 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 10 client.8337 fetch_fsmap learned FSMap version 108 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x56395b0eca90 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47c003c60 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7fa474062350 con 0x56395ac503b0 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 10 client.8337 fetch_fsmap finished waiting for FSMap version 108 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 10 client.8337 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 20 client.8337 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac0d280 0x56395b0e9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 4 client.8337 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:35:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.512+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "status"}) -- 0x56395af90fd0 con 0x56395ac0d280 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa47b400640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac0d280 0x56395b0e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa47b400640 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac0d280 0x56395b0e9320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfa01e0793bc2fa server_cookie=f140db19ece1aed in_seq=0 out_seq=0 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa471600640 10 client.8337 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa471600640 1 -- 172.21.15.37:0/3538949461 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x56395af90fd0 con 0x56395ac0d280 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa471600640 10 client.8337 handle_command_reply: tid=0 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa492437740 1 client.8337 shutdown 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 msgr2=0x56395b0e2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.513+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa474099fe0 tx=0x7fa47405f590 comp rx=0 tx=0).stop 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 shutdown_connections 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56395ac0d280 0x56395b0e9320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56395ac6d3c0 0x56395b067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395ac58590 0x56395b067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 --2- 172.21.15.37:0/3538949461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56395ac503b0 0x56395b0e2910 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa471600640 0 client.8337 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 >> 172.21.15.37:0/3538949461 conn(0x56395b061920 msgr2=0x56395b081a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 shutdown_connections 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 1 -- 172.21.15.37:0/3538949461 wait complete. 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.514+0000 7fa492437740 20 client.8337 trim_cache size 0 max 16384 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8175, 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:35:07.518 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 11.826280049999999, 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 108, 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 104, 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 104, 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 16.770574373999999 2024-09-21T06:35:07.519 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 msgr2=0x7fa46007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 0x7fa46007b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000fb0 tx=0x7fa46c0067a0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 msgr2=0x7fa4840b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa474011ae0 tx=0x7fa474009e90 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 shutdown_connections 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4840c6190 0x7fa4841f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4840b5200 0x7fa48415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 --2- 172.21.15.37:0/776533337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468004d40 0x7fa4840b2ed0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 >> 172.21.15.37:0/776533337 conn(0x7fa4840b1b90 msgr2=0x7fa4840aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 shutdown_connections 2024-09-21T06:35:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.515+0000 7fa48b200640 1 -- 172.21.15.37:0/776533337 wait complete. 2024-09-21T06:35:07.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:35:07.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 -- 172.21.15.37:0/1477880288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc06e500 msgr2=0x7f89fc0af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 --2- 172.21.15.37:0/1477880288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc06e500 0x7f89fc0af420 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009830 tx=0x7f89e402a5c0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 -- 172.21.15.37:0/1477880288 shutdown_connections 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 --2- 172.21.15.37:0/1477880288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc06e500 0x7f89fc0af420 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 -- 172.21.15.37:0/1477880288 >> 172.21.15.37:0/1477880288 conn(0x7f89fc0cca40 msgr2=0x7f89fc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 -- 172.21.15.37:0/1477880288 shutdown_connections 2024-09-21T06:35:07.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.764+0000 7f8a01600640 1 -- 172.21.15.37:0/1477880288 wait complete. 2024-09-21T06:35:07.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.767+0000 7f8a01600640 1 Processor -- start 2024-09-21T06:35:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.767+0000 7f8a01600640 1 -- start start 2024-09-21T06:35:07.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc09aa00 0x7f89fc1f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89fc09ae10 0x7f89fc1f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc0ce8a0 con 0x7f89fc15e7c0 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc0ce720 con 0x7f89fc09aa00 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f8a01600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f89fc0cea20 con 0x7f89fc09ae10 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f89faa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f89faa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58490/0 (socket says 172.21.15.37:58490) 2024-09-21T06:35:07.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.768+0000 7f89faa00640 1 -- 172.21.15.37:0/3198608316 learned_addr learned my addr 172.21.15.37:0/3198608316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 -- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89fc09ae10 msgr2=0x7f89fc1f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89fc09ae10 0x7f89fc1f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 -- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc09aa00 msgr2=0x7f89fc1f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc09aa00 0x7f89fc1f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 -- 172.21.15.37:0/3198608316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc09b570 con 0x7f89fc15e7c0 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.769+0000 7f89faa00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c430 tx=0x7f89ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.770+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec017070 con 0x7f89fc15e7c0 2024-09-21T06:35:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.770+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89ec0053e0 con 0x7f89fc15e7c0 2024-09-21T06:35:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.770+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec003850 con 0x7f89fc15e7c0 2024-09-21T06:35:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.770+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc09b850 con 0x7f89fc15e7c0 2024-09-21T06:35:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.770+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc09be10 con 0x7f89fc15e7c0 2024-09-21T06:35:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.771+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc0cb750 con 0x7f89fc15e7c0 2024-09-21T06:35:07.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.772+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89ec01a070 con 0x7f89fc15e7c0 2024-09-21T06:35:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.774+0000 7f89f2a00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 0x7f89d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.774+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f89ec09a2c0 con 0x7f89fc15e7c0 2024-09-21T06:35:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.774+0000 7f89f3e00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 0x7f89d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.775+0000 7f89f3e00640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 0x7f89d807b0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89e4034000 tx=0x7f89e40046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:07.778+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89ec062560 con 0x7f89fc15e7c0 2024-09-21T06:35:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.018+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89fc0b5f40 con 0x7f89fc15e7c0 2024-09-21T06:35:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.018+0000 7f89f2a00640 1 -- 172.21.15.37:0/3198608316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7f89ec061e60 con 0x7f89fc15e7c0 2024-09-21T06:35:08.019 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.020+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 msgr2=0x7f89d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.020+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 0x7f89d807b0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89e4034000 tx=0x7f89e40046c0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 msgr2=0x7f89fc1f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c430 tx=0x7f89ec00c8f0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 shutdown_connections 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f89d8078bc0 0x7f89d807b0d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89fc09ae10 0x7f89fc1f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc09aa00 0x7f89fc1f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 --2- 172.21.15.37:0/3198608316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89fc15e7c0 0x7f89fc1f4620 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 >> 172.21.15.37:0/3198608316 conn(0x7f89fc0cca40 msgr2=0x7f89fc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 shutdown_connections 2024-09-21T06:35:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08.021+0000 7f8a01600640 1 -- 172.21.15.37:0/3198608316 wait complete. 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:35:08.045 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:35:08.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.046 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-21T06:35:08.082 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:35:08.082 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:35:08.083 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42283sec preferred_lft 42283sec 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::281b:c9ff:feb9:7dbd/64 scope link 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.084 INFO:teuthology.orchestra.run.smithi037.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:24:09:2a:95:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f824:9ff:fe2a:95a8/64 scope link 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether da:d9:b0:69:d7:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d8d9:b0ff:fe69:d7de/64 scope link 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:35:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.085 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:35:08.176 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-21T06:35:08.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.179 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-21T06:35:08.215 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-21T06:35:08.215 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:35:08.215 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:35:08.215 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:35:08.215 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:35:08.216 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-21T06:35:08.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:35:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:08.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.413 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:35:08.448 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:35:08.448 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:35:08.448 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:35:08.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.449 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-21T06:35:08.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:35:08.545 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:35:08.580 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:35:08.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:35:08.580 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:35:08.656 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:35:08.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:35:08.854 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:35:08.854 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:35:08.854 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:35:08.874 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:35:08.874 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:35:08.874 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:35:08.875 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:35:08.875 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790" 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.150:6789,172.21.15.150:6790:/ 2024-09-21T06:35:08.905 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=13e9e1b2-9d52-4cf2-a477-95a0639530e0" will pass to kernel 2024-09-21T06:35:08.907 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:35:08.907 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:35:08.907 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:35:08.907 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:35:08.907 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:35:08.910 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:35:08.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:08.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:35:08.966 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:35:09.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:35:09.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:09.062 INFO:teuthology.orchestra.run.smithi037.stdout:client8349 2024-09-21T06:35:09.063 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:35:09.063 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:09.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T06:35:09.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 -- 172.21.15.37:0/4183940026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100ccfc0 msgr2=0x7fc2100b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 --2- 172.21.15.37:0/4183940026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100ccfc0 0x7fc2100b4f90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc2100cd790 tx=0x7fc1f802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 -- 172.21.15.37:0/4183940026 shutdown_connections 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 --2- 172.21.15.37:0/4183940026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100ccfc0 0x7fc2100b4f90 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 --2- 172.21.15.37:0/4183940026 >> v2:172.21.15.150:3300/0 conn(0x7fc210156c60 0x7fc2100cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 -- 172.21.15.37:0/4183940026 >> 172.21.15.37:0/4183940026 conn(0x7fc21006f920 msgr2=0x7fc2100b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 -- 172.21.15.37:0/4183940026 shutdown_connections 2024-09-21T06:35:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.369+0000 7fc20f200640 1 -- 172.21.15.37:0/4183940026 wait complete. 2024-09-21T06:35:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.371+0000 7fc20f200640 1 Processor -- start 2024-09-21T06:35:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.371+0000 7fc20f200640 1 -- start start 2024-09-21T06:35:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc21006e370 0x7fc2101598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210156c60 0x7fc2101f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc21006d560 con 0x7fc2101e7020 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc21006d3e0 con 0x7fc21006e370 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc21006d6e0 con 0x7fc210156c60 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc21006e370 0x7fc2101598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58502/0 (socket says 172.21.15.37:58502) 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc20e800640 1 -- 172.21.15.37:0/2793319048 learned_addr learned my addr 172.21.15.37:0/2793319048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.373+0000 7fc207e00640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210156c60 0x7fc2101f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 -- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210156c60 msgr2=0x7fc2101f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210156c60 0x7fc2101f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 -- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc21006e370 msgr2=0x7fc2101598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc21006e370 0x7fc2101598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 -- 172.21.15.37:0/2793319048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2101e7800 con 0x7fc2101e7020 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20e800640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e990 tx=0x7fc20800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208004d20 con 0x7fc2101e7020 2024-09-21T06:35:09.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc20800cb00 con 0x7fc2101e7020 2024-09-21T06:35:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208005020 con 0x7fc2101e7020 2024-09-21T06:35:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2101e7ae0 con 0x7fc2101e7020 2024-09-21T06:35:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.374+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2101e80a0 con 0x7fc2101e7020 2024-09-21T06:35:09.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.375+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2101431a0 con 0x7fc2101e7020 2024-09-21T06:35:09.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.377+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2080051c0 con 0x7fc2101e7020 2024-09-21T06:35:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.378+0000 7fc206000640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 0x7fc1ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.378+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc20809b7e0 con 0x7fc2101e7020 2024-09-21T06:35:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.378+0000 7fc20de00640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 0x7fc1ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.379+0000 7fc20de00640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 0x7fc1ec07b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc200001720 tx=0x7fc200009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:09.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.383+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2101431a0 con 0x7fc2101e7020 2024-09-21T06:35:09.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:09.620+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc2101e7590 con 0x7fc2101e7020 2024-09-21T06:35:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.314+0000 7fc206000640 1 -- 172.21.15.37:0/2793319048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v109) ==== 105+0+0 (secure 0 0 0) 0x7fc208016030 con 0x7fc2101e7020 2024-09-21T06:35:10.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 msgr2=0x7fc1ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 0x7fc1ec07b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc200001720 tx=0x7fc200009040 comp rx=0 tx=0).stop 2024-09-21T06:35:10.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 msgr2=0x7fc2100ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e990 tx=0x7fc20800ee50 comp rx=0 tx=0).stop 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 shutdown_connections 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1ec078bc0 0x7fc1ec07b0d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2101e7020 0x7fc2100ce3b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210156c60 0x7fc2101f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 --2- 172.21.15.37:0/2793319048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc21006e370 0x7fc2101598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 >> 172.21.15.37:0/2793319048 conn(0x7fc21006f920 msgr2=0x7fc210154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 shutdown_connections 2024-09-21T06:35:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.318+0000 7fc20f200640 1 -- 172.21.15.37:0/2793319048 wait complete. 2024-09-21T06:35:10.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.585+0000 7ff446600640 1 -- 172.21.15.37:0/795023197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 msgr2=0x7ff4400ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.585+0000 7ff446600640 1 --2- 172.21.15.37:0/795023197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ccce0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff4400b36f0 tx=0x7ff43802a980 comp rx=0 tx=0).stop 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 -- 172.21.15.37:0/795023197 shutdown_connections 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 --2- 172.21.15.37:0/795023197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ccce0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 --2- 172.21.15.37:0/795023197 >> v2:172.21.15.150:3300/0 conn(0x7ff44006e160 0x7ff44006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 --2- 172.21.15.37:0/795023197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44015dfe0 0x7ff440160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 -- 172.21.15.37:0/795023197 >> 172.21.15.37:0/795023197 conn(0x7ff44006f940 msgr2=0x7ff4400b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 -- 172.21.15.37:0/795023197 shutdown_connections 2024-09-21T06:35:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.586+0000 7ff446600640 1 -- 172.21.15.37:0/795023197 wait complete. 2024-09-21T06:35:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.588+0000 7ff446600640 1 Processor -- start 2024-09-21T06:35:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.588+0000 7ff446600640 1 -- start start 2024-09-21T06:35:10.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.589+0000 7ff446600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44006e160 0x7ff44013a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.589+0000 7ff446600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff446600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44015dfe0 0x7ff4400aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff446600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff44006d460 con 0x7ff4400cc8d0 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff446600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff44006d2e0 con 0x7ff44015dfe0 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff446600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff44006d5e0 con 0x7ff44006e160 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58530/0 (socket says 172.21.15.37:58530) 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 -- 172.21.15.37:0/2504191540 learned_addr learned my addr 172.21.15.37:0/2504191540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff445200640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44006e160 0x7ff44013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff445c00640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44015dfe0 0x7ff4400aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 -- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44006e160 msgr2=0x7ff44013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44006e160 0x7ff44013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 -- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44015dfe0 msgr2=0x7ff4400aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44015dfe0 0x7ff4400aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.590+0000 7ff43fe00640 1 -- 172.21.15.37:0/2504191540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4400ad9f0 con 0x7ff4400cc8d0 2024-09-21T06:35:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff43fe00640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c970 tx=0x7ff42c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c007bb0 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff42c007e40 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00f6b0 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4400adcd0 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.591+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4400ae290 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.592+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44006db00 con 0x7ff4400cc8d0 2024-09-21T06:35:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.593+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff42c00f850 con 0x7ff4400cc8d0 2024-09-21T06:35:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.594+0000 7ff43cc00640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 0x7ff42007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:10.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.595+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7ff42c09b460 con 0x7ff4400cc8d0 2024-09-21T06:35:10.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.595+0000 7ff445200640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 0x7ff42007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:10.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.596+0000 7ff445200640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 0x7ff42007b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff434004630 tx=0x7ff43400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.599+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff42c014030 con 0x7ff4400cc8d0 2024-09-21T06:35:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.839+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4401f9a50 con 0x7ff4400cc8d0 2024-09-21T06:35:10.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.840+0000 7ff43cc00640 1 -- 172.21.15.37:0/2504191540 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5250 (secure 0 0 0) 0x7ff42c0630b0 con 0x7ff4400cc8d0 2024-09-21T06:35:10.842 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:10.842 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"btime":"2024-09-21T06:35:10:319237+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:10.319217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8175,"mds_1":8172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:creating","state_seq":1,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175]},"id":7}]} 2024-09-21T06:35:10.842 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-09-21T06:35:10.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 msgr2=0x7ff42007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 0x7ff42007b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff434004630 tx=0x7ff43400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 msgr2=0x7ff4400ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c970 tx=0x7ff42c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 shutdown_connections 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff420078b90 0x7ff42007b0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4400cc8d0 0x7ff4400ae7b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44015dfe0 0x7ff4400aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 --2- 172.21.15.37:0/2504191540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff44006e160 0x7ff44013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 >> 172.21.15.37:0/2504191540 conn(0x7ff44006f940 msgr2=0x7ff4400b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 shutdown_connections 2024-09-21T06:35:10.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:10.842+0000 7ff446600640 1 -- 172.21.15.37:0/2504191540 wait complete. 2024-09-21T06:35:11.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:34:55.216486+0000', 'modified': '2024-09-21T06:35:10.319217+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8175, 'mds_1': 8172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8172': {'gid': 8172, 'name': 'c', 'rank': 1, 'incarnation': 110, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.37:6833/1950391294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 1950391294}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 1950391294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/912339030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 912339030}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 912339030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8175, 'qdb_cluster': [8175]} 2024-09-21T06:35:11.269 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8172:up:creating 2024-09-21T06:35:12.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 -- 172.21.15.37:0/3548429256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 msgr2=0x7f883806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 --2- 172.21.15.37:0/3548429256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883806ddf0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f88380ccf90 tx=0x7f882402a560 comp rx=0 tx=0).stop 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 -- 172.21.15.37:0/3548429256 shutdown_connections 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 --2- 172.21.15.37:0/3548429256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883806ddf0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 --2- 172.21.15.37:0/3548429256 >> v2:172.21.15.150:3300/0 conn(0x7f8838073f20 0x7f883815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 -- 172.21.15.37:0/3548429256 >> 172.21.15.37:0/3548429256 conn(0x7f8838073400 msgr2=0x7f8838073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 -- 172.21.15.37:0/3548429256 shutdown_connections 2024-09-21T06:35:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.506+0000 7f883ee00640 1 -- 172.21.15.37:0/3548429256 wait complete. 2024-09-21T06:35:12.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.509+0000 7f883ee00640 1 Processor -- start 2024-09-21T06:35:12.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.509+0000 7f883ee00640 1 -- start start 2024-09-21T06:35:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f883806e330 0x7f88381526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8838073f20 0x7f88380c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8838160700 con 0x7f88380afb80 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88380cfb80 con 0x7f8838073f20 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883ee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8838160880 con 0x7f883806e330 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58550/0 (socket says 172.21.15.37:58550) 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.510+0000 7f883e400640 1 -- 172.21.15.37:0/2670912937 learned_addr learned my addr 172.21.15.37:0/2670912937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883d000640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8838073f20 0x7f88380c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883da00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f883806e330 0x7f88381526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:12.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 -- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f883806e330 msgr2=0x7f88381526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:12.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f883806e330 0x7f88381526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 -- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8838073f20 msgr2=0x7f88380c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8838073f20 0x7f88380c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 -- 172.21.15.37:0/2670912937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380b37d0 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f883e400640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f883000ba10 tx=0x7f883000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.511+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830019070 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.512+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883000cc30 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.512+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8838152bf0 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.512+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000ce60 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.512+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88380b3d90 con 0x7f88380afb80 2024-09-21T06:35:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.513+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88380c58a0 con 0x7f88380afb80 2024-09-21T06:35:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.514+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f883001c070 con 0x7f88380afb80 2024-09-21T06:35:12.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.515+0000 7f8835600640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 0x7f881807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:12.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.515+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f883009aa70 con 0x7f88380afb80 2024-09-21T06:35:12.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.516+0000 7f883da00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 0x7f881807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.518+0000 7f883da00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 0x7f881807b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f882c00a490 tx=0x7f882c00a420 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.519+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f88380c58a0 con 0x7f88380afb80 2024-09-21T06:35:12.677 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:35:12.675+0000 7f3a2a800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T06:35:12.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.753+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8838160dc0 con 0x7f88380afb80 2024-09-21T06:35:12.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.754+0000 7f8835600640 1 -- 172.21.15.37:0/2670912937 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5272 (secure 0 0 0) 0x7f88300626c0 con 0x7f88380afb80 2024-09-21T06:35:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:12.756 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":113,"btime":"2024-09-21T06:35:12:331888+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:12.331878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[7079,8172,8175]},"id":7}]} 2024-09-21T06:35:12.756 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 113 2024-09-21T06:35:12.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 msgr2=0x7f881807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:12.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 0x7f881807b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f882c00a490 tx=0x7f882c00a420 comp rx=0 tx=0).stop 2024-09-21T06:35:12.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 msgr2=0x7f883815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f883000ba10 tx=0x7f883000bed0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 shutdown_connections 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8818078b90 0x7f881807b0a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88380afb80 0x7f883815e770 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8838073f20 0x7f88380c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 --2- 172.21.15.37:0/2670912937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f883806e330 0x7f88381526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 >> 172.21.15.37:0/2670912937 conn(0x7f8838073400 msgr2=0x7f88380aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 shutdown_connections 2024-09-21T06:35:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:12.756+0000 7f883ee00640 1 -- 172.21.15.37:0/2670912937 wait complete. 2024-09-21T06:35:13.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:34:55.216486+0000', 'modified': '2024-09-21T06:35:12.331878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8175, 'mds_1': 8172, 'mds_2': 7079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7079': {'gid': 7079, 'name': 'b', 'rank': 2, 'incarnation': 112, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.150:6837/1287330873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6835', 'nonce': 1287330873}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1287330873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8172': {'gid': 8172, 'name': 'c', 'rank': 1, 'incarnation': 110, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.37:6833/1950391294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 1950391294}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 1950391294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/912339030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 912339030}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 912339030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8175, 'qdb_cluster': [7079, 8172, 8175]} 2024-09-21T06:35:13.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T06:35:13.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 -- 172.21.15.37:0/2394363914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb0156a50 msgr2=0x7f5eb0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 --2- 172.21.15.37:0/2394363914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb0156a50 0x7f5eb0074620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0074160 tx=0x7f5ea002a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 -- 172.21.15.37:0/2394363914 shutdown_connections 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 --2- 172.21.15.37:0/2394363914 >> v2:172.21.15.150:3300/0 conn(0x7f5eb0074b60 0x7f5eb006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 --2- 172.21.15.37:0/2394363914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb0156a50 0x7f5eb0074620 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.411+0000 7f5eb7000640 1 -- 172.21.15.37:0/2394363914 >> 172.21.15.37:0/2394363914 conn(0x7f5eb00b1b90 msgr2=0x7f5eb00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.412+0000 7f5eb7000640 1 -- 172.21.15.37:0/2394363914 shutdown_connections 2024-09-21T06:35:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.412+0000 7f5eb7000640 1 -- 172.21.15.37:0/2394363914 wait complete. 2024-09-21T06:35:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.414+0000 7f5eb7000640 1 Processor -- start 2024-09-21T06:35:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.414+0000 7f5eb7000640 1 -- start start 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5eb0074b60 0x7f5eb01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 0x7f5eb01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00ce500 con 0x7f5eb006e330 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00ce380 con 0x7f5eb0156a50 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.415+0000 7f5eb7000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb00ce680 con 0x7f5eb0074b60 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58566/0 (socket says 172.21.15.37:58566) 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 -- 172.21.15.37:0/197134855 learned_addr learned my addr 172.21.15.37:0/197134855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eaf400640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5eb0074b60 0x7f5eb01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eb5000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 0x7f5eb01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 -- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5eb0074b60 msgr2=0x7f5eb01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5eb0074b60 0x7f5eb01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 -- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 msgr2=0x7f5eb01571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 0x7f5eb01571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 -- 172.21.15.37:0/197134855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb0159f00 con 0x7f5eb006e330 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eb5000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 0x7f5eb01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.416+0000 7f5eafe00640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00099c0 tx=0x7f5ea00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.417+0000 7f5ea7e00640 1 -- 172.21.15.37:0/197134855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea003e030 con 0x7f5eb006e330 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.417+0000 7f5ea7e00640 1 -- 172.21.15.37:0/197134855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea00047a0 con 0x7f5eb006e330 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.417+0000 7f5ea7e00640 1 -- 172.21.15.37:0/197134855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0035040 con 0x7f5eb006e330 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.417+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb00b39f0 con 0x7f5eb006e330 2024-09-21T06:35:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.417+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb00b5550 con 0x7f5eb006e330 2024-09-21T06:35:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.419+0000 7f5ea7e00640 1 -- 172.21.15.37:0/197134855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ea0041030 con 0x7f5eb006e330 2024-09-21T06:35:13.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.420+0000 7f5ea7e00640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 0x7f5e9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.420+0000 7f5ea7e00640 1 -- 172.21.15.37:0/197134855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5ea00bb4f0 con 0x7f5eb006e330 2024-09-21T06:35:13.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.420+0000 7f5eaf400640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 0x7f5e9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.421+0000 7f5ebe0b0740 1 Processor -- start 2024-09-21T06:35:13.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.421+0000 7f5ebe0b0740 1 -- start start 2024-09-21T06:35:13.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.421+0000 7f5eaf400640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 0x7f5e9007b0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00bf900 tx=0x7f5e9c00b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94e14270 0x55db951e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db951e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db9513e430 con 0x55db94df3590 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db9512e5d0 con 0x55db94deb3b0 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db9511da00 con 0x55db94e14270 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.422+0000 7f5eb5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58580/0 (socket says 172.21.15.37:58580) 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 -- 172.21.15.37:0/2542414630 learned_addr learned my addr 172.21.15.37:0/2542414630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eaf400640 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db951e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eafe00640 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94e14270 0x55db951e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 -- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94e14270 msgr2=0x55db951e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94e14270 0x55db951e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 -- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 msgr2=0x55db951e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db951e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 -- 172.21.15.37:0/2542414630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db950e6520 con 0x55db94df3590 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.423+0000 7f5eb5000640 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001c40 tx=0x7f5ea8002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354671f0f2abfc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ea4c00640 1 -- 172.21.15.37:0/2542414630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8019070 con 0x55db94df3590 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ea4c00640 1 -- 172.21.15.37:0/2542414630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea8003b80 con 0x55db94df3590 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ea4c00640 1 -- 172.21.15.37:0/2542414630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800e830 con 0x55db94df3590 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2542414630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 msgr2=0x55db951b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001c40 tx=0x7f5ea8002100 comp rx=0 tx=0).stop 2024-09-21T06:35:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2542414630 shutdown_connections 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951b4f60 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db951e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2542414630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94e14270 0x55db951e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.424+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2542414630 >> 172.21.15.37:0/2542414630 conn(0x55db94e083c0 msgr2=0x55db951ba970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.425+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2542414630 shutdown_connections 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.425+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2542414630 wait complete. 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.426+0000 7f5ebe0b0740 1 -- start start 2024-09-21T06:35:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.426+0000 7f5ebe0b0740 1 Processor -- start 2024-09-21T06:35:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94deb3b0 0x55db95200010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db952121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db950c4330 con 0x55db94df3590 2024-09-21T06:35:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db95121810 con 0x55db94e14270 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db94e8e620 con 0x55db94deb3b0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5ebe0b0740 10 client.0 _ll_register_callbacks cb 0x55db950c9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58590/0 (socket says 172.21.15.37:58590) 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 -- 172.21.15.37:0/2407080627 learned_addr learned my addr 172.21.15.37:0/2407080627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eafe00640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94deb3b0 0x55db95200010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eb5000640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db952121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 -- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94deb3b0 msgr2=0x55db95200010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94deb3b0 0x55db95200010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 -- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 msgr2=0x55db952121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db952121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.428+0000 7f5eaf400640 1 -- 172.21.15.37:0/2407080627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db94e7f850 con 0x55db94df3590 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5eaf400640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0042e0 tx=0x7f5e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708d0c9036350d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5e99600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c003d70 con 0x55db94df3590 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea8003b80 con 0x55db94df3590 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5e9c011440 con 0x55db94df3590 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c030050 con 0x55db94df3590 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5ebe0b0740 10 client.8379 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.429+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55db94ef66b0 con 0x55db94df3590 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea8003b80 con 0x55db94df3590 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7f5e9c033020 con 0x55db94df3590 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 10 client.8379 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 10 client.8379 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 20 client.8379 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db94dd2e40 0x55db9520ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 4 client.8379 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db95130520 con 0x55db94dd2e40 2024-09-21T06:35:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.430+0000 7f5eafe00640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db94dd2e40 0x55db9520ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.431+0000 7f5eafe00640 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db94dd2e40 0x55db9520ef40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adf5faf228871f95 server_cookie=dc0f98df17c00153 in_seq=0 out_seq=0 2024-09-21T06:35:13.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.431+0000 7f5e99600640 10 client.8379 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.432+0000 7f5e99600640 1 -- 172.21.15.37:0/2407080627 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db95130520 con 0x55db94dd2e40 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.432+0000 7f5e99600640 10 client.8379 handle_command_reply: tid=0 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.432+0000 7f5ebe0b0740 1 client.8379 shutdown 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 msgr2=0x55db951db0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0042e0 tx=0x7f5e9c004730 comp rx=0 tx=0).stop 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 shutdown_connections 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db94dd2e40 0x55db9520ef40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94df3590 0x55db951db0d0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db952121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/2407080627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94deb3b0 0x55db95200010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5e99600640 0 client.8379 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 >> 172.21.15.37:0/2407080627 conn(0x55db94e083c0 msgr2=0x55db952023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 shutdown_connections 2024-09-21T06:35:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/2407080627 wait complete. 2024-09-21T06:35:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.433+0000 7f5ebe0b0740 20 client.8379 trim_cache size 0 max 16384 2024-09-21T06:35:13.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.452+0000 7f5ebe0b0740 1 Processor -- start 2024-09-21T06:35:13.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.452+0000 7f5ebe0b0740 1 -- start start 2024-09-21T06:35:13.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db952085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db951e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb02028f0 con 0x55db94deb3b0 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0201b50 con 0x55db94e14270 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb0202220 con 0x55db94df3590 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58596/0 (socket says 172.21.15.37:58596) 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 -- 172.21.15.37:0/815351796 learned_addr learned my addr 172.21.15.37:0/815351796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eb5000640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db951e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eaf400640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db952085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 -- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 msgr2=0x55db952085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db952085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 -- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 msgr2=0x55db951e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db951e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eafe00640 1 -- 172.21.15.37:0/815351796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db951b93d0 con 0x55db94deb3b0 2024-09-21T06:35:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.454+0000 7f5eaf400640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db952085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.455+0000 7f5eafe00640 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005fd0 tx=0x7f5ea0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8083252a4227314 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.455+0000 7f5e9be00640 1 -- 172.21.15.37:0/815351796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0048040 con 0x55db94deb3b0 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.455+0000 7f5e9be00640 1 -- 172.21.15.37:0/815351796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea00a9030 con 0x55db94deb3b0 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.455+0000 7f5e9be00640 1 -- 172.21.15.37:0/815351796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00831d0 con 0x55db94deb3b0 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/815351796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 msgr2=0x55db9520f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005fd0 tx=0x7f5ea0086040 comp rx=0 tx=0).stop 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/815351796 shutdown_connections 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94e14270 0x55db951e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db952085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/815351796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94deb3b0 0x55db9520f640 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/815351796 >> 172.21.15.37:0/815351796 conn(0x55db94e083c0 msgr2=0x55db951e1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/815351796 shutdown_connections 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.456+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/815351796 wait complete. 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.457+0000 7f5ebe0b0740 1 -- start start 2024-09-21T06:35:13.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.457+0000 7f5ebe0b0740 1 Processor -- start 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db95201630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db9520f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db95212f80 con 0x55db94e14270 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00b5290 con 0x55db94deb3b0 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55db951dee90 con 0x55db94df3590 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.458+0000 7f5ebe0b0740 10 client.0 _ll_register_callbacks cb 0x7f5e9c010e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-09-21T06:35:13.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 -- 172.21.15.37:0/1632413524 learned_addr learned my addr 172.21.15.37:0/1632413524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eafe00640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db95201630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eaf400640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db9520f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 -- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 msgr2=0x55db9520f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db9520f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 -- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 msgr2=0x55db95201630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db95201630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 -- 172.21.15.37:0/1632413524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db951bb650 con 0x55db94e14270 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5eb5000640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009010 tx=0x7f5ea8006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b5b6ea2d224f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.459+0000 7f5e99600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8019040 con 0x55db94e14270 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5ea800ea30 con 0x55db94e14270 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea00a9030 con 0x55db94e14270 2024-09-21T06:35:13.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f5ea802c050 con 0x55db94e14270 2024-09-21T06:35:13.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 10 client.8385 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:13.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55db95201b70 con 0x55db94e14270 2024-09-21T06:35:13.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea00a9030 con 0x55db94e14270 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7f5ea802f020 con 0x55db94e14270 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 10 client.8385 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 10 client.8385 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 20 client.8385 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db951e28b0 0x55db951e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 4 client.8385 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.460+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "status"}) -- 0x55db9512bfd0 con 0x55db951e28b0 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.461+0000 7f5eafe00640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db951e28b0 0x55db951e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.461+0000 7f5eafe00640 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db951e28b0 0x55db951e2cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ea3a163d6e39675 server_cookie=e59119cc9c28d83a in_seq=0 out_seq=0 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.461+0000 7f5e99600640 10 client.8385 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5e99600640 1 -- 172.21.15.37:0/1632413524 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55db9512bfd0 con 0x55db951e28b0 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5e99600640 10 client.8385 handle_command_reply: tid=0 2024-09-21T06:35:13.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 client.8385 shutdown 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 msgr2=0x55db9527fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009010 tx=0x7f5ea8006d80 comp rx=0 tx=0).stop 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 shutdown_connections 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55db951e28b0 0x55db951e2cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db94e14270 0x55db9527fa90 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db94df3590 0x55db9520f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5e99600640 0 client.8385 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 --2- 172.21.15.37:0/1632413524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db94deb3b0 0x55db95201630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 >> 172.21.15.37:0/1632413524 conn(0x55db94e083c0 msgr2=0x55db951dad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 shutdown_connections 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 1 -- 172.21.15.37:0/1632413524 wait complete. 2024-09-21T06:35:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.462+0000 7f5ebe0b0740 20 client.8385 trim_cache size 0 max 16384 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7079, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 2.1312763490000002, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 113, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 104, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 104, 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 22.727857225000001 2024-09-21T06:35:13.468 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 msgr2=0x7f5e9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 0x7f5e9007b0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00bf900 tx=0x7f5e9c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 msgr2=0x7f5eb01f4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00099c0 tx=0x7f5ea00039e0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 shutdown_connections 2024-09-21T06:35:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5e90078bc0 0x7f5e9007b0d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb0156a50 0x7f5eb01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5eb0074b60 0x7f5eb01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 --2- 172.21.15.37:0/197134855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb006e330 0x7f5eb01f4f60 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 >> 172.21.15.37:0/197134855 conn(0x7f5eb00b1b90 msgr2=0x7f5eb015a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 shutdown_connections 2024-09-21T06:35:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.463+0000 7f5eb7000640 1 -- 172.21.15.37:0/197134855 wait complete. 2024-09-21T06:35:13.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:35:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.712+0000 7f435e200640 1 -- 172.21.15.37:0/341345349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435806e330 msgr2=0x7f43580cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.712+0000 7f435e200640 1 --2- 172.21.15.37:0/341345349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435806e330 0x7f43580cdc70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f43580cca40 tx=0x7f435002aa30 comp rx=0 tx=0).stop 2024-09-21T06:35:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 -- 172.21.15.37:0/341345349 shutdown_connections 2024-09-21T06:35:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 --2- 172.21.15.37:0/341345349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435806e330 0x7f43580cdc70 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 --2- 172.21.15.37:0/341345349 >> v2:172.21.15.150:3300/0 conn(0x7f4358074b60 0x7f435806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 --2- 172.21.15.37:0/341345349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4358156a50 0x7f4358074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 -- 172.21.15.37:0/341345349 >> 172.21.15.37:0/341345349 conn(0x7f43580b1b90 msgr2=0x7f43580b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 -- 172.21.15.37:0/341345349 shutdown_connections 2024-09-21T06:35:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.713+0000 7f435e200640 1 -- 172.21.15.37:0/341345349 wait complete. 2024-09-21T06:35:13.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.715+0000 7f435e200640 1 Processor -- start 2024-09-21T06:35:13.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.715+0000 7f435e200640 1 -- start start 2024-09-21T06:35:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.716+0000 7f435e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f435806e330 0x7f4358155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.716+0000 7f435e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f435e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4358156a50 0x7f4358158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f435e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43580ce330 con 0x7f4358074b60 2024-09-21T06:35:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f435e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43580ce1b0 con 0x7f4358156a50 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f435e200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43580ce4b0 con 0x7f435806e330 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58626/0 (socket says 172.21.15.37:58626) 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 -- 172.21.15.37:0/3554763492 learned_addr learned my addr 172.21.15.37:0/3554763492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4357400640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f435806e330 0x7f4358155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4357e00640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4358156a50 0x7f4358158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 -- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f435806e330 msgr2=0x7f4358155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f435806e330 0x7f4358155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 -- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4358156a50 msgr2=0x7f4358158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4358156a50 0x7f4358158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.717+0000 7f4356a00640 1 -- 172.21.15.37:0/3554763492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4358157bd0 con 0x7f4358074b60 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f4356a00640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f434400e710 tx=0x7f434400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f434f400640 1 -- 172.21.15.37:0/3554763492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434400cab0 con 0x7f4358074b60 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f434f400640 1 -- 172.21.15.37:0/3554763492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f434400cd40 con 0x7f4358074b60 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4358157eb0 con 0x7f4358074b60 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f434f400640 1 -- 172.21.15.37:0/3554763492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344010690 con 0x7f4358074b60 2024-09-21T06:35:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.718+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4358158470 con 0x7f4358074b60 2024-09-21T06:35:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.720+0000 7f434f400640 1 -- 172.21.15.37:0/3554763492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4344010830 con 0x7f4358074b60 2024-09-21T06:35:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.721+0000 7f434f400640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 0x7f433807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.721+0000 7f434f400640 1 -- 172.21.15.37:0/3554763492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f434409b730 con 0x7f4358074b60 2024-09-21T06:35:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.722+0000 7f4365421740 1 Processor -- start 2024-09-21T06:35:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.722+0000 7f4357400640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 0x7f433807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.722+0000 7f4365421740 1 -- start start 2024-09-21T06:35:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4357400640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 0x7f433807b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4348009b30 tx=0x7f4348005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d02ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c59cfb9430 con 0x55c59cc663b0 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c59cfa95d0 con 0x55c59cc8f270 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.723+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55c59cf98a00 con 0x55c59cc6e590 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58628/0 (socket says 172.21.15.37:58628) 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 -- 172.21.15.37:0/1154383546 learned_addr learned my addr 172.21.15.37:0/1154383546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4357400640 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4357e00640 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d02ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 -- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 msgr2=0x55c59d02ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d02ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 -- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 msgr2=0x55c59d05f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 -- 172.21.15.37:0/1154383546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59cf61520 con 0x55c59cc663b0 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4356a00640 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f43440633a0 tx=0x7f4344062b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca4c5af18cd660c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.724+0000 7f4343400640 1 -- 172.21.15.37:0/1154383546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434406a270 con 0x55c59cc663b0 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4343400640 1 -- 172.21.15.37:0/1154383546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4344067e40 con 0x55c59cc663b0 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4343400640 1 -- 172.21.15.37:0/1154383546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344062890 con 0x55c59cc663b0 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 -- 172.21.15.37:0/1154383546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 msgr2=0x55c59d05feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f43440633a0 tx=0x7f4344062b80 comp rx=0 tx=0).stop 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 -- 172.21.15.37:0/1154383546 shutdown_connections 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d05feb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d02ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 --2- 172.21.15.37:0/1154383546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 -- 172.21.15.37:0/1154383546 >> 172.21.15.37:0/1154383546 conn(0x55c59cc833c0 msgr2=0x55c59d035970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 -- 172.21.15.37:0/1154383546 shutdown_connections 2024-09-21T06:35:13.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.725+0000 7f4365421740 1 -- 172.21.15.37:0/1154383546 wait complete. 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.726+0000 7f4365421740 1 -- start start 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.727+0000 7f4365421740 1 Processor -- start 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc663b0 0x55c59d0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d08d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c59cf3f330 con 0x55c59cc6e590 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c59cf9c810 con 0x55c59cc8f270 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55c59cd09620 con 0x55c59cc663b0 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4365421740 10 client.0 _ll_register_callbacks cb 0x55c59cf44420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.728+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58642/0 (socket says 172.21.15.37:58642) 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 -- 172.21.15.37:0/4080898440 learned_addr learned my addr 172.21.15.37:0/4080898440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4357e00640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d08d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4357400640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc663b0 0x55c59d0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 -- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc663b0 msgr2=0x55c59d0756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc663b0 0x55c59d0756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 -- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 msgr2=0x55c59d08d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d08d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 -- 172.21.15.37:0/4080898440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59ccfa850 con 0x55c59cc6e590 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4356a00640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f43440633a0 tx=0x7f4344062900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e04339cb3a25df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4337e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434407d020 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4344067e40 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.729+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f434405ed60 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f434406ccf0 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 10 client.8397 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55c59cd716b0 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4344067e40 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7f434405ef60 con 0x55c59cc6e590 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 10 client.8397 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 10 client.8397 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 20 client.8397 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59cc4de40 0x55c59d0899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 4 client.8397 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c59cfab520 con 0x55c59cc4de40 2024-09-21T06:35:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.730+0000 7f4357400640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59cc4de40 0x55c59d0899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.731+0000 7f4357400640 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59cc4de40 0x55c59d0899d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87c8e2610beb69dd server_cookie=b73439b72ea45ea8 in_seq=0 out_seq=0 2024-09-21T06:35:13.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.731+0000 7f4337e00640 10 client.8397 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:13.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4337e00640 1 -- 172.21.15.37:0/4080898440 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c59cfab520 con 0x55c59cc4de40 2024-09-21T06:35:13.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4337e00640 10 client.8397 handle_command_reply: tid=0 2024-09-21T06:35:13.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 client.8397 shutdown 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 msgr2=0x55c59d074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f43440633a0 tx=0x7f4344062900 comp rx=0 tx=0).stop 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 shutdown_connections 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59cc4de40 0x55c59d0899d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d074560 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d08d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 --2- 172.21.15.37:0/4080898440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc663b0 0x55c59d0756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4337e00640 0 client.8397 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.732+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 >> 172.21.15.37:0/4080898440 conn(0x55c59cc833c0 msgr2=0x55c59d07c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.733+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 shutdown_connections 2024-09-21T06:35:13.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.733+0000 7f4365421740 1 -- 172.21.15.37:0/4080898440 wait complete. 2024-09-21T06:35:13.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.733+0000 7f4365421740 20 client.8397 trim_cache size 0 max 16384 2024-09-21T06:35:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.752+0000 7f4365421740 1 Processor -- start 2024-09-21T06:35:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.752+0000 7f4365421740 1 -- start start 2024-09-21T06:35:13.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4358201940 con 0x55c59cc663b0 2024-09-21T06:35:13.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4358202010 con 0x55c59cc8f270 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f43580a0210 con 0x55c59cc6e590 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58658/0 (socket says 172.21.15.37:58658) 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 -- 172.21.15.37:0/3294550458 learned_addr learned my addr 172.21.15.37:0/3294550458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357e00640 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4356a00640 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 -- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 msgr2=0x55c59d0737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d0737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 -- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 msgr2=0x55c59d05a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.754+0000 7f4357400640 1 -- 172.21.15.37:0/3294550458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59d035020 con 0x55c59cc663b0 2024-09-21T06:35:13.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.755+0000 7f4357400640 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4348009840 tx=0x7f4348014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd0aa8659967d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.755+0000 7f4342a00640 1 -- 172.21.15.37:0/3294550458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348003bb0 con 0x55c59cc663b0 2024-09-21T06:35:13.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.755+0000 7f4342a00640 1 -- 172.21.15.37:0/3294550458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4348003d50 con 0x55c59cc663b0 2024-09-21T06:35:13.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.755+0000 7f4342a00640 1 -- 172.21.15.37:0/3294550458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434802a690 con 0x55c59cc663b0 2024-09-21T06:35:13.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 -- 172.21.15.37:0/3294550458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 msgr2=0x55c59d07c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4348009840 tx=0x7f4348014000 comp rx=0 tx=0).stop 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 -- 172.21.15.37:0/3294550458 shutdown_connections 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc8f270 0x55c59d05a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc6e590 0x55c59d0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 --2- 172.21.15.37:0/3294550458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc663b0 0x55c59d07c8c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 -- 172.21.15.37:0/3294550458 >> 172.21.15.37:0/3294550458 conn(0x55c59cc833c0 msgr2=0x55c59d0357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 -- 172.21.15.37:0/3294550458 shutdown_connections 2024-09-21T06:35:13.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.756+0000 7f4365421740 1 -- 172.21.15.37:0/3294550458 wait complete. 2024-09-21T06:35:13.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.757+0000 7f4365421740 1 -- start start 2024-09-21T06:35:13.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.757+0000 7f4365421740 1 Processor -- start 2024-09-21T06:35:13.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.758+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc663b0 0x55c59d0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc8f270 0x55c59d0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c59cf3f330 con 0x55c59cc6e590 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c59cf9c810 con 0x55c59cc663b0 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55c59d05b3f0 con 0x55c59cc8f270 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4365421740 10 client.0 _ll_register_callbacks cb 0x7f4344063270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58660/0 (socket says 172.21.15.37:58660) 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 -- 172.21.15.37:0/2754068316 learned_addr learned my addr 172.21.15.37:0/2754068316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4357400640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc663b0 0x55c59d0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4357e00640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc8f270 0x55c59d0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 -- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc8f270 msgr2=0x55c59d0737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc8f270 0x55c59d0737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 -- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc663b0 msgr2=0x55c59d0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc663b0 0x55c59d0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.759+0000 7f4356a00640 1 -- 172.21.15.37:0/2754068316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59d07b330 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4356a00640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4344060680 tx=0x7f434405f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ec85f5b6edbab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344073060 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f43440622f0 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348003d50 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f434407d040 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 10 client.8403 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55c59d0dfc90 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4348003d50 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7f434406c230 con 0x55c59cc6e590 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 10 client.8403 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 10 client.8403 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 20 client.8403 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59d0e2d60 0x55c59d09c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 4 client.8403 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.760+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "status"}) -- 0x55c59cfa6fd0 con 0x55c59d0e2d60 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.761+0000 7f4357400640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59d0e2d60 0x55c59d09c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.761+0000 7f4357400640 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59d0e2d60 0x55c59d09c4c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f818b6505c8b4e server_cookie=818579483da73736 in_seq=0 out_seq=0 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.761+0000 7f4337e00640 10 client.8403 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.761+0000 7f4337e00640 1 -- 172.21.15.37:0/2754068316 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55c59cfa6fd0 con 0x55c59d0e2d60 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.761+0000 7f4337e00640 10 client.8403 handle_command_reply: tid=0 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 client.8403 shutdown 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 msgr2=0x55c59d0df6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4344060680 tx=0x7f434405f470 comp rx=0 tx=0).stop 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 shutdown_connections 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55c59d0e2d60 0x55c59d09c4c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c59cc6e590 0x55c59d0df6f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4337e00640 0 client.8403 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c59cc8f270 0x55c59d0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 --2- 172.21.15.37:0/2754068316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c59cc663b0 0x55c59d0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 >> 172.21.15.37:0/2754068316 conn(0x55c59cc833c0 msgr2=0x55c59d07c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 shutdown_connections 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 1 -- 172.21.15.37:0/2754068316 wait complete. 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.762+0000 7f4365421740 20 client.8403 trim_cache size 0 max 16384 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8172, 2024-09-21T06:35:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 3.4412716620000001, 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 113, 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 104, 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 104, 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 23.024076000000001 2024-09-21T06:35:13.770 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 msgr2=0x7f433807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 0x7f433807b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4348009b30 tx=0x7f4348005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 msgr2=0x7f43580bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f434400e710 tx=0x7f434400ebd0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 shutdown_connections 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4338078b90 0x7f433807b0a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4358074b60 0x7f43580bc7e0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4358156a50 0x7f4358158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 --2- 172.21.15.37:0/3554763492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f435806e330 0x7f4358155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 >> 172.21.15.37:0/3554763492 conn(0x7f43580b1b90 msgr2=0x7f43580afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 shutdown_connections 2024-09-21T06:35:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:13.763+0000 7f435e200640 1 -- 172.21.15.37:0/3554763492 wait complete. 2024-09-21T06:35:13.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:35:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 -- 172.21.15.37:0/900765638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 msgr2=0x7fa4d80b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/900765638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d80b7710 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa4d808d450 tx=0x7fa4c802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 -- 172.21.15.37:0/900765638 shutdown_connections 2024-09-21T06:35:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/900765638 >> v2:172.21.15.150:3300/0 conn(0x7fa4d806e070 0x7fa4d81681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/900765638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d80b7710 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 -- 172.21.15.37:0/900765638 >> 172.21.15.37:0/900765638 conn(0x7fa4d80d1960 msgr2=0x7fa4d80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 -- 172.21.15.37:0/900765638 shutdown_connections 2024-09-21T06:35:14.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.017+0000 7fa4d7e00640 1 -- 172.21.15.37:0/900765638 wait complete. 2024-09-21T06:35:14.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.019+0000 7fa4d7e00640 1 Processor -- start 2024-09-21T06:35:14.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.019+0000 7fa4d7e00640 1 -- start start 2024-09-21T06:35:14.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 0x7fa4d81f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4d80ced40 0x7fa4d81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8168720 con 0x7fa4d80b36c0 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d806e7a0 con 0x7fa4d806e070 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d81688a0 con 0x7fa4d80ced40 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40992/0 (socket says 172.21.15.37:40992) 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 -- 172.21.15.37:0/693114886 learned_addr learned my addr 172.21.15.37:0/693114886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 0x7fa4d81f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d7400640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4d80ced40 0x7fa4d81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 -- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4d80ced40 msgr2=0x7fa4d81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4d80ced40 0x7fa4d81594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 -- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 msgr2=0x7fa4d81f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 0x7fa4d81f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4ce000640 1 -- 172.21.15.37:0/693114886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d81f6370 con 0x7fa4d80b36c0 2024-09-21T06:35:14.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.021+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 0x7fa4d81f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:14.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4ce000640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000c3a0 tx=0x7fa4c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4cf400640 1 -- 172.21.15.37:0/693114886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0017070 con 0x7fa4d80b36c0 2024-09-21T06:35:14.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4cf400640 1 -- 172.21.15.37:0/693114886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4c00053c0 con 0x7fa4d80b36c0 2024-09-21T06:35:14.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d81f55b0 con 0x7fa4d80b36c0 2024-09-21T06:35:14.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4cf400640 1 -- 172.21.15.37:0/693114886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0003850 con 0x7fa4d80b36c0 2024-09-21T06:35:14.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.022+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d81f5b70 con 0x7fa4d80b36c0 2024-09-21T06:35:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.025+0000 7fa4cf400640 1 -- 172.21.15.37:0/693114886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4c001a070 con 0x7fa4d80b36c0 2024-09-21T06:35:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.026+0000 7fa4cf400640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 0x7fa4b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.026+0000 7fa4cf400640 1 -- 172.21.15.37:0/693114886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa4c009a3d0 con 0x7fa4d80b36c0 2024-09-21T06:35:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.026+0000 7fa4e4dec740 1 Processor -- start 2024-09-21T06:35:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.026+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 0x7fa4b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.026+0000 7fa4e4dec740 1 -- start start 2024-09-21T06:35:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.027+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 0x7fa4b407b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8033000 tx=0x7fa4c8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556add32c430 con 0x556adcfd93b0 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556add31c5d0 con 0x556add002270 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556add30ba00 con 0x556adcfe1590 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40994/0 (socket says 172.21.15.37:40994) 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 -- 172.21.15.37:0/3219371070 learned_addr learned my addr 172.21.15.37:0/3219371070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4d7400640 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 -- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 msgr2=0x556add3a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 -- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 msgr2=0x556add3d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.028+0000 7fa4ce000640 1 -- 172.21.15.37:0/3219371070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556add2d4520 con 0x556adcfd93b0 2024-09-21T06:35:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.029+0000 7fa4ce000640 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0098640 tx=0x7fa4c0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b1eda7f20d5b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.029+0000 7fa4c7400640 1 -- 172.21.15.37:0/3219371070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0061e60 con 0x556adcfd93b0 2024-09-21T06:35:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.029+0000 7fa4c7400640 1 -- 172.21.15.37:0/3219371070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4c006b880 con 0x556adcfd93b0 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.029+0000 7fa4c7400640 1 -- 172.21.15.37:0/3219371070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0088170 con 0x556adcfd93b0 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 -- 172.21.15.37:0/3219371070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 msgr2=0x556add3d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0098640 tx=0x7fa4c0000ed0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 -- 172.21.15.37:0/3219371070 shutdown_connections 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add3d30a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/3219371070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 -- 172.21.15.37:0/3219371070 >> 172.21.15.37:0/3219371070 conn(0x556adcff63c0 msgr2=0x556add3a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 -- 172.21.15.37:0/3219371070 shutdown_connections 2024-09-21T06:35:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.030+0000 7fa4e4dec740 1 -- 172.21.15.37:0/3219371070 wait complete. 2024-09-21T06:35:14.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.031+0000 7fa4e4dec740 1 -- start start 2024-09-21T06:35:14.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.031+0000 7fa4e4dec740 1 Processor -- start 2024-09-21T06:35:14.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556add2b2330 con 0x556adcfd93b0 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556add30f810 con 0x556add002270 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556add07c620 con 0x556adcfe1590 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4e4dec740 10 client.0 _ll_register_callbacks cb 0x556add2b7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41004/0 (socket says 172.21.15.37:41004) 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2890304485 learned_addr learned my addr 172.21.15.37:0/2890304485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d7400640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 msgr2=0x556add3ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 msgr2=0x556add3b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2890304485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556add06d850 con 0x556adcfd93b0 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.033+0000 7fa4d7400640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009850 tx=0x7fa4c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511171bfdc417b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:14.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c804d070 con 0x556adcfd93b0 2024-09-21T06:35:14.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c006b880 con 0x556adcfd93b0 2024-09-21T06:35:14.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa4c8004bb0 con 0x556adcfd93b0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fa4c8058050 con 0x556adcfd93b0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4e4dec740 10 client.8418 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.034+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x556add0e46b0 con 0x556adcfd93b0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4c006b880 con 0x556adcfd93b0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7fa4c805b020 con 0x556adcfd93b0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 10 client.8418 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 10 client.8418 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 20 client.8418 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556adcfc0e40 0x556add3cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 4 client.8418 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556add31e520 con 0x556adcfc0e40 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.035+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556adcfc0e40 0x556add3cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.036+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556adcfc0e40 0x556add3cbe40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e139e3753a2457fb server_cookie=fecb7f0bc8d8f01 in_seq=0 out_seq=0 2024-09-21T06:35:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.036+0000 7fa4bbe00640 10 client.8418 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.036+0000 7fa4bbe00640 1 -- 172.21.15.37:0/2890304485 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556add31e520 con 0x556adcfc0e40 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.036+0000 7fa4bbe00640 10 client.8418 handle_command_reply: tid=0 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.036+0000 7fa4e4dec740 1 client.8418 shutdown 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 msgr2=0x556add4004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009850 tx=0x7fa4c8009260 comp rx=0 tx=0).stop 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 shutdown_connections 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556adcfc0e40 0x556add3cbe40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556add002270 0x556add3b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2890304485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adcfd93b0 0x556add4004c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4bbe00640 0 client.8418 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 >> 172.21.15.37:0/2890304485 conn(0x556adcff63c0 msgr2=0x556add3efab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 shutdown_connections 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2890304485 wait complete. 2024-09-21T06:35:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.037+0000 7fa4e4dec740 20 client.8418 trim_cache size 0 max 16384 2024-09-21T06:35:14.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.056+0000 7fa4e4dec740 1 Processor -- start 2024-09-21T06:35:14.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.056+0000 7fa4e4dec740 1 -- start start 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 0x556add3d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 0x556add410d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d82017a0 con 0x556add002270 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8201e70 con 0x556adcfe1590 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d8156930 con 0x556adcfd93b0 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41016/0 (socket says 172.21.15.37:41016) 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 -- 172.21.15.37:0/2831491634 learned_addr learned my addr 172.21.15.37:0/2831491634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 0x556add410d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 0x556add3d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 -- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 msgr2=0x556add3d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 0x556add3d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 -- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 msgr2=0x556add410d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.058+0000 7fa4d7400640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 0x556add410d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4d7400640 1 -- 172.21.15.37:0/2831491634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556add3fcd00 con 0x556add002270 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4ce000640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 0x556add410d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 0x556add3d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:14.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4d7400640 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009950 tx=0x7fa4d0009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38b136a512685c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4c6a00640 1 -- 172.21.15.37:0/2831491634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0004580 con 0x556add002270 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4c6a00640 1 -- 172.21.15.37:0/2831491634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d0004720 con 0x556add002270 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.059+0000 7fa4c6a00640 1 -- 172.21.15.37:0/2831491634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0004d80 con 0x556add002270 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2831491634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 msgr2=0x556add411260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009950 tx=0x7fa4d0009e10 comp rx=0 tx=0).stop 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2831491634 shutdown_connections 2024-09-21T06:35:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add411260 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfe1590 0x556add410d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/2831491634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfd93b0 0x556add3d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2831491634 >> 172.21.15.37:0/2831491634 conn(0x556adcff63c0 msgr2=0x556add400230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2831491634 shutdown_connections 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.060+0000 7fa4e4dec740 1 -- 172.21.15.37:0/2831491634 wait complete. 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.061+0000 7fa4e4dec740 1 -- start start 2024-09-21T06:35:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.061+0000 7fa4e4dec740 1 Processor -- start 2024-09-21T06:35:14.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.062+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfd93b0 0x556add3b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.062+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4e4dec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556add3bace0 con 0x556add002270 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556add4011f0 con 0x556adcfd93b0 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4e4dec740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556add3be130 con 0x556adcfe1590 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4e4dec740 10 client.0 _ll_register_callbacks cb 0x7fa4c80044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41030/0 (socket says 172.21.15.37:41030) 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 -- 172.21.15.37:0/870258161 learned_addr learned my addr 172.21.15.37:0/870258161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfd93b0 0x556add3b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4ce000640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 -- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 msgr2=0x556add3d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 -- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfd93b0 msgr2=0x556add3b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfd93b0 0x556add3b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.063+0000 7fa4d7400640 1 -- 172.21.15.37:0/870258161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556add3c8ba0 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4d7400640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004410 tx=0x7fa4d0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef05ac7dfeec1dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0004580 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa4d001ca30 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d0004720 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fa4d0003ca0 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4e4dec740 10 client.8424 fetch_fsmap learned FSMap version 113 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.064+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x556add3d26c0 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d0004720 con 0x556add002270 2024-09-21T06:35:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 113) ==== 3542+0+0 (secure 0 0 0) 0x7fa4d0011440 con 0x556add002270 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 10 client.8424 fetch_fsmap finished waiting for FSMap version 113 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 10 client.8424 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 20 client.8424 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556add46d660 0x556add3cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 4 client.8424 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "status"}) -- 0x556add319fd0 con 0x556add46d660 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556add46d660 0x556add3cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4d6a00640 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556add46d660 0x556add3cab60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cc3bcf25a6ed11 server_cookie=8f4fa088ac2c9f80 in_seq=0 out_seq=0 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.065+0000 7fa4bbe00640 10 client.8424 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4bbe00640 1 -- 172.21.15.37:0/870258161 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x556add319fd0 con 0x556add46d660 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4bbe00640 10 client.8424 handle_command_reply: tid=0 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 client.8424 shutdown 2024-09-21T06:35:14.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 msgr2=0x556add410b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004410 tx=0x7fa4d0005ea0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 shutdown_connections 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x556add46d660 0x556add3cab60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556add002270 0x556add410b90 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556adcfe1590 0x556add3d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 --2- 172.21.15.37:0/870258161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556adcfd93b0 0x556add3b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4bbe00640 0 client.8424 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 >> 172.21.15.37:0/870258161 conn(0x556adcff63c0 msgr2=0x556add3bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 shutdown_connections 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 1 -- 172.21.15.37:0/870258161 wait complete. 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.066+0000 7fa4e4dec740 20 client.8424 trim_cache size 0 max 16384 2024-09-21T06:35:14.071 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "13e9e1b2-9d52-4cf2-a477-95a0639530e0", 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8175, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 18.379719964, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 113, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 104, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 104, 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 23.324014265999999 2024-09-21T06:35:14.072 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 msgr2=0x7fa4b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 0x7fa4b407b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8033000 tx=0x7fa4c8004730 comp rx=0 tx=0).stop 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 msgr2=0x7fa4d8167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000c3a0 tx=0x7fa4c000c860 comp rx=0 tx=0).stop 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 shutdown_connections 2024-09-21T06:35:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa4b4078b90 0x7fa4b407b0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d80b36c0 0x7fa4d8167e20 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4d80ced40 0x7fa4d81594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 --2- 172.21.15.37:0/693114886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d806e070 0x7fa4d81f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 >> 172.21.15.37:0/693114886 conn(0x7fa4d80d1960 msgr2=0x7fa4d815a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 shutdown_connections 2024-09-21T06:35:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.067+0000 7fa4d7e00640 1 -- 172.21.15.37:0/693114886 wait complete. 2024-09-21T06:35:14.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 -- 172.21.15.37:0/4189191065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7580b4990 msgr2=0x7fe7580b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 --2- 172.21.15.37:0/4189191065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7580b4990 0x7fe7580b4da0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe758146280 tx=0x7fe75002aa10 comp rx=0 tx=0).stop 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 -- 172.21.15.37:0/4189191065 shutdown_connections 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 --2- 172.21.15.37:0/4189191065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7580b4990 0x7fe7580b4da0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 --2- 172.21.15.37:0/4189191065 >> v2:172.21.15.150:3300/0 conn(0x7fe7580cc8d0 0x7fe7580ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 --2- 172.21.15.37:0/4189191065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe75815cb20 0x7fe7580cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 -- 172.21.15.37:0/4189191065 >> 172.21.15.37:0/4189191065 conn(0x7fe75806f940 msgr2=0x7fe7580b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 -- 172.21.15.37:0/4189191065 shutdown_connections 2024-09-21T06:35:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.320+0000 7fe757200640 1 -- 172.21.15.37:0/4189191065 wait complete. 2024-09-21T06:35:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.322+0000 7fe757200640 1 Processor -- start 2024-09-21T06:35:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.322+0000 7fe757200640 1 -- start start 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7580b4990 0x7fe7581f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7580cc8d0 0x7fe7581f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe75806d3f0 con 0x7fe75815cb20 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe75806d270 con 0x7fe7580b4990 2024-09-21T06:35:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe757200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe75806d570 con 0x7fe7580cc8d0 2024-09-21T06:35:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe756800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe756800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41060/0 (socket says 172.21.15.37:41060) 2024-09-21T06:35:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.324+0000 7fe756800640 1 -- 172.21.15.37:0/679021943 learned_addr learned my addr 172.21.15.37:0/679021943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe755e00640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7580b4990 0x7fe7581f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe755400640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7580cc8d0 0x7fe7581f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 -- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7580cc8d0 msgr2=0x7fe7581f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7580cc8d0 0x7fe7581f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 -- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7580b4990 msgr2=0x7fe7581f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7580b4990 0x7fe7581f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 -- 172.21.15.37:0/679021943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75815c660 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe756800640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d60 tx=0x7fe750003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750033070 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe750004240 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750004630 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75815aa70 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.325+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75815b030 con 0x7fe75815cb20 2024-09-21T06:35:14.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.327+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7580b3200 con 0x7fe75815cb20 2024-09-21T06:35:14.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.328+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe750036070 con 0x7fe75815cb20 2024-09-21T06:35:14.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.329+0000 7fe74e000640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 0x7fe73807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:14.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.329+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe7500c0270 con 0x7fe75815cb20 2024-09-21T06:35:14.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.329+0000 7fe755e00640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 0x7fe73807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:14.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.330+0000 7fe755e00640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 0x7fe73807b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe748009ac0 tx=0x7fe748005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:14.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.334+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe7580b3200 con 0x7fe75815cb20 2024-09-21T06:35:14.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.572+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe75815c3f0 con 0x7fe75815cb20 2024-09-21T06:35:14.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.572+0000 7fe74e000640 1 -- 172.21.15.37:0/679021943 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5272 (secure 0 0 0) 0x7fe7500885a0 con 0x7fe75815cb20 2024-09-21T06:35:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:14.575 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":113,"btime":"2024-09-21T06:35:12:331888+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:12.331878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[7079,8172,8175]},"id":7}]} 2024-09-21T06:35:14.575 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 113 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 msgr2=0x7fe73807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 0x7fe73807b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe748009ac0 tx=0x7fe748005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 msgr2=0x7fe7581599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d60 tx=0x7fe750003ef0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 shutdown_connections 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe738078b90 0x7fe73807b0a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe75815cb20 0x7fe7581599c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7580cc8d0 0x7fe7581f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 --2- 172.21.15.37:0/679021943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7580b4990 0x7fe7581f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 >> 172.21.15.37:0/679021943 conn(0x7fe75806f940 msgr2=0x7fe75815e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 shutdown_connections 2024-09-21T06:35:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:14.575+0000 7fe757200640 1 -- 172.21.15.37:0/679021943 wait complete. 2024-09-21T06:35:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:14.603 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:35:14.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:14.650 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-21T06:35:14.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:14.731 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-21T06:35:14.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:14.809 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-21T06:35:14.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:14.885 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-21T06:35:14.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:35:15.247 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:35:15.248 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:35:15.248 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:35:17.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:35:17.507 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T06:35:17.507 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:35:17.508 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:35:19.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T06:35:20.731 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:35:20.735 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:35:20.923 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-21T06:35:20.923 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:35:20.924 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-21T06:35:20.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:35:21.169 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:35:21.170 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:35:21.170 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:35:23.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:35:23.427 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:35:23.428 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:35:23.428 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:35:25.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:35:25.686 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T06:35:25.687 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T06:35:25.687 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:35:27.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T06:35:27.951 INFO:teuthology.orchestra.run.smithi037.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T06:35:27.951 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:35:27.951 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:35:27.952 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T06:35:28.201 INFO:teuthology.orchestra.run.smithi150.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}}] 2024-09-21T06:35:28.202 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T06:35:28.202 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-21T06:35:28.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2024-09-21T06:35:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.428+0000 7fe6fce00640 1 -- 172.21.15.37:0/1595319775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f806e330 msgr2=0x7fe6f80cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.428+0000 7fe6fce00640 1 --2- 172.21.15.37:0/1595319775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f806e330 0x7fe6f80cdc70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b2c70 tx=0x7fe6f002aa30 comp rx=0 tx=0).stop 2024-09-21T06:35:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 -- 172.21.15.37:0/1595319775 shutdown_connections 2024-09-21T06:35:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 --2- 172.21.15.37:0/1595319775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f806e330 0x7fe6f80cdc70 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 --2- 172.21.15.37:0/1595319775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 0x7fe6f806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 --2- 172.21.15.37:0/1595319775 >> v2:172.21.15.150:3300/0 conn(0x7fe6f8156a50 0x7fe6f8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 -- 172.21.15.37:0/1595319775 >> 172.21.15.37:0/1595319775 conn(0x7fe6f80b1b90 msgr2=0x7fe6f80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 -- 172.21.15.37:0/1595319775 shutdown_connections 2024-09-21T06:35:28.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.429+0000 7fe6fce00640 1 -- 172.21.15.37:0/1595319775 wait complete. 2024-09-21T06:35:28.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.431+0000 7fe6fce00640 1 Processor -- start 2024-09-21T06:35:28.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.431+0000 7fe6fce00640 1 -- start start 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.432+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 0x7fe6f8155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.432+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 0x7fe6f8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.432+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.432+0000 7fe6fce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f80ce330 con 0x7fe6f8156a50 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.432+0000 7fe6fce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f80ce1b0 con 0x7fe6f806e330 2024-09-21T06:35:28.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6fce00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f80ce4b0 con 0x7fe6f8074b60 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51156/0 (socket says 172.21.15.37:51156) 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 0x7fe6f8155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 -- 172.21.15.37:0/3801290286 learned_addr learned my addr 172.21.15.37:0/3801290286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6efe00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 0x7fe6f8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 -- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 msgr2=0x7fe6f8158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 0x7fe6f8158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 -- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 msgr2=0x7fe6f8155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 0x7fe6f8155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f6800640 1 -- 172.21.15.37:0/3801290286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f815aca0 con 0x7fe6f8156a50 2024-09-21T06:35:28.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.433+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 0x7fe6f8155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009d60 tx=0x7fe6f0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6ee000640 1 -- 172.21.15.37:0/3801290286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0033050 con 0x7fe6f8156a50 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6ee000640 1 -- 172.21.15.37:0/3801290286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f0004450 con 0x7fe6f8156a50 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6ee000640 1 -- 172.21.15.37:0/3801290286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0038450 con 0x7fe6f8156a50 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f815af40 con 0x7fe6f8156a50 2024-09-21T06:35:28.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.434+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f815b500 con 0x7fe6f8156a50 2024-09-21T06:35:28.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.437+0000 7fe6ee000640 1 -- 172.21.15.37:0/3801290286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6f0036070 con 0x7fe6f8156a50 2024-09-21T06:35:28.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.438+0000 7fe6ee000640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 0x7fe6d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.438+0000 7fe6ee000640 1 -- 172.21.15.37:0/3801290286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe6f00bc360 con 0x7fe6f8156a50 2024-09-21T06:35:28.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.438+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 0x7fe6d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.438+0000 7fe703e91740 1 Processor -- start 2024-09-21T06:35:28.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.438+0000 7fe703e91740 1 -- start start 2024-09-21T06:35:28.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.439+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 0x7fe6d407b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800bf40 tx=0x7fe6e800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a659560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a666150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f51a59f1c0 con 0x55f51a26f4d0 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f51a5b2870 con 0x55f51a2776b0 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f51a5a1ca0 con 0x55f51a298390 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a659560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a659560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:51798/0 (socket says 172.21.15.37:51798) 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51164/0 (socket says 172.21.15.37:51164) 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6f5e00640 1 -- 172.21.15.37:0/3765792660 learned_addr learned my addr 172.21.15.37:0/3765792660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.440+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a666150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 -- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 msgr2=0x55f51a659560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a659560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 -- 172.21.15.37:0/3765792660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 msgr2=0x55f51a6850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 -- 172.21.15.37:0/3765792660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51a37a7d0 con 0x55f51a2776b0 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6efe00640 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a666150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00ba630 tx=0x7fe6f0083900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c691e0cd8c378013 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6e6a00640 1 -- 172.21.15.37:0/3765792660 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0083ac0 con 0x55f51a2776b0 2024-09-21T06:35:28.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6e6a00640 1 -- 172.21.15.37:0/3765792660 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f00889e0 con 0x55f51a2776b0 2024-09-21T06:35:28.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.441+0000 7fe6e6a00640 1 -- 172.21.15.37:0/3765792660 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f008d890 con 0x55f51a2776b0 2024-09-21T06:35:28.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 -- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 msgr2=0x55f51a666150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a666150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00ba630 tx=0x7fe6f0083900 comp rx=0 tx=0).stop 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 -- 172.21.15.37:0/3765792660 shutdown_connections 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a26f4d0 0x55f51a6850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a666150 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 --2- 172.21.15.37:0/3765792660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a659560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 -- 172.21.15.37:0/3765792660 >> 172.21.15.37:0/3765792660 conn(0x55f51a28c4e0 msgr2=0x55f51a63f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 -- 172.21.15.37:0/3765792660 shutdown_connections 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.442+0000 7fe703e91740 1 -- 172.21.15.37:0/3765792660 wait complete. 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.443+0000 7fe703e91740 1 -- start start 2024-09-21T06:35:28.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.443+0000 7fe703e91740 1 Processor -- start 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a657dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a668ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f51a548410 con 0x55f51a2776b0 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f51a5a5ab0 con 0x55f51a26f4d0 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f51a312740 con 0x55f51a298390 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe703e91740 10 client.0 _ll_register_callbacks cb 0x55f51a525360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51178/0 (socket says 172.21.15.37:51178) 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a657dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.445+0000 7fe6efe00640 1 -- 172.21.15.37:0/1434047867 learned_addr learned my addr 172.21.15.37:0/1434047867 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6f6800640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a668ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 -- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 msgr2=0x55f51a668ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a668ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 -- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 msgr2=0x55f51a657dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a657dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 -- 172.21.15.37:0/1434047867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51a303970 con 0x55f51a2776b0 2024-09-21T06:35:28.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6efe00640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0010820 tx=0x7fe6e0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf6d8e8266ef17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6db400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.446+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000fb10 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe6e000fcb0 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f00889e0 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fe6e0023050 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 10 client.8442 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55f51a2e16a0 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f00889e0 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7fe6e0026020 con 0x55f51a2776b0 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 10 client.8442 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 10 client.8442 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 20 client.8442 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a256f60 0x55f51a681ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 4 client.8442 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:28.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.447+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f51a5b47c0 con 0x55f51a256f60 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.448+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a256f60 0x55f51a681ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.448+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a256f60 0x55f51a681ef0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c111e4bcf73bea84 server_cookie=2e8e7e965ff84e0f in_seq=0 out_seq=0 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.448+0000 7fe6db400640 10 client.8442 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe6db400640 1 -- 172.21.15.37:0/1434047867 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f51a5b47c0 con 0x55f51a256f60 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe6db400640 10 client.8442 handle_command_reply: tid=0 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 client.8442 shutdown 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 msgr2=0x55f51a6685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0010820 tx=0x7fe6e0010c20 comp rx=0 tx=0).stop 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 shutdown_connections 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a256f60 0x55f51a681ef0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a6685a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a668ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 --2- 172.21.15.37:0/1434047867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a657dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe6db400640 0 client.8442 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 >> 172.21.15.37:0/1434047867 conn(0x55f51a28c4e0 msgr2=0x55f51a63a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 shutdown_connections 2024-09-21T06:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.449+0000 7fe703e91740 1 -- 172.21.15.37:0/1434047867 wait complete. 2024-09-21T06:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.450+0000 7fe703e91740 20 client.8442 trim_cache size 0 max 16384 2024-09-21T06:35:28.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.465+0000 7fe703e91740 1 Processor -- start 2024-09-21T06:35:28.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.465+0000 7fe703e91740 1 -- start start 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a640120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a63a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81fe1c0 con 0x55f51a2776b0 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f80cc230 con 0x55f51a26f4d0 2024-09-21T06:35:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.466+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f81fdab0 con 0x55f51a298390 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51182/0 (socket says 172.21.15.37:51182) 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 -- 172.21.15.37:0/812821663 learned_addr learned my addr 172.21.15.37:0/812821663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6f6800640 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a63a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a640120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 -- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 msgr2=0x55f51a63a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a63a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 -- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 msgr2=0x55f51a640120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a640120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.467+0000 7fe6efe00640 1 -- 172.21.15.37:0/812821663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51a63a730 con 0x55f51a2776b0 2024-09-21T06:35:28.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe6efe00640 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000f860 tx=0x7fe6e000a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b3cb1efeebe48c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe6e6000640 1 -- 172.21.15.37:0/812821663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000fb10 con 0x55f51a2776b0 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe6e6000640 1 -- 172.21.15.37:0/812821663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6e002c070 con 0x55f51a2776b0 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe6e6000640 1 -- 172.21.15.37:0/812821663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00055d0 con 0x55f51a2776b0 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe703e91740 1 -- 172.21.15.37:0/812821663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 msgr2=0x55f51a640660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.468+0000 7fe703e91740 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000f860 tx=0x7fe6e000a270 comp rx=0 tx=0).stop 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 -- 172.21.15.37:0/812821663 shutdown_connections 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a2776b0 0x55f51a640660 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a298390 0x55f51a63a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 --2- 172.21.15.37:0/812821663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a26f4d0 0x55f51a640120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 -- 172.21.15.37:0/812821663 >> 172.21.15.37:0/812821663 conn(0x55f51a28c4e0 msgr2=0x55f51a63f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 -- 172.21.15.37:0/812821663 shutdown_connections 2024-09-21T06:35:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.469+0000 7fe703e91740 1 -- 172.21.15.37:0/812821663 wait complete. 2024-09-21T06:35:28.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.470+0000 7fe703e91740 1 -- start start 2024-09-21T06:35:28.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.470+0000 7fe703e91740 1 Processor -- start 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a26f4d0 0x55f51a6fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a640120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f51a684e00 con 0x55f51a298390 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8157bf0 con 0x55f51a2776b0 2024-09-21T06:35:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f51a6abcb0 con 0x55f51a26f4d0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.471+0000 7fe703e91740 10 client.0 _ll_register_callbacks cb 0x7fe6e000a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a640120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51192/0 (socket says 172.21.15.37:51192) 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 -- 172.21.15.37:0/3726632604 learned_addr learned my addr 172.21.15.37:0/3726632604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a26f4d0 0x55f51a6fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 -- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a26f4d0 msgr2=0x55f51a6fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a26f4d0 0x55f51a6fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 -- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 msgr2=0x55f51a640120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a640120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 -- 172.21.15.37:0/3726632604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51a6fe730 con 0x55f51a298390 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6f6800640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0083a70 tx=0x7fe6f00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c7fc8b5a0b6b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6db400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f009e030 con 0x55f51a298390 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe6f008d600 con 0x55f51a298390 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.472+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e002c070 con 0x55f51a298390 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fe6f00889e0 con 0x55f51a298390 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 10 client.8448 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55f51a668b60 con 0x55f51a298390 2024-09-21T06:35:28.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6e002c070 con 0x55f51a298390 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7fe6f0040070 con 0x55f51a298390 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 10 client.8448 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 10 client.8448 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 20 client.8448 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a6aa8e0 0x55f51a68f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 4 client.8448 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f51a680890 con 0x55f51a6aa8e0 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.473+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a6aa8e0 0x55f51a68f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.474+0000 7fe6f5e00640 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a6aa8e0 0x55f51a68f2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9ea5b0c8a7424a server_cookie=e55145ede04a7c82 in_seq=0 out_seq=0 2024-09-21T06:35:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.474+0000 7fe6db400640 10 client.8448 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:28.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe6db400640 1 -- 172.21.15.37:0/3726632604 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f51a680890 con 0x55f51a6aa8e0 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe6db400640 10 client.8448 handle_command_reply: tid=0 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 client.8448 shutdown 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 msgr2=0x55f51a640660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0083a70 tx=0x7fe6f00094a0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 shutdown_connections 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f51a6aa8e0 0x55f51a68f2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51a298390 0x55f51a640660 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51a2776b0 0x55f51a640120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 --2- 172.21.15.37:0/3726632604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f51a26f4d0 0x55f51a6fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe6db400640 0 client.8448 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:28.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.491+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 >> 172.21.15.37:0/3726632604 conn(0x55f51a28c4e0 msgr2=0x55f51a67ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 shutdown_connections 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe703e91740 1 -- 172.21.15.37:0/3726632604 wait complete. 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe703e91740 20 client.8448 trim_cache size 0 max 16384 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:28.496 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 msgr2=0x7fe6d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 0x7fe6d407b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800bf40 tx=0x7fe6e800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 msgr2=0x7fe6f80c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009d60 tx=0x7fe6f0003f10 comp rx=0 tx=0).stop 2024-09-21T06:35:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 shutdown_connections 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe6d4078b90 0x7fe6d407b0a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f8156a50 0x7fe6f80c70d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f8074b60 0x7fe6f8158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.492+0000 7fe6fce00640 1 --2- 172.21.15.37:0/3801290286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f806e330 0x7fe6f8155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.493+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 >> 172.21.15.37:0/3801290286 conn(0x7fe6f80b1b90 msgr2=0x7fe6f806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.493+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 shutdown_connections 2024-09-21T06:35:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.493+0000 7fe6fce00640 1 -- 172.21.15.37:0/3801290286 wait complete. 2024-09-21T06:35:28.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.747+0000 7f8975400640 1 -- 172.21.15.37:0/2027750190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897006ddf0 msgr2=0x7f897006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.747+0000 7f8975400640 1 --2- 172.21.15.37:0/2027750190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897006ddf0 0x7f897006e200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8958009830 tx=0x7f895802a540 comp rx=0 tx=0).stop 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.747+0000 7f8975400640 1 -- 172.21.15.37:0/2027750190 shutdown_connections 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.747+0000 7f8975400640 1 --2- 172.21.15.37:0/2027750190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897006ddf0 0x7f897006e200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.747+0000 7f8975400640 1 -- 172.21.15.37:0/2027750190 >> 172.21.15.37:0/2027750190 conn(0x7f89700b1b90 msgr2=0x7f89700b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.748+0000 7f8975400640 1 -- 172.21.15.37:0/2027750190 shutdown_connections 2024-09-21T06:35:28.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.748+0000 7f8975400640 1 -- 172.21.15.37:0/2027750190 wait complete. 2024-09-21T06:35:28.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.750+0000 7f8975400640 1 Processor -- start 2024-09-21T06:35:28.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.750+0000 7f8975400640 1 -- start start 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 0x7f897015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 0x7f89701ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8970160fd0 con 0x7f897006ddf0 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8970160e50 con 0x7f8970156a50 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.751+0000 7f8975400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f89700cfa20 con 0x7f89700cef00 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51200/0 (socket says 172.21.15.37:51200) 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 -- 172.21.15.37:0/289436128 learned_addr learned my addr 172.21.15.37:0/289436128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f8965e00640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 0x7f897015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896f200640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 0x7f89701ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 -- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 msgr2=0x7f897015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 0x7f897015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 -- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 msgr2=0x7f89701ef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 0x7f89701ef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896e800640 1 -- 172.21.15.37:0/289436128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89700b5dc0 con 0x7f897006ddf0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f8965e00640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 0x7f897015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.752+0000 7f896f200640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 0x7f89701ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f896e800640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f896000c680 tx=0x7f896000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f8967400640 1 -- 172.21.15.37:0/289436128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960016020 con 0x7f897006ddf0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f8967400640 1 -- 172.21.15.37:0/289436128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896000a930 con 0x7f897006ddf0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f8967400640 1 -- 172.21.15.37:0/289436128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896000ad40 con 0x7f897006ddf0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89700b2f70 con 0x7f897006ddf0 2024-09-21T06:35:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.753+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89700b3210 con 0x7f897006ddf0 2024-09-21T06:35:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.755+0000 7f8967400640 1 -- 172.21.15.37:0/289436128 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8960019020 con 0x7f897006ddf0 2024-09-21T06:35:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.756+0000 7f8967400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 0x7f894c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.756+0000 7f8967400640 1 -- 172.21.15.37:0/289436128 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f896009b270 con 0x7f897006ddf0 2024-09-21T06:35:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.757+0000 7f897c49f740 1 Processor -- start 2024-09-21T06:35:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.757+0000 7f8965e00640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 0x7f894c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.757+0000 7f897c49f740 1 -- start start 2024-09-21T06:35:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.757+0000 7f8965e00640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 0x7f894c07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8958034000 tx=0x7f8958033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d108220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b39d03e1c0 con 0x55b39cd37390 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b39d051870 con 0x55b39cd0e4d0 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.758+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b39d040ca0 con 0x55b39cd166b0 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51208/0 (socket says 172.21.15.37:51208) 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 -- 172.21.15.37:0/313731666 learned_addr learned my addr 172.21.15.37:0/313731666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f8965e00640 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896f200640 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 -- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 msgr2=0x55b39d108760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d108760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 -- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 msgr2=0x55b39d108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 -- 172.21.15.37:0/313731666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39ce197d0 con 0x55b39cd37390 2024-09-21T06:35:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f896e800640 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f896005fff0 tx=0x7f8960062800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48364717a612e3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.759+0000 7f895f000640 1 -- 172.21.15.37:0/313731666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896009b030 con 0x55b39cd37390 2024-09-21T06:35:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f895f000640 1 -- 172.21.15.37:0/313731666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896005eff0 con 0x55b39cd37390 2024-09-21T06:35:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f895f000640 1 -- 172.21.15.37:0/313731666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896006c570 con 0x55b39cd37390 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 -- 172.21.15.37:0/313731666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 msgr2=0x55b39d125550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f896005fff0 tx=0x7f8960062800 comp rx=0 tx=0).stop 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 -- 172.21.15.37:0/313731666 shutdown_connections 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 --2- 172.21.15.37:0/313731666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d125550 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 -- 172.21.15.37:0/313731666 >> 172.21.15.37:0/313731666 conn(0x55b39cd2b4e0 msgr2=0x55b39d0de440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.760+0000 7f897c49f740 1 -- 172.21.15.37:0/313731666 shutdown_connections 2024-09-21T06:35:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.761+0000 7f897c49f740 1 -- 172.21.15.37:0/313731666 wait complete. 2024-09-21T06:35:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.762+0000 7f897c49f740 1 -- start start 2024-09-21T06:35:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.762+0000 7f897c49f740 1 Processor -- start 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd0e4d0 0x55b39d0f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd166b0 0x55b39d132260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b39cfe7410 con 0x55b39cd37390 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b39d044ab0 con 0x55b39cd166b0 2024-09-21T06:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b39cdb1740 con 0x55b39cd0e4d0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.763+0000 7f897c49f740 10 client.0 _ll_register_callbacks cb 0x55b39cfc4360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51212/0 (socket says 172.21.15.37:51212) 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 -- 172.21.15.37:0/3837411460 learned_addr learned my addr 172.21.15.37:0/3837411460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f8965e00640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd166b0 0x55b39d132260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896e800640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd0e4d0 0x55b39d0f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 -- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd0e4d0 msgr2=0x55b39d0f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd0e4d0 0x55b39d0f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 -- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd166b0 msgr2=0x55b39d132260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd166b0 0x55b39d132260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 -- 172.21.15.37:0/3837411460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39cda2970 con 0x55b39cd37390 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f896f200640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f896800f480 tx=0x7f896800f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad943a3186a2e60c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.764+0000 7f8953400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968017020 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896005eff0 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f8968003b80 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f8968021050 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 10 client.8460 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55b39cd806a0 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896005eff0 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7f8968024020 con 0x55b39cd37390 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 10 client.8460 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 10 client.8460 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 20 client.8460 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39ccf5f60 0x55b39d11dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 4 client.8460 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.765+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39d0537c0 con 0x55b39ccf5f60 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.766+0000 7f896e800640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39ccf5f60 0x55b39d11dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.766+0000 7f896e800640 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39ccf5f60 0x55b39d11dac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=275c0070f4d29f56 server_cookie=9e7901ae3ad9360a in_seq=0 out_seq=0 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.766+0000 7f8953400640 10 client.8460 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f8953400640 1 -- 172.21.15.37:0/3837411460 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b39d0537c0 con 0x55b39ccf5f60 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f8953400640 10 client.8460 handle_command_reply: tid=0 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 client.8460 shutdown 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 msgr2=0x55b39d11b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f896800f480 tx=0x7f896800f940 comp rx=0 tx=0).stop 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 shutdown_connections 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39ccf5f60 0x55b39d11dac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd37390 0x55b39d11b680 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd166b0 0x55b39d132260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 --2- 172.21.15.37:0/3837411460 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd0e4d0 0x55b39d0f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f8953400640 0 client.8460 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 >> 172.21.15.37:0/3837411460 conn(0x55b39cd2b4e0 msgr2=0x55b39d0d7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 shutdown_connections 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.767+0000 7f897c49f740 1 -- 172.21.15.37:0/3837411460 wait complete. 2024-09-21T06:35:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.768+0000 7f897c49f740 20 client.8460 trim_cache size 0 max 16384 2024-09-21T06:35:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.783+0000 7f897c49f740 1 Processor -- start 2024-09-21T06:35:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.784+0000 7f897c49f740 1 -- start start 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d11a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd37390 0x55b39d11a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8970202720 con 0x55b39cd0e4d0 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89701f6090 con 0x55b39cd37390 2024-09-21T06:35:28.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8970202040 con 0x55b39cd166b0 2024-09-21T06:35:28.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f896e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51220/0 (socket says 172.21.15.37:51220) 2024-09-21T06:35:28.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f896e800640 1 -- 172.21.15.37:0/1156479448 learned_addr learned my addr 172.21.15.37:0/1156479448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f896f200640 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd37390 0x55b39d11a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.785+0000 7f8965e00640 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d11a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 -- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 msgr2=0x55b39d11a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d11a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 -- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd37390 msgr2=0x55b39d11a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd37390 0x55b39d11a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 -- 172.21.15.37:0/1156479448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39d11abd0 con 0x55b39cd0e4d0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f896e800640 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8960062d10 tx=0x7f8960066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5feb10d303c8266 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f895e600640 1 -- 172.21.15.37:0/1156479448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960070070 con 0x55b39cd0e4d0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f895e600640 1 -- 172.21.15.37:0/1156479448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8960085170 con 0x55b39cd0e4d0 2024-09-21T06:35:28.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.786+0000 7f895e600640 1 -- 172.21.15.37:0/1156479448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896006c570 con 0x55b39cd0e4d0 2024-09-21T06:35:28.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 -- 172.21.15.37:0/1156479448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 msgr2=0x55b39d0d7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8960062d10 tx=0x7f8960066000 comp rx=0 tx=0).stop 2024-09-21T06:35:28.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 -- 172.21.15.37:0/1156479448 shutdown_connections 2024-09-21T06:35:28.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd37390 0x55b39d11a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd166b0 0x55b39d11a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 --2- 172.21.15.37:0/1156479448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd0e4d0 0x55b39d0d7a20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 -- 172.21.15.37:0/1156479448 >> 172.21.15.37:0/1156479448 conn(0x55b39cd2b4e0 msgr2=0x55b39d0d9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 -- 172.21.15.37:0/1156479448 shutdown_connections 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.787+0000 7f897c49f740 1 -- 172.21.15.37:0/1156479448 wait complete. 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.788+0000 7f897c49f740 1 -- start start 2024-09-21T06:35:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.788+0000 7f897c49f740 1 Processor -- start 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d11a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd37390 0x55b39d135c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b39d040eb0 con 0x55b39cd166b0 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89701f96d0 con 0x55b39cd0e4d0 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b39d11c080 con 0x55b39cd37390 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f897c49f740 10 client.0 _ll_register_callbacks cb 0x7f89680015d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f8965e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f8965e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51232/0 (socket says 172.21.15.37:51232) 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f8965e00640 1 -- 172.21.15.37:0/825368689 learned_addr learned my addr 172.21.15.37:0/825368689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f896e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d11a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.790+0000 7f896f200640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd37390 0x55b39d135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 -- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd37390 msgr2=0x55b39d135c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd37390 0x55b39d135c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 -- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 msgr2=0x55b39d11a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d11a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 -- 172.21.15.37:0/825368689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39d1a5260 con 0x55b39cd166b0 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8965e00640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8958009230 tx=0x7f8958004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19969125f1ccc2e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8953400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958053440 con 0x55b39cd166b0 2024-09-21T06:35:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f895803c420 con 0x55b39cd166b0 2024-09-21T06:35:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8960085170 con 0x55b39cd166b0 2024-09-21T06:35:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f895803c620 con 0x55b39cd166b0 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f897c49f740 10 client.8466 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.791+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55b39d1259f0 con 0x55b39cd166b0 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8960085170 con 0x55b39cd166b0 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7f895805b050 con 0x55b39cd166b0 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 10 client.8466 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 10 client.8466 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 20 client.8466 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39d136190 0x55b39d0f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 4 client.8466 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b39d0ffd90 con 0x55b39d136190 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.792+0000 7f896e800640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39d136190 0x55b39d0f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.793+0000 7f8953400640 10 client.8466 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:28.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.793+0000 7f896e800640 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39d136190 0x55b39d0f5f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34bfe0306f3058ed server_cookie=1a4e1a1d91f2085 in_seq=0 out_seq=0 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f8953400640 1 -- 172.21.15.37:0/825368689 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b39d0ffd90 con 0x55b39d136190 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f8953400640 10 client.8466 handle_command_reply: tid=0 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 client.8466 shutdown 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 msgr2=0x55b39d1254b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8958009230 tx=0x7f8958004730 comp rx=0 tx=0).stop 2024-09-21T06:35:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 shutdown_connections 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55b39d136190 0x55b39d0f5f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39cd166b0 0x55b39d1254b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b39cd37390 0x55b39d135c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 --2- 172.21.15.37:0/825368689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b39cd0e4d0 0x55b39d11a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f8953400640 0 client.8466 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.802+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 >> 172.21.15.37:0/825368689 conn(0x55b39cd2b4e0 msgr2=0x55b39d0df3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.803+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 shutdown_connections 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.803+0000 7f897c49f740 1 -- 172.21.15.37:0/825368689 wait complete. 2024-09-21T06:35:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.803+0000 7f897c49f740 20 client.8466 trim_cache size 0 max 16384 2024-09-21T06:35:28.807 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:28.807 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:35:28.807 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:28.807 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 msgr2=0x7f894c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 0x7f894c07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8958034000 tx=0x7f8958033040 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 msgr2=0x7f89700b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f896000c680 tx=0x7f896000cb40 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 shutdown_connections 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f894c078b90 0x7f894c07b0a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8970156a50 0x7f89701ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89700cef00 0x7f897015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 --2- 172.21.15.37:0/289436128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f897006ddf0 0x7f89700b54a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 >> 172.21.15.37:0/289436128 conn(0x7f89700b1b90 msgr2=0x7f89700b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:28.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 shutdown_connections 2024-09-21T06:35:28.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:28.804+0000 7f8975400640 1 -- 172.21.15.37:0/289436128 wait complete. 2024-09-21T06:35:28.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 -- 172.21.15.37:0/3539225843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7870005270 msgr2=0x7f787000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 --2- 172.21.15.37:0/3539225843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7870005270 0x7f787000b9c0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7884005fd0 tx=0x7f7870005830 comp rx=0 tx=0).stop 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 -- 172.21.15.37:0/3539225843 shutdown_connections 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 --2- 172.21.15.37:0/3539225843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7870005270 0x7f787000b9c0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 -- 172.21.15.37:0/3539225843 >> 172.21.15.37:0/3539225843 conn(0x7f788c0b1b90 msgr2=0x7f788c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 -- 172.21.15.37:0/3539225843 shutdown_connections 2024-09-21T06:35:29.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.056+0000 7f7892e00640 1 -- 172.21.15.37:0/3539225843 wait complete. 2024-09-21T06:35:29.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.058+0000 7f7892e00640 1 Processor -- start 2024-09-21T06:35:29.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.059+0000 7f7892e00640 1 -- start start 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 0x7f788c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 0x7f788c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f788c06e590 con 0x7f788c0c6770 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f788c06e410 con 0x7f7870005270 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7892e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f788c06e710 con 0x7f788c0c9840 2024-09-21T06:35:29.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51240/0 (socket says 172.21.15.37:51240) 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.060+0000 7f7890e00640 1 -- 172.21.15.37:0/282964968 learned_addr learned my addr 172.21.15.37:0/282964968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f788be00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 0x7f788c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7883e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 0x7f788c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 -- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 msgr2=0x7f788c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 0x7f788c0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 -- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 msgr2=0x7f788c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 0x7f788c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 -- 172.21.15.37:0/282964968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c09f880 con 0x7f788c0c6770 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7883e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 0x7f788c0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f788be00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 0x7f788c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.061+0000 7f7890e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7884009e20 tx=0x7f78840042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.062+0000 7f7888c00640 1 -- 172.21.15.37:0/282964968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884009200 con 0x7f788c0c6770 2024-09-21T06:35:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.062+0000 7f7888c00640 1 -- 172.21.15.37:0/282964968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7884042070 con 0x7f788c0c6770 2024-09-21T06:35:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.062+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c09fb60 con 0x7f788c0c6770 2024-09-21T06:35:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.062+0000 7f7888c00640 1 -- 172.21.15.37:0/282964968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884009ad0 con 0x7f788c0c6770 2024-09-21T06:35:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.062+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c0cac20 con 0x7f788c0c6770 2024-09-21T06:35:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.064+0000 7f7888c00640 1 -- 172.21.15.37:0/282964968 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7884047460 con 0x7f788c0c6770 2024-09-21T06:35:29.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.065+0000 7f7888c00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 0x7f786807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.065+0000 7f7888c00640 1 -- 172.21.15.37:0/282964968 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f78840bc020 con 0x7f788c0c6770 2024-09-21T06:35:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.066+0000 7f788be00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 0x7f786807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.066+0000 7f7899e46740 1 Processor -- start 2024-09-21T06:35:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.066+0000 7f7899e46740 1 -- start start 2024-09-21T06:35:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.066+0000 7f788be00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 0x7f786807b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f787c000c00 tx=0x7f787c006b10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89cd4d0 0x55fcc8dc7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8dc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8cfd1c0 con 0x55fcc89f6390 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8d10870 con 0x55fcc89cd4d0 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55fcc8cffca0 con 0x55fcc89d56b0 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.067+0000 7f788be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51252/0 (socket says 172.21.15.37:51252) 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 -- 172.21.15.37:0/899748238 learned_addr learned my addr 172.21.15.37:0/899748238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f7883e00640 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89cd4d0 0x55fcc8dc7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f7890e00640 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8dc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 -- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 msgr2=0x55fcc8dc7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8dc7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 -- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89cd4d0 msgr2=0x55fcc8dc7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89cd4d0 0x55fcc8dc7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 -- 172.21.15.37:0/899748238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc8ad87d0 con 0x55fcc89f6390 2024-09-21T06:35:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f788be00640 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f787c001900 tx=0x7f787c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ab5e43f4c97751 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.068+0000 7f7880c00640 1 -- 172.21.15.37:0/899748238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c024030 con 0x55fcc89f6390 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7880c00640 1 -- 172.21.15.37:0/899748238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f787c016070 con 0x55fcc89f6390 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7880c00640 1 -- 172.21.15.37:0/899748238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c012be0 con 0x55fcc89f6390 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 -- 172.21.15.37:0/899748238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 msgr2=0x55fcc8de4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f787c001900 tx=0x7f787c014000 comp rx=0 tx=0).stop 2024-09-21T06:35:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 -- 172.21.15.37:0/899748238 shutdown_connections 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8dc7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89cd4d0 0x55fcc8dc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 --2- 172.21.15.37:0/899748238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8de4550 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.069+0000 7f7899e46740 1 -- 172.21.15.37:0/899748238 >> 172.21.15.37:0/899748238 conn(0x55fcc89ea4e0 msgr2=0x55fcc8d9d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.070+0000 7f7899e46740 1 -- 172.21.15.37:0/899748238 shutdown_connections 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.070+0000 7f7899e46740 1 -- 172.21.15.37:0/899748238 wait complete. 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.071+0000 7f7899e46740 1 -- start start 2024-09-21T06:35:29.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.071+0000 7f7899e46740 1 Processor -- start 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.072+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dc2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.072+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8df0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.072+0000 7f7899e46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.072+0000 7f7899e46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8ca6410 con 0x55fcc89f6390 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.072+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8d03ab0 con 0x55fcc89d56b0 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55fcc8a70740 con 0x55fcc89cd4d0 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7899e46740 10 client.0 _ll_register_callbacks cb 0x55fcc8c83360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51254/0 (socket says 172.21.15.37:51254) 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f788be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dc2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 -- 172.21.15.37:0/3280120694 learned_addr learned my addr 172.21.15.37:0/3280120694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7883e00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8df0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 -- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 msgr2=0x55fcc8dc2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dc2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 -- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 msgr2=0x55fcc8df0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8df0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.073+0000 7f7890e00640 1 -- 172.21.15.37:0/3280120694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc8a61970 con 0x55fcc89f6390 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7890e00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f78840ba320 tx=0x7f78840833b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef5794d3009316 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840842c0 con 0x55fcc89f6390 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f78840881e0 con 0x55fcc89f6390 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f787c016070 con 0x55fcc89f6390 2024-09-21T06:35:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f7884083bc0 con 0x55fcc89f6390 2024-09-21T06:35:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 10 client.8478 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55fcc8a3f6a0 con 0x55fcc89f6390 2024-09-21T06:35:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f787c016070 con 0x55fcc89f6390 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7f7884099ce0 con 0x55fcc89f6390 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 10 client.8478 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 10 client.8478 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.074+0000 7f7899e46740 20 client.8478 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc89b4f60 0x55fcc8dc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f7899e46740 4 client.8478 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc8d127c0 con 0x55fcc89b4f60 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f788be00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc89b4f60 0x55fcc8dc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f788be00640 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc89b4f60 0x55fcc8dc3bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4b296059e67fb39 server_cookie=2e8908e41329f0f0 in_seq=0 out_seq=0 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.075+0000 7f7879600640 10 client.8478 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.076+0000 7f7879600640 1 -- 172.21.15.37:0/3280120694 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcc8d127c0 con 0x55fcc89b4f60 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.076+0000 7f7879600640 10 client.8478 handle_command_reply: tid=0 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.076+0000 7f7899e46740 1 client.8478 shutdown 2024-09-21T06:35:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 msgr2=0x55fcc8dc0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f78840ba320 tx=0x7f78840833b0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 shutdown_connections 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc89b4f60 0x55fcc8dc3bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8dc0620 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8df0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 --2- 172.21.15.37:0/3280120694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dc2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7879600640 0 client.8478 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 >> 172.21.15.37:0/3280120694 conn(0x55fcc89ea4e0 msgr2=0x55fcc8db69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 shutdown_connections 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 1 -- 172.21.15.37:0/3280120694 wait complete. 2024-09-21T06:35:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.077+0000 7f7899e46740 20 client.8478 trim_cache size 0 max 16384 2024-09-21T06:35:29.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.093+0000 7f7899e46740 1 Processor -- start 2024-09-21T06:35:29.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.093+0000 7f7899e46740 1 -- start start 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.094+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.094+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8db41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.094+0000 7f7899e46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7899e46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f788c202c40 con 0x55fcc89f6390 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f788c201e10 con 0x55fcc89d56b0 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f788c202510 con 0x55fcc89cd4d0 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51270/0 (socket says 172.21.15.37:51270) 2024-09-21T06:35:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 -- 172.21.15.37:0/837085515 learned_addr learned my addr 172.21.15.37:0/837085515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 -- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 msgr2=0x55fcc8dbf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 -- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 msgr2=0x55fcc8db41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8db41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 -- 172.21.15.37:0/837085515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc8d99fa0 con 0x55fcc89f6390 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.095+0000 7f7890e00640 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7884009f90 tx=0x7f7884083440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b3d1c58a5681c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f787be00640 1 -- 172.21.15.37:0/837085515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840842c0 con 0x55fcc89f6390 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f787be00640 1 -- 172.21.15.37:0/837085515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f788408d2d0 con 0x55fcc89f6390 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f787be00640 1 -- 172.21.15.37:0/837085515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884088420 con 0x55fcc89f6390 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 -- 172.21.15.37:0/837085515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 msgr2=0x55fcc8db4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7884009f90 tx=0x7f7884083440 comp rx=0 tx=0).stop 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 -- 172.21.15.37:0/837085515 shutdown_connections 2024-09-21T06:35:29.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89f6390 0x55fcc8db4700 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89d56b0 0x55fcc8db41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 --2- 172.21.15.37:0/837085515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89cd4d0 0x55fcc8dbf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 -- 172.21.15.37:0/837085515 >> 172.21.15.37:0/837085515 conn(0x55fcc89ea4e0 msgr2=0x55fcc8d92f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.096+0000 7f7899e46740 1 -- 172.21.15.37:0/837085515 shutdown_connections 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.097+0000 7f7899e46740 1 -- 172.21.15.37:0/837085515 wait complete. 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.097+0000 7f7899e46740 1 -- start start 2024-09-21T06:35:29.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.098+0000 7f7899e46740 1 Processor -- start 2024-09-21T06:35:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8ddab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89f6390 0x55fcc8e03e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f788c200e90 con 0x55fcc89cd4d0 2024-09-21T06:35:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8db4c40 con 0x55fcc89f6390 2024-09-21T06:35:29.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55fcc8ddb560 con 0x55fcc89d56b0 2024-09-21T06:35:29.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f7899e46740 10 client.0 _ll_register_callbacks cb 0x7f7884083510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:29.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f788be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f788be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51272/0 (socket says 172.21.15.37:51272) 2024-09-21T06:35:29.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.099+0000 7f788be00640 1 -- 172.21.15.37:0/3244343133 learned_addr learned my addr 172.21.15.37:0/3244343133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:29.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7890e00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89f6390 0x55fcc8e03e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7883e00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8ddab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 -- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 msgr2=0x55fcc8ddab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8ddab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 -- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89f6390 msgr2=0x55fcc8e03e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89f6390 0x55fcc8e03e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 -- 172.21.15.37:0/3244343133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc8d9d3b0 con 0x55fcc89cd4d0 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f788be00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f787c012810 tx=0x7f787c0064b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d20c204cec25ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7879600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c02a450 con 0x55fcc89cd4d0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f787c02c8b0 con 0x55fcc89cd4d0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.100+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f788408d2d0 con 0x55fcc89cd4d0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f787c02cab0 con 0x55fcc89cd4d0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 10 client.8484 fetch_fsmap learned FSMap version 115 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55fcc8ddb300 con 0x55fcc89cd4d0 2024-09-21T06:35:29.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f788408d2d0 con 0x55fcc89cd4d0 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 115) ==== 3554+0+0 (secure 0 0 0) 0x7f787c02cc50 con 0x55fcc89cd4d0 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 10 client.8484 fetch_fsmap finished waiting for FSMap version 115 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 10 client.8484 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 20 client.8484 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc8db2f80 0x55fcc8db3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 4 client.8484 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcc8dc13d0 con 0x55fcc8db2f80 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.101+0000 7f7883e00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc8db2f80 0x55fcc8db3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.102+0000 7f7883e00640 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc8db2f80 0x55fcc8db3390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfceca5fe3590566 server_cookie=a2fef38594526979 in_seq=0 out_seq=0 2024-09-21T06:35:29.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.102+0000 7f7879600640 10 client.8484 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:29.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.112+0000 7f7879600640 1 -- 172.21.15.37:0/3244343133 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcc8dc13d0 con 0x55fcc8db2f80 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.112+0000 7f7879600640 10 client.8484 handle_command_reply: tid=0 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.112+0000 7f7899e46740 1 client.8484 shutdown 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 msgr2=0x55fcc8e607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f787c012810 tx=0x7f787c0064b0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 shutdown_connections 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55fcc8db2f80 0x55fcc8db3390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fcc89f6390 0x55fcc8e03e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fcc89d56b0 0x55fcc8ddab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 --2- 172.21.15.37:0/3244343133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fcc89cd4d0 0x55fcc8e607c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 >> 172.21.15.37:0/3244343133 conn(0x55fcc89ea4e0 msgr2=0x55fcc8dbe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7879600640 0 client.8484 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 shutdown_connections 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 1 -- 172.21.15.37:0/3244343133 wait complete. 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.113+0000 7f7899e46740 20 client.8484 trim_cache size 0 max 16384 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:29.117 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:35:29.118 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:29.118 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:29.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 msgr2=0x7f786807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 0x7f786807b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f787c000c00 tx=0x7f787c006b10 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 msgr2=0x7f788c0cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7884009e20 tx=0x7f78840042d0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 shutdown_connections 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7868078bc0 0x7f786807b0d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f788c0c6770 0x7f788c0cb140 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788c0c9840 0x7f788c0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 --2- 172.21.15.37:0/282964968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7870005270 0x7f788c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 >> 172.21.15.37:0/282964968 conn(0x7f788c0b1b90 msgr2=0x7f788c0738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 shutdown_connections 2024-09-21T06:35:29.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29.114+0000 7f7892e00640 1 -- 172.21.15.37:0/282964968 wait complete. 2024-09-21T06:35:29.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:29.138 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-21T06:35:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/ 2024-09-21T06:35:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4 2024-09-21T06:35:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5 2024-09-21T06:35:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6 2024-09-21T06:35:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-21T06:35:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-21T06:35:29.185 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:29.326 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-09-21T06:35:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:29.336 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.336 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.337 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.337 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.337 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-21T06:35:29.337 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:29.477 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-09-21T06:35:29.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:29.488 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.488 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.488 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.488 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.488 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-21T06:35:29.489 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:29.628 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-09-21T06:35:29.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-21T06:35:29.640 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:29.780 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627781 2024-09-21T06:35:29.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:29.791 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.791 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.792 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.792 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.792 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-21T06:35:29.792 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627782 2024-09-21T06:35:29.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:29.943 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-21T06:35:29.944 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-21T06:35:29.944 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-21T06:35:29.944 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-21T06:35:29.944 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-21T06:35:29.944 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-21T06:35:30.085 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627783 2024-09-21T06:35:30.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:35:30.096 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-21T06:35:30.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.281+0000 7f9b01486f40 1 Processor -- start 2024-09-21T06:35:30.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.281+0000 7f9b01486f40 1 -- start start 2024-09-21T06:35:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2e0aac810 0x55a2e0977d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 0x55a2e0979bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2e089ac60 con 0x7f9ae00028c0 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2e08f91d0 con 0x55a2e0aacc20 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9b01486f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2e0a22e50 con 0x55a2e0aac810 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9afa600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.282+0000 7f9afa600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51292/0 (socket says 172.21.15.37:51292) 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 -- 172.21.15.37:0/2979932040 learned_addr learned my addr 172.21.15.37:0/2979932040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9af9c00640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2e0aac810 0x55a2e0977d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afb000640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 0x55a2e0979bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 -- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2e0aac810 msgr2=0x55a2e0977d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2e0aac810 0x55a2e0977d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 -- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 msgr2=0x55a2e0979bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 0x55a2e0979bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 -- 172.21.15.37:0/2979932040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e0a18490 con 0x7f9ae00028c0 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afb000640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 0x55a2e0979bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9afa600640 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b810 tx=0x7f9aec00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5c0e21a1da4c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.283+0000 7f9af3e00640 1 -- 172.21.15.37:0/2979932040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ad60 con 0x7f9ae00028c0 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9af3e00640 1 -- 172.21.15.37:0/2979932040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aec016440 con 0x7f9ae00028c0 2024-09-21T06:35:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9af3e00640 1 -- 172.21.15.37:0/2979932040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec016850 con 0x7f9ae00028c0 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 -- 172.21.15.37:0/2979932040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 msgr2=0x55a2e0a24c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b810 tx=0x7f9aec00bcd0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 -- 172.21.15.37:0/2979932040 shutdown_connections 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aacc20 0x55a2e0979bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a2e0aac810 0x55a2e0977d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 --2- 172.21.15.37:0/2979932040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ae00028c0 0x55a2e0a24c30 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 -- 172.21.15.37:0/2979932040 >> 172.21.15.37:0/2979932040 conn(0x55a2e086bac0 msgr2=0x55a2e0976a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 -- 172.21.15.37:0/2979932040 shutdown_connections 2024-09-21T06:35:30.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.284+0000 7f9b01486f40 1 -- 172.21.15.37:0/2979932040 wait complete. 2024-09-21T06:35:30.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.285+0000 7f9b01486f40 1 Processor -- start 2024-09-21T06:35:30.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.286+0000 7f9b01486f40 1 -- start start 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aac810 0x55a2e0a13310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 0x55a2e0aba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ae00028c0 0x55a2e09799b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2e0995330 con 0x55a2e0aacc20 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2e0995580 con 0x55a2e0aac810 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9b01486f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55a2e0a25590 con 0x7f9ae00028c0 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9af9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 0x55a2e0aba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9af9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 0x55a2e0aba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51308/0 (socket says 172.21.15.37:51308) 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9af9c00640 1 -- 172.21.15.37:0/1486685759 learned_addr learned my addr 172.21.15.37:0/1486685759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9afa600640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aac810 0x55a2e0a13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.287+0000 7f9afb000640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ae00028c0 0x55a2e09799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 -- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ae00028c0 msgr2=0x55a2e09799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ae00028c0 0x55a2e09799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 -- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aac810 msgr2=0x55a2e0a13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2e0aac810 0x55a2e0a13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 -- 172.21.15.37:0/1486685759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e0a13850 con 0x55a2e0aacc20 2024-09-21T06:35:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af9c00640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 0x55a2e0aba800 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8003fd0 tx=0x7f9ae8001ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af2a00640 1 -- 172.21.15.37:0/1486685759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8063070 con 0x55a2e0aacc20 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af2a00640 1 -- 172.21.15.37:0/1486685759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae804dd00 con 0x55a2e0aacc20 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9af2a00640 1 -- 172.21.15.37:0/1486685759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae805f690 con 0x55a2e0aacc20 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.288+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a2e0a1b4c0 con 0x55a2e0aacc20 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.289+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e0aba000 con 0x55a2e0aacc20 2024-09-21T06:35:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.291+0000 7f9af2a00640 1 -- 172.21.15.37:0/1486685759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ae8066070 con 0x55a2e0aacc20 2024-09-21T06:35:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9af2a00640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9adc077570 0x7f9adc079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9af2a00640 1 -- 172.21.15.37:0/1486685759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f9ae80ddbd0 con 0x55a2e0aacc20 2024-09-21T06:35:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9b01486f40 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55a2e0a13a50 0x55a2e0a16740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 15.12 15:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55a2e0a16c80 con 0x55a2e0a13a50 2024-09-21T06:35:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9afa600640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9adc077570 0x7f9adc079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.292+0000 7f9afb000640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55a2e0a13a50 0x55a2e0a16740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.293+0000 7f9afa600640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9adc077570 0x7f9adc079a80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b810 tx=0x7f9aec00e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.293+0000 7f9afb000640 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55a2e0a13a50 0x55a2e0a16740 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9afb000640 1 -- 172.21.15.37:0/1486685759 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v104'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9af4003040 con 0x55a2e0a13a50 2024-09-21T06:35:30.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55a2e0a13a50 msgr2=0x55a2e0a16740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x55a2e0a13a50 0x55a2e0a16740 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9adc077570 msgr2=0x7f9adc079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9adc077570 0x7f9adc079a80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b810 tx=0x7f9aec00e040 comp rx=0 tx=0).stop 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 msgr2=0x55a2e0aba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 --2- 172.21.15.37:0/1486685759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2e0aacc20 0x55a2e0aba800 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8003fd0 tx=0x7f9ae8001ce0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9afb000640 1 -- 172.21.15.37:0/1486685759 reap_dead start 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 shutdown_connections 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 >> 172.21.15.37:0/1486685759 conn(0x55a2e086bac0 msgr2=0x55a2e0976a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 shutdown_connections 2024-09-21T06:35:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.295+0000 7f9b01486f40 1 -- 172.21.15.37:0/1486685759 wait complete. 2024-09-21T06:35:30.311 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-21T06:35:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.453+0000 7f4e29b96f40 1 Processor -- start 2024-09-21T06:35:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.453+0000 7f4e29b96f40 1 -- start start 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf81aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf7db460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561abf728c60 con 0x561abf824630 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561abf7871d0 con 0x561abf8975c0 2024-09-21T06:35:30.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561abf823bd0 con 0x561abf7dc9b0 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.454+0000 7f4e22400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51320/0 (socket says 172.21.15.37:51320) 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 -- 172.21.15.37:0/2270355585 learned_addr learned my addr 172.21.15.37:0/2270355585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22e00640 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf81aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e23800640 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf7db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 -- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 msgr2=0x561abf81aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf81aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 -- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 msgr2=0x561abf7db460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf7db460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 -- 172.21.15.37:0/2270355585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abf7dc7a0 con 0x561abf824630 2024-09-21T06:35:30.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.455+0000 7f4e22400640 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b630 tx=0x7f4e1000baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2643f305d500baa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e21000640 1 -- 172.21.15.37:0/2270355585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000a8e0 con 0x561abf824630 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e21000640 1 -- 172.21.15.37:0/2270355585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e1000aa80 con 0x561abf824630 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e21000640 1 -- 172.21.15.37:0/2270355585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10008450 con 0x561abf824630 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 -- 172.21.15.37:0/2270355585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 msgr2=0x561abf93c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b630 tx=0x7f4e1000baf0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 -- 172.21.15.37:0/2270355585 shutdown_connections 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf93c410 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf7db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/2270355585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf81aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 -- 172.21.15.37:0/2270355585 >> 172.21.15.37:0/2270355585 conn(0x561abf6f9ac0 msgr2=0x561abf819b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.456+0000 7f4e29b96f40 1 -- 172.21.15.37:0/2270355585 shutdown_connections 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.457+0000 7f4e29b96f40 1 -- 172.21.15.37:0/2270355585 wait complete. 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.458+0000 7f4e29b96f40 1 Processor -- start 2024-09-21T06:35:30.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.458+0000 7f4e29b96f40 1 -- start start 2024-09-21T06:35:30.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf939f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf890550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf945920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561abf806670 con 0x561abf824630 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561abf820c50 con 0x561abf8975c0 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e29b96f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561abf806390 con 0x561abf7dc9b0 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e22400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf890550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e22400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf890550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51322/0 (socket says 172.21.15.37:51322) 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.459+0000 7f4e22400640 1 -- 172.21.15.37:0/1600229493 learned_addr learned my addr 172.21.15.37:0/1600229493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e23800640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf945920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22e00640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf939f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 -- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 msgr2=0x561abf939f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561abf7dc9b0 0x561abf939f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 -- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 msgr2=0x561abf945920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561abf8975c0 0x561abf945920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 -- 172.21.15.37:0/1600229493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abf93caf0 con 0x561abf824630 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e22400640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf890550 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x561abf80a690 tx=0x7f4e10004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.460+0000 7f4e1b400640 1 -- 172.21.15.37:0/1600229493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10016c40 con 0x561abf824630 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.461+0000 7f4e1b400640 1 -- 172.21.15.37:0/1600229493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e10016de0 con 0x561abf824630 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.461+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561abf893390 con 0x561abf824630 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.461+0000 7f4e1b400640 1 -- 172.21.15.37:0/1600229493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10020420 con 0x561abf824630 2024-09-21T06:35:30.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.461+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abf93bc20 con 0x561abf824630 2024-09-21T06:35:30.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.463+0000 7f4e1b400640 1 -- 172.21.15.37:0/1600229493 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e100205e0 con 0x561abf824630 2024-09-21T06:35:30.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.464+0000 7f4e1b400640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e04077570 0x7f4e04079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.464+0000 7f4e1b400640 1 -- 172.21.15.37:0/1600229493 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4e1009bda0 con 0x561abf824630 2024-09-21T06:35:30.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.464+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x561abf8096f0 0x561abf951ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.464+0000 7f4e22e00640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e04077570 0x7f4e04079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.464+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- osd_op(unknown.0.0:1 15.12 15:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x561abf8afa10 con 0x561abf8096f0 2024-09-21T06:35:30.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.465+0000 7f4e23800640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x561abf8096f0 0x561abf951ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.465+0000 7f4e22e00640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e04077570 0x7f4e04079a80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x561abf80a6c0 tx=0x7f4e14001730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.465+0000 7f4e23800640 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x561abf8096f0 0x561abf951ee0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e23800640 1 -- 172.21.15.37:0/1600229493 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v104'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4e1c00b040 con 0x561abf8096f0 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x561abf8096f0 msgr2=0x561abf951ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x561abf8096f0 0x561abf951ee0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e04077570 msgr2=0x7f4e04079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4e04077570 0x7f4e04079a80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x561abf80a6c0 tx=0x7f4e14001730 comp rx=0 tx=0).stop 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 msgr2=0x561abf890550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 --2- 172.21.15.37:0/1600229493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561abf824630 0x561abf890550 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x561abf80a690 tx=0x7f4e10004ec0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e23800640 1 -- 172.21.15.37:0/1600229493 reap_dead start 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 shutdown_connections 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.467+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 >> 172.21.15.37:0/1600229493 conn(0x561abf6f9ac0 msgr2=0x561abf895340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.468+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 shutdown_connections 2024-09-21T06:35:30.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.468+0000 7f4e29b96f40 1 -- 172.21.15.37:0/1600229493 wait complete. 2024-09-21T06:35:30.483 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-21T06:35:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.625+0000 7f24b87e7f40 1 Processor -- start 2024-09-21T06:35:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.625+0000 7f24b87e7f40 1 -- start start 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a53f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a4bdfd0 0x557a9a5dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557a9a4211d0 con 0x557a9a544280 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557a9a42df90 con 0x557a9a476b80 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557a9a3c2c60 con 0x557a9a4bdfd0 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51340/0 (socket says 172.21.15.37:51340) 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 -- 172.21.15.37:0/2512040050 learned_addr learned my addr 172.21.15.37:0/2512040050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b2400640 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a4bdfd0 0x557a9a5dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1a00640 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a53f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 -- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a4bdfd0 msgr2=0x557a9a5dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a4bdfd0 0x557a9a5dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 -- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 msgr2=0x557a9a53f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a53f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.627+0000 7f24b1000640 1 -- 172.21.15.37:0/2512040050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9a543550 con 0x557a9a544280 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.628+0000 7f24b1000640 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001d80 tx=0x7f24a0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f55a11840323781 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.628+0000 7f24ab400640 1 -- 172.21.15.37:0/2512040050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000bd70 con 0x557a9a544280 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.628+0000 7f24ab400640 1 -- 172.21.15.37:0/2512040050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24a000f400 con 0x557a9a544280 2024-09-21T06:35:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.628+0000 7f24ab400640 1 -- 172.21.15.37:0/2512040050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000f850 con 0x557a9a544280 2024-09-21T06:35:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 -- 172.21.15.37:0/2512040050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 msgr2=0x557a9a53fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001d80 tx=0x7f24a0001e70 comp rx=0 tx=0).stop 2024-09-21T06:35:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 -- 172.21.15.37:0/2512040050 shutdown_connections 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a544280 0x557a9a53fe10 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a4bdfd0 0x557a9a5dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/2512040050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a53f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 -- 172.21.15.37:0/2512040050 >> 172.21.15.37:0/2512040050 conn(0x557a9a393ac0 msgr2=0x557a9a4bd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 -- 172.21.15.37:0/2512040050 shutdown_connections 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.629+0000 7f24b87e7f40 1 -- 172.21.15.37:0/2512040050 wait complete. 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.630+0000 7f24b87e7f40 1 Processor -- start 2024-09-21T06:35:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.630+0000 7f24b87e7f40 1 -- start start 2024-09-21T06:35:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a4bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a544280 0x557a9a4bc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 0x557a9a5d3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557a9a49ea20 con 0x557a9a53ebb0 2024-09-21T06:35:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557a9a540350 con 0x557a9a476b80 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.631+0000 7f24b87e7f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557a9a49fa30 con 0x557a9a544280 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 0x557a9a5d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 0x557a9a5d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51354/0 (socket says 172.21.15.37:51354) 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b2400640 1 -- 172.21.15.37:0/1698711450 learned_addr learned my addr 172.21.15.37:0/1698711450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1a00640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a4bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a544280 0x557a9a4bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 -- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 msgr2=0x557a9a4bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557a9a476b80 0x557a9a4bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 -- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 msgr2=0x557a9a5d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 0x557a9a5d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 -- 172.21.15.37:0/1698711450 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9a5d4ac0 con 0x557a9a544280 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b2400640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a9a53ebb0 0x557a9a5d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.632+0000 7f24b1000640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a544280 0x557a9a4bc930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001d50 tx=0x7f24a000fed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.633+0000 7f24aa000640 1 -- 172.21.15.37:0/1698711450 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a001b430 con 0x557a9a544280 2024-09-21T06:35:30.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.633+0000 7f24aa000640 1 -- 172.21.15.37:0/1698711450 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24a0020020 con 0x557a9a544280 2024-09-21T06:35:30.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.633+0000 7f24aa000640 1 -- 172.21.15.37:0/1698711450 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a001b7a0 con 0x557a9a544280 2024-09-21T06:35:30.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.633+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a9a4b89f0 con 0x557a9a544280 2024-09-21T06:35:30.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.633+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x557a9a5d4da0 con 0x557a9a544280 2024-09-21T06:35:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.636+0000 7f24aa000640 1 -- 172.21.15.37:0/1698711450 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24a0017070 con 0x557a9a544280 2024-09-21T06:35:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24aa000640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2494077570 0x7f2494079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24aa000640 1 -- 172.21.15.37:0/1698711450 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f24a009ab60 con 0x557a9a544280 2024-09-21T06:35:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x557a9a4b91f0 0x557a9a4b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- osd_op(unknown.0.0:1 15.5 15:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x557a9a4b9b40 con 0x557a9a4b91f0 2024-09-21T06:35:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24b1a00640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2494077570 0x7f2494079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.637+0000 7f24b2400640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x557a9a4b91f0 0x557a9a4b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.638+0000 7f24b1a00640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2494077570 0x7f2494079a80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f24a4002a10 tx=0x7f24a40029a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.638+0000 7f24b2400640 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x557a9a4b91f0 0x557a9a4b9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b2400640 1 -- 172.21.15.37:0/1698711450 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v104'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f24ac00b040 con 0x557a9a4b91f0 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x557a9a4b91f0 msgr2=0x557a9a4b9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x557a9a4b91f0 0x557a9a4b9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2494077570 msgr2=0x7f2494079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2494077570 0x7f2494079a80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f24a4002a10 tx=0x7f24a40029a0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a544280 msgr2=0x557a9a4bc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 --2- 172.21.15.37:0/1698711450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557a9a544280 0x557a9a4bc930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001d50 tx=0x7f24a000fed0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b2400640 1 -- 172.21.15.37:0/1698711450 reap_dead start 2024-09-21T06:35:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 shutdown_connections 2024-09-21T06:35:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 >> 172.21.15.37:0/1698711450 conn(0x557a9a393ac0 msgr2=0x557a9a4bd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 shutdown_connections 2024-09-21T06:35:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.640+0000 7f24b87e7f40 1 -- 172.21.15.37:0/1698711450 wait complete. 2024-09-21T06:35:30.656 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-21T06:35:30.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.798+0000 7f716bd65f40 1 Processor -- start 2024-09-21T06:35:30.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.798+0000 7f716bd65f40 1 -- start start 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe32c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe32ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556dbe24fc60 con 0x556dbe3d1280 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dbe2ae1d0 con 0x556dbe34b360 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.799+0000 7f716bd65f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556dbe303dc0 con 0x556dbe3039b0 2024-09-21T06:35:30.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51372/0 (socket says 172.21.15.37:51372) 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 -- 172.21.15.37:0/3683882531 learned_addr learned my addr 172.21.15.37:0/3683882531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f715fe00640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe32ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7164e00640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe32c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 -- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 msgr2=0x556dbe32c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe32c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 -- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 msgr2=0x556dbe32ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe32ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7165800640 1 -- 172.21.15.37:0/3683882531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbe462b30 con 0x556dbe3d1280 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f715fe00640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe32ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.800+0000 7f7164e00640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe32c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f7165800640 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f716000a4b0 tx=0x7f716000a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b7a30e3d88064c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f715ea00640 1 -- 172.21.15.37:0/3683882531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160015070 con 0x556dbe3d1280 2024-09-21T06:35:30.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f715ea00640 1 -- 172.21.15.37:0/3683882531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f716000e730 con 0x556dbe3d1280 2024-09-21T06:35:30.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f715ea00640 1 -- 172.21.15.37:0/3683882531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716000eb80 con 0x556dbe3d1280 2024-09-21T06:35:30.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 -- 172.21.15.37:0/3683882531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 msgr2=0x556dbe46a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f716000a4b0 tx=0x7f716000a970 comp rx=0 tx=0).stop 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 -- 172.21.15.37:0/3683882531 shutdown_connections 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe46a0b0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe32ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3683882531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe32c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.801+0000 7f716bd65f40 1 -- 172.21.15.37:0/3683882531 >> 172.21.15.37:0/3683882531 conn(0x556dbe220ac0 msgr2=0x556dbe34a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.802+0000 7f716bd65f40 1 -- 172.21.15.37:0/3683882531 shutdown_connections 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.802+0000 7f716bd65f40 1 -- 172.21.15.37:0/3683882531 wait complete. 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.803+0000 7f716bd65f40 1 Processor -- start 2024-09-21T06:35:30.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.803+0000 7f716bd65f40 1 -- start start 2024-09-21T06:35:30.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe3d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe300b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe301050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556dbe3b7ca0 con 0x556dbe3d1280 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dbe3d4430 con 0x556dbe34b360 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f716bd65f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x556dbe3d6e40 con 0x556dbe3039b0 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f7165800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe301050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f7165800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe301050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51374/0 (socket says 172.21.15.37:51374) 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.804+0000 7f7165800640 1 -- 172.21.15.37:0/3072683272 learned_addr learned my addr 172.21.15.37:0/3072683272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f715fe00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe300b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7164e00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe3d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 -- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 msgr2=0x556dbe3d66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556dbe3039b0 0x556dbe3d66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 -- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 msgr2=0x556dbe300b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbe34b360 0x556dbe300b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 -- 172.21.15.37:0/3072683272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbe3d2050 con 0x556dbe3d1280 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f7165800640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe301050 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7160013040 tx=0x7f716000efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f715d600640 1 -- 172.21.15.37:0/3072683272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160020020 con 0x556dbe3d1280 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f715d600640 1 -- 172.21.15.37:0/3072683272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71600107f0 con 0x556dbe3d1280 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556dbe3d2330 con 0x556dbe3d1280 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.805+0000 7f715d600640 1 -- 172.21.15.37:0/3072683272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160015040 con 0x556dbe3d1280 2024-09-21T06:35:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.806+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbe3d28f0 con 0x556dbe3d1280 2024-09-21T06:35:30.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.808+0000 7f715d600640 1 -- 172.21.15.37:0/3072683272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7160023020 con 0x556dbe3d1280 2024-09-21T06:35:30.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f715d600640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7144077570 0x7f7144079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f715d600640 1 -- 172.21.15.37:0/3072683272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f716009ac40 con 0x556dbe3d1280 2024-09-21T06:35:30.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x556dbe3d3ab0 0x556dbe3d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- osd_op(unknown.0.0:1 15.9 15:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x556dbe47a4f0 con 0x556dbe3d3ab0 2024-09-21T06:35:30.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f7164e00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7144077570 0x7f7144079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.809+0000 7f715fe00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x556dbe3d3ab0 0x556dbe3d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.810+0000 7f715fe00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x556dbe3d3ab0 0x556dbe3d3ec0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.810+0000 7f7164e00640 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7144077570 0x7f7144079a80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7158055000 tx=0x7f7158058040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f715fe00640 1 -- 172.21.15.37:0/3072683272 <== osd.2 v2:172.21.15.37:6800/2037276396 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v104'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f715000a650 con 0x556dbe3d3ab0 2024-09-21T06:35:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x556dbe3d3ab0 msgr2=0x556dbe3d3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x556dbe3d3ab0 0x556dbe3d3ec0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7144077570 msgr2=0x7f7144079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7144077570 0x7f7144079a80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7158055000 tx=0x7f7158058040 comp rx=0 tx=0).stop 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 msgr2=0x556dbe301050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 --2- 172.21.15.37:0/3072683272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556dbe3d1280 0x556dbe301050 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7160013040 tx=0x7f716000efc0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f7165800640 1 -- 172.21.15.37:0/3072683272 reap_dead start 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 shutdown_connections 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 >> 172.21.15.37:0/3072683272 conn(0x556dbe220ac0 msgr2=0x556dbe3c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 shutdown_connections 2024-09-21T06:35:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.812+0000 7f716bd65f40 1 -- 172.21.15.37:0/3072683272 wait complete. 2024-09-21T06:35:30.825 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-21T06:35:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.968+0000 7f37de9eef40 1 Processor -- start 2024-09-21T06:35:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.968+0000 7f37de9eef40 1 -- start start 2024-09-21T06:35:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be654ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5568be586dc0 con 0x5568be654280 2024-09-21T06:35:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568be4d2c60 con 0x5568be5869b0 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37de9eef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5568be586f40 con 0x5568be5ce360 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.969+0000 7f37d8600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51396/0 (socket says 172.21.15.37:51396) 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 -- 172.21.15.37:0/2513465147 learned_addr learned my addr 172.21.15.37:0/2513465147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d7c00640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be654ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d7200640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 -- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 msgr2=0x5568be5afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 -- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 msgr2=0x5568be654ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be654ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 -- 172.21.15.37:0/2513465147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568be4aaf10 con 0x5568be654280 2024-09-21T06:35:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d7c00640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be654ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d8600640 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c680 tx=0x7f37d000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de332471667211b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d5e00640 1 -- 172.21.15.37:0/2513465147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0004020 con 0x5568be654280 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d5e00640 1 -- 172.21.15.37:0/2513465147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37d00042b0 con 0x5568be654280 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.970+0000 7f37d5e00640 1 -- 172.21.15.37:0/2513465147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000aa40 con 0x5568be654280 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 -- 172.21.15.37:0/2513465147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 msgr2=0x5568be65bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c680 tx=0x7f37d000cb40 comp rx=0 tx=0).stop 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 -- 172.21.15.37:0/2513465147 shutdown_connections 2024-09-21T06:35:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be65bd20 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 --2- 172.21.15.37:0/2513465147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be654ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 -- 172.21.15.37:0/2513465147 >> 172.21.15.37:0/2513465147 conn(0x5568be4a3ac0 msgr2=0x5568be5cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 -- 172.21.15.37:0/2513465147 shutdown_connections 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.971+0000 7f37de9eef40 1 -- 172.21.15.37:0/2513465147 wait complete. 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.972+0000 7f37de9eef40 1 Processor -- start 2024-09-21T06:35:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.972+0000 7f37de9eef40 1 -- start start 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be64c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be5b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5568be6506d0 con 0x5568be654280 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568be64bdf0 con 0x5568be5869b0 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37de9eef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5568be6f58d0 con 0x5568be5ce360 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37d8600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be5b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37d8600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be5b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51404/0 (socket says 172.21.15.37:51404) 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37d8600640 1 -- 172.21.15.37:0/3948196366 learned_addr learned my addr 172.21.15.37:0/3948196366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37d7c00640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be64c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.974+0000 7f37d7200640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 -- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 msgr2=0x5568be5afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5568be5ce360 0x5568be5afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 -- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 msgr2=0x5568be64c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568be5869b0 0x5568be64c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 -- 172.21.15.37:0/3948196366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568be6f5b20 con 0x5568be654280 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d8600640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be5b0080 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f37d000afd0 tx=0x7f37d000cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d4a00640 1 -- 172.21.15.37:0/3948196366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000f670 con 0x5568be654280 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d4a00640 1 -- 172.21.15.37:0/3948196366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37d000f810 con 0x5568be654280 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37d4a00640 1 -- 172.21.15.37:0/3948196366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000fc00 con 0x5568be654280 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568be6f5e00 con 0x5568be654280 2024-09-21T06:35:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.975+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568be64b870 con 0x5568be654280 2024-09-21T06:35:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.978+0000 7f37d4a00640 1 -- 172.21.15.37:0/3948196366 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f37d002a440 con 0x5568be654280 2024-09-21T06:35:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.978+0000 7f37d4a00640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f37b8077570 0x7f37b8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.979+0000 7f37d4a00640 1 -- 172.21.15.37:0/3948196366 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f37d009bca0 con 0x5568be654280 2024-09-21T06:35:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.979+0000 7f37de9eef40 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x5568be655c00 0x5568be701c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.979+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- osd_op(unknown.0.0:1 15.5 15:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5568be6f5480 con 0x5568be655c00 2024-09-21T06:35:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.979+0000 7f37d7200640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x5568be655c00 0x5568be701c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.979+0000 7f37d7c00640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f37b8077570 0x7f37b8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.980+0000 7f37d7200640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x5568be655c00 0x5568be701c30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.980+0000 7f37d7c00640 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f37b8077570 0x7f37b8079a80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f37c8055000 tx=0x7f37c8003020 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37d7200640 1 -- 172.21.15.37:0/3948196366 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v104'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f37c400b5a0 con 0x5568be655c00 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x5568be655c00 msgr2=0x5568be701c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x5568be655c00 0x5568be701c30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f37b8077570 msgr2=0x7f37b8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f37b8077570 0x7f37b8079a80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f37c8055000 tx=0x7f37c8003020 comp rx=0 tx=0).stop 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 msgr2=0x5568be5b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 --2- 172.21.15.37:0/3948196366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5568be654280 0x5568be5b0080 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f37d000afd0 tx=0x7f37d000cfc0 comp rx=0 tx=0).stop 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37d8600640 1 -- 172.21.15.37:0/3948196366 reap_dead start 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 shutdown_connections 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.981+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 >> 172.21.15.37:0/3948196366 conn(0x5568be4a3ac0 msgr2=0x5568be65bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.982+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 shutdown_connections 2024-09-21T06:35:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:30.982+0000 7f37de9eef40 1 -- 172.21.15.37:0/3948196366 wait complete. 2024-09-21T06:35:30.994 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-21T06:35:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.136+0000 7ff55e081f40 1 Processor -- start 2024-09-21T06:35:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.136+0000 7ff55e081f40 1 -- start start 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.137+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13ea69b0 0x561a13f7d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13f74280 0x561a13ed1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff55e081f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a13df2c60 con 0x561a13eee360 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff55e081f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561a13e511d0 con 0x561a13ea69b0 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff55e081f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561a13ea6dc0 con 0x561a13f74280 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51430/0 (socket says 172.21.15.37:51430) 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 -- 172.21.15.37:0/2079680593 learned_addr learned my addr 172.21.15.37:0/2079680593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff557200640 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13ea69b0 0x561a13f7d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff557c00640 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13f74280 0x561a13ed1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 -- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13f74280 msgr2=0x561a13ed1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13f74280 0x561a13ed1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 -- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13ea69b0 msgr2=0x561a13f7d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13ea69b0 0x561a13f7d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 -- 172.21.15.37:0/2079680593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a13ee9490 con 0x561a13eee360 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.138+0000 7ff556800640 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff544025f00 tx=0x7ff544002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788826f48352cef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff555400640 1 -- 172.21.15.37:0/2079680593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544056470 con 0x561a13eee360 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff555400640 1 -- 172.21.15.37:0/2079680593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff544056610 con 0x561a13eee360 2024-09-21T06:35:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff555400640 1 -- 172.21.15.37:0/2079680593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544056a60 con 0x561a13eee360 2024-09-21T06:35:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 -- 172.21.15.37:0/2079680593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 msgr2=0x561a13ee8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff544025f00 tx=0x7ff544002fb0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 -- 172.21.15.37:0/2079680593 shutdown_connections 2024-09-21T06:35:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13eee360 0x561a13ee8930 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13f74280 0x561a13ed1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 --2- 172.21.15.37:0/2079680593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13ea69b0 0x561a13f7d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.139+0000 7ff55e081f40 1 -- 172.21.15.37:0/2079680593 >> 172.21.15.37:0/2079680593 conn(0x561a13dc3ac0 msgr2=0x561a13ee8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.140+0000 7ff55e081f40 1 -- 172.21.15.37:0/2079680593 shutdown_connections 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.140+0000 7ff55e081f40 1 -- 172.21.15.37:0/2079680593 wait complete. 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.141+0000 7ff55e081f40 1 Processor -- start 2024-09-21T06:35:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.141+0000 7ff55e081f40 1 -- start start 2024-09-21T06:35:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 0x561a13f668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13eee360 0x561a140060a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13f74280 0x561a140065e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a13ed3720 con 0x561a13ea69b0 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561a13f67580 con 0x561a13f74280 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff55e081f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561a13f6a6c0 con 0x561a13eee360 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff557200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 0x561a13f668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff557200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 0x561a13f668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51438/0 (socket says 172.21.15.37:51438) 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff557200640 1 -- 172.21.15.37:0/531041590 learned_addr learned my addr 172.21.15.37:0/531041590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff556800640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13eee360 0x561a140060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.142+0000 7ff557c00640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13f74280 0x561a140065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 -- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13eee360 msgr2=0x561a140060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561a13eee360 0x561a140060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 -- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13f74280 msgr2=0x561a140065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a13f74280 0x561a140065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 -- 172.21.15.37:0/531041590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a13f6a4a0 con 0x561a13ea69b0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff557200640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 0x561a13f668b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff54800a750 tx=0x7ff54800ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff54fe00640 1 -- 172.21.15.37:0/531041590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800ae60 con 0x561a13ea69b0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff54fe00640 1 -- 172.21.15.37:0/531041590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff548009d50 con 0x561a13ea69b0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.143+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a13ee8b80 con 0x561a13ea69b0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.144+0000 7ff54fe00640 1 -- 172.21.15.37:0/531041590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548017690 con 0x561a13ea69b0 2024-09-21T06:35:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.144+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a13f672c0 con 0x561a13ea69b0 2024-09-21T06:35:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.146+0000 7ff54fe00640 1 -- 172.21.15.37:0/531041590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff548017830 con 0x561a13ea69b0 2024-09-21T06:35:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff54fe00640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff538077570 0x7ff538079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff54fe00640 1 -- 172.21.15.37:0/531041590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7ff54809a6c0 con 0x561a13ea69b0 2024-09-21T06:35:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff556800640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff538077570 0x7ff538079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff55e081f40 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x561a13f7d220 0x561a1401ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- osd_op(unknown.0.0:1 15.1d 15:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x561a13f66f20 con 0x561a13f7d220 2024-09-21T06:35:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.147+0000 7ff557c00640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x561a13f7d220 0x561a1401ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.148+0000 7ff556800640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff538077570 0x7ff538079a80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff544062000 tx=0x7ff544004940 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.148+0000 7ff557c00640 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x561a13f7d220 0x561a1401ddd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff557c00640 1 -- 172.21.15.37:0/531041590 <== osd.0 v2:172.21.15.37:6808/787824175 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v104'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff550010580 con 0x561a13f7d220 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x561a13f7d220 msgr2=0x561a1401ddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x561a13f7d220 0x561a1401ddd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff538077570 msgr2=0x7ff538079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff538077570 0x7ff538079a80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff544062000 tx=0x7ff544004940 comp rx=0 tx=0).stop 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 msgr2=0x561a13f668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff55e081f40 1 --2- 172.21.15.37:0/531041590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a13ea69b0 0x561a13f668b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff54800a750 tx=0x7ff54800ac10 comp rx=0 tx=0).stop 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.149+0000 7ff557c00640 1 -- 172.21.15.37:0/531041590 reap_dead start 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.150+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 shutdown_connections 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.150+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 >> 172.21.15.37:0/531041590 conn(0x561a13dc3ac0 msgr2=0x561a13eed8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.150+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 shutdown_connections 2024-09-21T06:35:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.150+0000 7ff55e081f40 1 -- 172.21.15.37:0/531041590 wait complete. 2024-09-21T06:35:31.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:31.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 -- 172.21.15.37:0/2156744561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb990145cc0 msgr2=0x7fb9900cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 --2- 172.21.15.37:0/2156744561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb990145cc0 0x7fb9900cf030 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb988005fd0 tx=0x7fb98802aa90 comp rx=0 tx=0).stop 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 -- 172.21.15.37:0/2156744561 shutdown_connections 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 --2- 172.21.15.37:0/2156744561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb990145cc0 0x7fb9900cf030 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 --2- 172.21.15.37:0/2156744561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb990160670 0x7fb9900ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 -- 172.21.15.37:0/2156744561 >> 172.21.15.37:0/2156744561 conn(0x7fb9900d0460 msgr2=0x7fb99006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 -- 172.21.15.37:0/2156744561 shutdown_connections 2024-09-21T06:35:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.391+0000 7fb98fe00640 1 -- 172.21.15.37:0/2156744561 wait complete. 2024-09-21T06:35:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.393+0000 7fb98fe00640 1 Processor -- start 2024-09-21T06:35:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.394+0000 7fb98fe00640 1 -- start start 2024-09-21T06:35:31.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9900b2e60 0x7fb9901f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb990145cc0 0x7fb99013e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb99006ff20 con 0x7fb9901417f0 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb99006fda0 con 0x7fb990145cc0 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98fe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900700a0 con 0x7fb9900b2e60 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51452/0 (socket says 172.21.15.37:51452) 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.395+0000 7fb98f400640 1 -- 172.21.15.37:0/849501268 learned_addr learned my addr 172.21.15.37:0/849501268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb986000640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb990145cc0 0x7fb99013e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98ea00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9900b2e60 0x7fb9901f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 -- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9900b2e60 msgr2=0x7fb9901f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9900b2e60 0x7fb9901f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 -- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb990145cc0 msgr2=0x7fb99013e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb990145cc0 0x7fb99013e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 -- 172.21.15.37:0/849501268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990145250 con 0x7fb9901417f0 2024-09-21T06:35:31.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98f400640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb988006010 tx=0x7fb988003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb987400640 1 -- 172.21.15.37:0/849501268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988044070 con 0x7fb9901417f0 2024-09-21T06:35:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb987400640 1 -- 172.21.15.37:0/849501268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb988004040 con 0x7fb9901417f0 2024-09-21T06:35:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9900cb740 con 0x7fb9901417f0 2024-09-21T06:35:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb987400640 1 -- 172.21.15.37:0/849501268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988004430 con 0x7fb9901417f0 2024-09-21T06:35:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.396+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb990141ff0 con 0x7fb9901417f0 2024-09-21T06:35:31.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.399+0000 7fb987400640 1 -- 172.21.15.37:0/849501268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb988047070 con 0x7fb9901417f0 2024-09-21T06:35:31.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.400+0000 7fb987400640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 0x7fb96c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.400+0000 7fb987400640 1 -- 172.21.15.37:0/849501268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fb9880bbd00 con 0x7fb9901417f0 2024-09-21T06:35:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.400+0000 7fb99c8bd740 1 Processor -- start 2024-09-21T06:35:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.401+0000 7fb99c8bd740 1 -- start start 2024-09-21T06:35:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.401+0000 7fb98ea00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 0x7fb96c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.401+0000 7fb98ea00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 0x7fb96c07b0a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb9800015e0 tx=0x7fb980009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf16330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616bae2e0c0 con 0x5616bab069a0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5616bae41f90 con 0x5616baafe7c0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5616bae313c0 con 0x5616bab27680 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.402+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf16330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb986000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51458/0 (socket says 172.21.15.37:51458) 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 -- 172.21.15.37:0/2290305218 learned_addr learned my addr 172.21.15.37:0/2290305218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf16330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:52098/0 (socket says 172.21.15.37:52098) 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb986000640 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:33580/0 (socket says 172.21.15.37:33580) 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 -- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 msgr2=0x5616baf16330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf16330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 -- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 msgr2=0x5616baef9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 -- 172.21.15.37:0/2290305218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bac07a50 con 0x5616bab069a0 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb986000640 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.403+0000 7fb98f400640 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb988083300 tx=0x7fb9880800b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138337d0c97e6b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb97f400640 1 -- 172.21.15.37:0/2290305218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988083930 con 0x5616bab069a0 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb97f400640 1 -- 172.21.15.37:0/2290305218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb98808d2c0 con 0x5616bab069a0 2024-09-21T06:35:31.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb97f400640 1 -- 172.21.15.37:0/2290305218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98808ce30 con 0x5616bab069a0 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb99c8bd740 1 -- 172.21.15.37:0/2290305218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 msgr2=0x5616baef9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb988083300 tx=0x7fb9880800b0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.404+0000 7fb99c8bd740 1 -- 172.21.15.37:0/2290305218 shutdown_connections 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baef9540 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/2290305218 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf16330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 -- 172.21.15.37:0/2290305218 >> 172.21.15.37:0/2290305218 conn(0x5616bab1b7d0 msgr2=0x5616baecf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 -- 172.21.15.37:0/2290305218 shutdown_connections 2024-09-21T06:35:31.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.405+0000 7fb99c8bd740 1 -- 172.21.15.37:0/2290305218 wait complete. 2024-09-21T06:35:31.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.406+0000 7fb99c8bd740 1 -- start start 2024-09-21T06:35:31.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.406+0000 7fb99c8bd740 1 Processor -- start 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616badd6af0 con 0x5616baafe7c0 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5616bae351d0 con 0x5616bab069a0 2024-09-21T06:35:31.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5616baba1a30 con 0x5616bab27680 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.407+0000 7fb99c8bd740 10 client.0 _ll_register_callbacks cb 0x5616baddcb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51460/0 (socket says 172.21.15.37:51460) 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 -- 172.21.15.37:0/1895408932 learned_addr learned my addr 172.21.15.37:0/1895408932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb986000640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 -- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 msgr2=0x5616baf124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 -- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 msgr2=0x5616baf11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 -- 172.21.15.37:0/1895408932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616bab70990 con 0x5616baafe7c0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb98ea00640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb9800039a0 tx=0x7fb980000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9493bc408c0bd017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.408+0000 7fb973e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980027060 con 0x5616baafe7c0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fb980010040 con 0x5616baafe7c0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb98808d2c0 con 0x5616baafe7c0 2024-09-21T06:35:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fb98000ec40 con 0x5616baafe7c0 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 10 client.8559 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x5616bac681e0 con 0x5616baafe7c0 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb98808d2c0 con 0x5616baafe7c0 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fb980013420 con 0x5616baafe7c0 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 10 client.8559 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 10 client.8559 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 20 client.8559 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.409+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baae6250 0x5616baf26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.410+0000 7fb99c8bd740 4 client.8559 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.410+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616bae43ee0 con 0x5616baae6250 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.410+0000 7fb986000640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baae6250 0x5616baf26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.410+0000 7fb986000640 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baae6250 0x5616baf26520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103e05aa6ed43519 server_cookie=4ade934c09b7cb5d in_seq=0 out_seq=0 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.410+0000 7fb973e00640 10 client.8559 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb973e00640 1 -- 172.21.15.37:0/1895408932 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616bae43ee0 con 0x5616baae6250 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb973e00640 10 client.8559 handle_command_reply: tid=0 2024-09-21T06:35:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 client.8559 shutdown 2024-09-21T06:35:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 msgr2=0x5616baee7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb9800039a0 tx=0x7fb980000ed0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 shutdown_connections 2024-09-21T06:35:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baae6250 0x5616baf26520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baf124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/1895408932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616baafe7c0 0x5616baee7790 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb973e00640 0 client.8559 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.411+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 >> 172.21.15.37:0/1895408932 conn(0x5616bab1b7d0 msgr2=0x5616baf17390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.412+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 shutdown_connections 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.412+0000 7fb99c8bd740 1 -- 172.21.15.37:0/1895408932 wait complete. 2024-09-21T06:35:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.412+0000 7fb99c8bd740 20 client.8559 trim_cache size 0 max 16384 2024-09-21T06:35:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.431+0000 7fb99c8bd740 1 Processor -- start 2024-09-21T06:35:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.431+0000 7fb99c8bd740 1 -- start start 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baed0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb990202690 con 0x5616bab069a0 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb990201830 con 0x5616baafe7c0 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb990201f60 con 0x5616bab27680 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb986000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb986000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51466/0 (socket says 172.21.15.37:51466) 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.432+0000 7fb986000640 1 -- 172.21.15.37:0/3099989899 learned_addr learned my addr 172.21.15.37:0/3099989899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb98ea00640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb98f400640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baed0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 -- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 msgr2=0x5616baed0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baed0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 -- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 msgr2=0x5616baef5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 -- 172.21.15.37:0/3099989899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bac05d50 con 0x5616bab069a0 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb98ea00640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb98f400640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baed0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb986000640 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb990141680 tx=0x7fb978005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad87b7c09c69038 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb97ea00640 1 -- 172.21.15.37:0/3099989899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978029070 con 0x5616bab069a0 2024-09-21T06:35:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb97ea00640 1 -- 172.21.15.37:0/3099989899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb978003d50 con 0x5616bab069a0 2024-09-21T06:35:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.433+0000 7fb97ea00640 1 -- 172.21.15.37:0/3099989899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780126b0 con 0x5616bab069a0 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3099989899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 msgr2=0x5616baecff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb990141680 tx=0x7fb978005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3099989899 shutdown_connections 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab069a0 0x5616baecff00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616bab27680 0x5616baed0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3099989899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616baafe7c0 0x5616baef5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3099989899 >> 172.21.15.37:0/3099989899 conn(0x5616bab1b7d0 msgr2=0x5616baece900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3099989899 shutdown_connections 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.434+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3099989899 wait complete. 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.435+0000 7fb99c8bd740 1 -- start start 2024-09-21T06:35:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.435+0000 7fb99c8bd740 1 Processor -- start 2024-09-21T06:35:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616baafe7c0 0x5616baf16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616baec9ef0 con 0x5616bab27680 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb990144f50 con 0x5616bab069a0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5616badfdd80 con 0x5616baafe7c0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb99c8bd740 10 client.0 _ll_register_callbacks cb 0x7fb9800035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb98f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51482/0 (socket says 172.21.15.37:51482) 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb98f400640 1 -- 172.21.15.37:0/3230569008 learned_addr learned my addr 172.21.15.37:0/3230569008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb98ea00640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616baafe7c0 0x5616baf16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.437+0000 7fb986000640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 -- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616baafe7c0 msgr2=0x5616baf16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616baafe7c0 0x5616baf16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 -- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 msgr2=0x5616baf0f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf0f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 -- 172.21.15.37:0/3230569008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616baf98220 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb98f400640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9880ba090 tx=0x7fb988088310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793ac25e9c74910f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb973e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98808d2c0 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fb988004b40 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb978003d50 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fb988083930 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb99c8bd740 10 client.8565 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.438+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x5616baf0e900 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb978003d50 con 0x5616bab27680 2024-09-21T06:35:31.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fb988082b90 con 0x5616bab27680 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 10 client.8565 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 10 client.8565 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 20 client.8565 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baf935d0 0x5616baee6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 4 client.8565 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x5616bae3f990 con 0x5616baf935d0 2024-09-21T06:35:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.439+0000 7fb98ea00640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baf935d0 0x5616baee6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.445 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:35:31.442+0000 7f18fae00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1) n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x561b018f5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.445 INFO:tasks.ceph.mds.b.smithi150.stderr:2024-09-21T06:35:31.442+0000 7f18fae00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/AsLsXsFsx@2},l=8349 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x561b0166c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.440+0000 7fb98ea00640 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baf935d0 0x5616baee6d70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e44dda5fd75fcb server_cookie=f224e260de84dd8e in_seq=0 out_seq=0 2024-09-21T06:35:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.440+0000 7fb973e00640 10 client.8565 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.440+0000 7fb973e00640 1 -- 172.21.15.37:0/3230569008 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616bae3f990 con 0x5616baf935d0 2024-09-21T06:35:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.440+0000 7fb973e00640 10 client.8565 handle_command_reply: tid=0 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.440+0000 7fb99c8bd740 1 client.8565 shutdown 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 msgr2=0x5616baf97ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9880ba090 tx=0x7fb988088310 comp rx=0 tx=0).stop 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 shutdown_connections 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5616baf935d0 0x5616baee6d70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616bab27680 0x5616baf97ce0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616bab069a0 0x5616baf0f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 --2- 172.21.15.37:0/3230569008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616baafe7c0 0x5616baf16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 >> 172.21.15.37:0/3230569008 conn(0x5616bab1b7d0 msgr2=0x5616baed03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb973e00640 0 client.8565 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 shutdown_connections 2024-09-21T06:35:31.448 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:35:31.441+0000 7f3a25800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1) n(v3 rc2024-09-21T06:35:14.958937+0000 6=0+6) (inest mix dirty) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55f942027080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:35:14.958937+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 1 -- 172.21.15.37:0/3230569008 wait complete. 2024-09-21T06:35:31.448 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:35:31.441+0000 7f4147800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1) n(v2 rc2024-09-21T06:35:14.958937+0000 4=0+4) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x556ca3bbf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:35:14.958937+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.441+0000 7fb99c8bd740 20 client.8565 trim_cache size 0 max 16384 2024-09-21T06:35:31.449 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:31.449 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-21T06:35:31.449 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:31.449 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-21T06:35:31.450 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 msgr2=0x7fb96c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 0x7fb96c07b0a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb9800015e0 tx=0x7fb980009040 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 msgr2=0x7fb990073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb988006010 tx=0x7fb988003ce0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 shutdown_connections 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb96c078b90 0x7fb96c07b0a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901417f0 0x7fb990073150 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.442+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb990145cc0 0x7fb99013e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.443+0000 7fb98fe00640 1 --2- 172.21.15.37:0/849501268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9900b2e60 0x7fb9901f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.443+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 >> 172.21.15.37:0/849501268 conn(0x7fb9900d0460 msgr2=0x7fb9900cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.443+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 shutdown_connections 2024-09-21T06:35:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.443+0000 7fb98fe00640 1 -- 172.21.15.37:0/849501268 wait complete. 2024-09-21T06:35:31.452 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-21T06:35:31.443+0000 7f3a2da00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1) n(v4 rc2024-09-21T06:35:14.958937+0000 5=0+5) (inest sync->mix g=1,2) caps={8349=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55f942027600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:35:14.958937+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.452 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-09-21T06:35:31.444+0000 7f414fa00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1) n(v1 rc2024-09-21T06:35:14.958937+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={8349=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x556ca3c18b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:35:14.958937+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:35:31.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:31.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 -- 172.21.15.37:0/2878686838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830005e50 msgr2=0x7fe830006300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 --2- 172.21.15.37:0/2878686838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830005e50 0x7fe830006300 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe844001d90 tx=0x7fe830006940 comp rx=0 tx=0).stop 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe850a00640 1 -- 172.21.15.37:0/2878686838 reap_dead start 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 -- 172.21.15.37:0/2878686838 shutdown_connections 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 -- 172.21.15.37:0/2878686838 >> 172.21.15.37:0/2878686838 conn(0x7fe84c06f920 msgr2=0x7fe84c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 -- 172.21.15.37:0/2878686838 shutdown_connections 2024-09-21T06:35:31.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.698+0000 7fe851400640 1 -- 172.21.15.37:0/2878686838 wait complete. 2024-09-21T06:35:31.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.701+0000 7fe851400640 1 Processor -- start 2024-09-21T06:35:31.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.701+0000 7fe851400640 1 -- start start 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 0x7fe84c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 0x7fe84c1f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c070e70 con 0x7fe84c0b1af0 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c070cf0 con 0x7fe830005e50 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe851400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe84c0cc6c0 con 0x7fe84c1f91d0 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51496/0 (socket says 172.21.15.37:51496) 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.702+0000 7fe843e00640 1 -- 172.21.15.37:0/3036397739 learned_addr learned my addr 172.21.15.37:0/3036397739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe850a00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 0x7fe84c1f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe84be00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 0x7fe84c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 -- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 msgr2=0x7fe84c1f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 0x7fe84c1f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 -- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 msgr2=0x7fe84c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 0x7fe84c0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 -- 172.21.15.37:0/3036397739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c0ca080 con 0x7fe84c0b1af0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe850a00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 0x7fe84c1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe84be00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 0x7fe84c0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe843e00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe834001980 tx=0x7fe8340013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe848c00640 1 -- 172.21.15.37:0/3036397739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834015070 con 0x7fe84c0b1af0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe848c00640 1 -- 172.21.15.37:0/3036397739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe834011940 con 0x7fe84c0b1af0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.703+0000 7fe848c00640 1 -- 172.21.15.37:0/3036397739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834011d90 con 0x7fe84c0b1af0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.704+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84c1f95e0 con 0x7fe84c0b1af0 2024-09-21T06:35:31.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.704+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84c0cbc60 con 0x7fe84c0b1af0 2024-09-21T06:35:31.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.706+0000 7fe848c00640 1 -- 172.21.15.37:0/3036397739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe83400f430 con 0x7fe84c0b1af0 2024-09-21T06:35:31.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.707+0000 7fe848c00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.707+0000 7fe848c00640 1 -- 172.21.15.37:0/3036397739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe83409bd30 con 0x7fe84c0b1af0 2024-09-21T06:35:31.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.707+0000 7fe84be00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.708+0000 7fe859c00740 1 Processor -- start 2024-09-21T06:35:31.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.708+0000 7fe859c00740 1 -- start start 2024-09-21T06:35:31.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.708+0000 7fe84be00640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 0x7fe82807b0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009c50 tx=0x7fe83c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b940390 0x55f10bd2f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd12530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f10bbfbc10 con 0x55f10b91f6b0 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f10bc5a9f0 con 0x55f10b940390 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.709+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f10bc49e20 con 0x55f10b9174d0 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51500/0 (socket says 172.21.15.37:51500) 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 -- 172.21.15.37:0/2296247697 learned_addr learned my addr 172.21.15.37:0/2296247697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe84be00640 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b940390 0x55f10bd2f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe843e00640 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd12530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 -- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 msgr2=0x55f10bd12530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd12530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 -- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b940390 msgr2=0x55f10bd2f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b940390 0x55f10bd2f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 -- 172.21.15.37:0/2296247697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10ba227d0 con 0x55f10b91f6b0 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.710+0000 7fe850a00640 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe844001e80 tx=0x7fe844006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5435d3bde6df38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.711+0000 7fe83be00640 1 -- 172.21.15.37:0/2296247697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844003c70 con 0x55f10b91f6b0 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.711+0000 7fe83be00640 1 -- 172.21.15.37:0/2296247697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8440060b0 con 0x55f10b91f6b0 2024-09-21T06:35:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.711+0000 7fe83be00640 1 -- 172.21.15.37:0/2296247697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8440064c0 con 0x55f10b91f6b0 2024-09-21T06:35:31.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.711+0000 7fe859c00740 1 -- 172.21.15.37:0/2296247697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 msgr2=0x55f10bd12a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.711+0000 7fe859c00740 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe844001e80 tx=0x7fe844006040 comp rx=0 tx=0).stop 2024-09-21T06:35:31.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 -- 172.21.15.37:0/2296247697 shutdown_connections 2024-09-21T06:35:31.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bd12a70 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd12530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 --2- 172.21.15.37:0/2296247697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b940390 0x55f10bd2f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 -- 172.21.15.37:0/2296247697 >> 172.21.15.37:0/2296247697 conn(0x55f10b9344e0 msgr2=0x55f10bce8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 -- 172.21.15.37:0/2296247697 shutdown_connections 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.712+0000 7fe859c00740 1 -- 172.21.15.37:0/2296247697 wait complete. 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.713+0000 7fe859c00740 1 -- start start 2024-09-21T06:35:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.713+0000 7fe859c00740 1 Processor -- start 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.714+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd00420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f10bbef790 con 0x55f10b940390 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f10bc4dc30 con 0x55f10b91f6b0 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f10b9ba740 con 0x55f10b9174d0 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe859c00740 10 client.0 _ll_register_callbacks cb 0x55f10bbff260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51512/0 (socket says 172.21.15.37:51512) 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 -- 172.21.15.37:0/3429978497 learned_addr learned my addr 172.21.15.37:0/3429978497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe843e00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd00420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe84be00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 -- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 msgr2=0x55f10bd263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 -- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 msgr2=0x55f10bd00420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd00420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.715+0000 7fe850a00640 1 -- 172.21.15.37:0/3429978497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10b9896a0 con 0x55f10b940390 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe850a00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe84402ac80 tx=0x7fe844006f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c62a341ee438bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe838c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84404f060 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe844006360 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8440060b0 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fe844055050 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe859c00740 10 client.8577 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55f10ba82f60 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.716+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8440060b0 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fe844058020 con 0x55f10b940390 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 10 client.8577 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 10 client.8577 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 20 client.8577 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10b8fef60 0x55f10bd23ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 4 client.8577 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:31.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f10bc5c940 con 0x55f10b8fef60 2024-09-21T06:35:31.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.717+0000 7fe84be00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10b8fef60 0x55f10bd23ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.718+0000 7fe84be00640 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10b8fef60 0x55f10bd23ef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30d5859b614aeef server_cookie=479d68663fedaf8a in_seq=0 out_seq=0 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.718+0000 7fe838c00640 10 client.8577 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.718+0000 7fe838c00640 1 -- 172.21.15.37:0/3429978497 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f10bc5c940 con 0x55f10b8fef60 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.718+0000 7fe838c00640 10 client.8577 handle_command_reply: tid=0 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.718+0000 7fe859c00740 1 client.8577 shutdown 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 msgr2=0x55f10bd281f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe84402ac80 tx=0x7fe844006f80 comp rx=0 tx=0).stop 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 shutdown_connections 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10b8fef60 0x55f10bd23ef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bd281f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd00420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 --2- 172.21.15.37:0/3429978497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe838c00640 0 client.8577 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 >> 172.21.15.37:0/3429978497 conn(0x55f10b9344e0 msgr2=0x55f10bd0f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 shutdown_connections 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 1 -- 172.21.15.37:0/3429978497 wait complete. 2024-09-21T06:35:31.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.719+0000 7fe859c00740 20 client.8577 trim_cache size 0 max 16384 2024-09-21T06:35:31.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.737+0000 7fe859c00740 1 Processor -- start 2024-09-21T06:35:31.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.737+0000 7fe859c00740 1 -- start start 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd3f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c202b30 con 0x55f10b940390 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c201cd0 con 0x55f10b91f6b0 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fe84c202400 con 0x55f10b9174d0 2024-09-21T06:35:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51516/0 (socket says 172.21.15.37:51516) 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 -- 172.21.15.37:0/3222221242 learned_addr learned my addr 172.21.15.37:0/3222221242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe84be00640 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe843e00640 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd3f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 -- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 msgr2=0x55f10bd011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 -- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 msgr2=0x55f10bd3f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd3f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.739+0000 7fe850a00640 1 -- 172.21.15.37:0/3222221242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10ba20ad0 con 0x55f10b940390 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.740+0000 7fe850a00640 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe84402ac50 tx=0x7fe844001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dafafa6dbc1572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.740+0000 7fe83b400640 1 -- 172.21.15.37:0/3222221242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8440061b0 con 0x55f10b940390 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.740+0000 7fe83b400640 1 -- 172.21.15.37:0/3222221242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe844006350 con 0x55f10b940390 2024-09-21T06:35:31.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.740+0000 7fe83b400640 1 -- 172.21.15.37:0/3222221242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8440336a0 con 0x55f10b940390 2024-09-21T06:35:31.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 -- 172.21.15.37:0/3222221242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 msgr2=0x55f10bce61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe84402ac50 tx=0x7fe844001c60 comp rx=0 tx=0).stop 2024-09-21T06:35:31.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 -- 172.21.15.37:0/3222221242 shutdown_connections 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b940390 0x55f10bce61b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b91f6b0 0x55f10bd3f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 --2- 172.21.15.37:0/3222221242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b9174d0 0x55f10bd011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 -- 172.21.15.37:0/3222221242 >> 172.21.15.37:0/3222221242 conn(0x55f10b9344e0 msgr2=0x55f10bce8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 -- 172.21.15.37:0/3222221242 shutdown_connections 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.741+0000 7fe859c00740 1 -- 172.21.15.37:0/3222221242 wait complete. 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.742+0000 7fe859c00740 1 -- start start 2024-09-21T06:35:31.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.742+0000 7fe859c00740 1 Processor -- start 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 0x55f10bd41dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b940390 0x55f10bdb0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f10bc500d0 con 0x55f10b91f6b0 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1f6430 con 0x55f10b9174d0 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f10bd0f140 con 0x55f10b940390 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe859c00740 10 client.0 _ll_register_callbacks cb 0x7fe83c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51524/0 (socket says 172.21.15.37:51524) 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 -- 172.21.15.37:0/1027085958 learned_addr learned my addr 172.21.15.37:0/1027085958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe84be00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 0x55f10bd41dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe850a00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b940390 0x55f10bdb0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 -- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b940390 msgr2=0x55f10bdb0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b940390 0x55f10bdb0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 -- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 msgr2=0x55f10bd41dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 0x55f10bd41dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.744+0000 7fe843e00640 1 -- 172.21.15.37:0/1027085958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10bce8dc0 con 0x55f10b91f6b0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe84be00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 0x55f10bd41dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe843e00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe834001a30 tx=0x7fe834063e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4408ddd034ed57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe838c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834082060 con 0x55f10b91f6b0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe83407d9c0 con 0x55f10b91f6b0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe844006350 con 0x55f10b91f6b0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fe834020040 con 0x55f10b91f6b0 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe859c00740 10 client.8583 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:31.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55f10bd46990 con 0x55f10b91f6b0 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.745+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe844006350 con 0x55f10b91f6b0 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fe83405f190 con 0x55f10b91f6b0 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 10 client.8583 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 10 client.8583 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 20 client.8583 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10bd27a90 0x55f10bd0dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 4 client.8583 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55f10bc583f0 con 0x55f10bd27a90 2024-09-21T06:35:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe84be00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10bd27a90 0x55f10bd0dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe84be00640 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10bd27a90 0x55f10bd0dd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1299e02c1e31a573 server_cookie=8f0832320a1076b6 in_seq=0 out_seq=0 2024-09-21T06:35:31.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.746+0000 7fe838c00640 10 client.8583 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe838c00640 1 -- 172.21.15.37:0/1027085958 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f10bc583f0 con 0x55f10bd27a90 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe838c00640 10 client.8583 handle_command_reply: tid=0 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 client.8583 shutdown 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 msgr2=0x55f10bdac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe834001a30 tx=0x7fe834063e60 comp rx=0 tx=0).stop 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 shutdown_connections 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f10bd27a90 0x55f10bd0dd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f10b91f6b0 0x55f10bdac610 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f10b940390 0x55f10bdb0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 --2- 172.21.15.37:0/1027085958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f10b9174d0 0x55f10bd41dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe838c00640 0 client.8583 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 >> 172.21.15.37:0/1027085958 conn(0x55f10b9344e0 msgr2=0x55f10bd50fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 shutdown_connections 2024-09-21T06:35:31.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 1 -- 172.21.15.37:0/1027085958 wait complete. 2024-09-21T06:35:31.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.747+0000 7fe859c00740 20 client.8583 trim_cache size 0 max 16384 2024-09-21T06:35:31.753 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:35:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:31.753 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 msgr2=0x7fe82807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 0x7fe82807b0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009c50 tx=0x7fe83c005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 msgr2=0x7fe84c1f8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe834001980 tx=0x7fe8340013b0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 shutdown_connections 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe84c0b1af0 0x7fe84c1f8c90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe84c1f91d0 0x7fe84c1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 --2- 172.21.15.37:0/3036397739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830005e50 0x7fe84c0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 >> 172.21.15.37:0/3036397739 conn(0x7fe84c06f920 msgr2=0x7fe84c155310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 shutdown_connections 2024-09-21T06:35:31.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:31.748+0000 7fe851400640 1 -- 172.21.15.37:0/3036397739 wait complete. 2024-09-21T06:35:31.776 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:31.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-21T06:35:32.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 -- 172.21.15.37:0/1673011162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 msgr2=0x7faa54003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 --2- 172.21.15.37:0/1673011162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa54003ec0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c310 tx=0x7faa54005830 comp rx=0 tx=0).stop 2024-09-21T06:35:32.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 -- 172.21.15.37:0/1673011162 shutdown_connections 2024-09-21T06:35:32.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 --2- 172.21.15.37:0/1673011162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa54003ec0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 -- 172.21.15.37:0/1673011162 >> 172.21.15.37:0/1673011162 conn(0x7faa6c0b46e0 msgr2=0x7faa6c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 -- 172.21.15.37:0/1673011162 shutdown_connections 2024-09-21T06:35:32.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.005+0000 7faa74800640 1 -- 172.21.15.37:0/1673011162 wait complete. 2024-09-21T06:35:32.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.007+0000 7faa74800640 1 Processor -- start 2024-09-21T06:35:32.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.007+0000 7faa74800640 1 -- start start 2024-09-21T06:35:32.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa6c155e70 0x7faa6c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 0x7faa6c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0d0670 con 0x7faa54003a50 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0cda30 con 0x7faa6c152450 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa74800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0d07f0 con 0x7faa6c155e70 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51550/0 (socket says 172.21.15.37:51550) 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 -- 172.21.15.37:0/3409530351 learned_addr learned my addr 172.21.15.37:0/3409530351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa72800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 0x7faa6c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71400640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa6c155e70 0x7faa6c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 -- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa6c155e70 msgr2=0x7faa6c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa6c155e70 0x7faa6c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 -- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 msgr2=0x7faa6c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 0x7faa6c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa71e00640 1 -- 172.21.15.37:0/3409530351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c0b0fe0 con 0x7faa54003a50 2024-09-21T06:35:32.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.009+0000 7faa72800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 0x7faa6c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa71e00640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7faa6002b140 tx=0x7faa60004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa66000640 1 -- 172.21.15.37:0/3409530351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60031040 con 0x7faa54003a50 2024-09-21T06:35:32.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa66000640 1 -- 172.21.15.37:0/3409530351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa60006420 con 0x7faa54003a50 2024-09-21T06:35:32.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c0b12c0 con 0x7faa54003a50 2024-09-21T06:35:32.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa66000640 1 -- 172.21.15.37:0/3409530351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60006c90 con 0x7faa54003a50 2024-09-21T06:35:32.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.010+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c152da0 con 0x7faa54003a50 2024-09-21T06:35:32.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.013+0000 7faa66000640 1 -- 172.21.15.37:0/3409530351 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa60034070 con 0x7faa54003a50 2024-09-21T06:35:32.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.013+0000 7faa66000640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 0x7faa5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.014+0000 7faa66000640 1 -- 172.21.15.37:0/3409530351 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7faa600bb1f0 con 0x7faa54003a50 2024-09-21T06:35:32.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.014+0000 7faa7b7db740 1 Processor -- start 2024-09-21T06:35:32.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.014+0000 7faa7b7db740 1 -- start start 2024-09-21T06:35:32.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.014+0000 7faa71400640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 0x7faa5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.015+0000 7faa71400640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 0x7faa5007b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0b8440 tx=0x7faa5c006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 0x562bf44dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 0x562bf44c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562bf43f61c0 con 0x562bf40c64d0 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562bf4409870 con 0x562bf40ce6b0 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562bf43f8ca0 con 0x562bf40ef390 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51552/0 (socket says 172.21.15.37:51552) 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 -- 172.21.15.37:0/1793501426 learned_addr learned my addr 172.21.15.37:0/1793501426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71e00640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 0x562bf44dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa72800640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 0x562bf44c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 -- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 msgr2=0x562bf44dd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 0x562bf44dd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 -- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 msgr2=0x562bf44c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 0x562bf44c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71400640 1 -- 172.21.15.37:0/1793501426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf41d17d0 con 0x562bf40c64d0 2024-09-21T06:35:32.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa71e00640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 0x562bf44dd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.016+0000 7faa72800640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 0x562bf44c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.017+0000 7faa71400640 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7faa5c012930 tx=0x7faa5c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c176b94fad28966a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.017+0000 7faa5a600640 1 -- 172.21.15.37:0/1793501426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c024030 con 0x562bf40c64d0 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.017+0000 7faa5a600640 1 -- 172.21.15.37:0/1793501426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5c016070 con 0x562bf40c64d0 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.017+0000 7faa5a600640 1 -- 172.21.15.37:0/1793501426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c011bb0 con 0x562bf40c64d0 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 -- 172.21.15.37:0/1793501426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 msgr2=0x562bf44c0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7faa5c012930 tx=0x7faa5c014000 comp rx=0 tx=0).stop 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 -- 172.21.15.37:0/1793501426 shutdown_connections 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44c0220 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ce6b0 0x562bf44c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1793501426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ef390 0x562bf44dd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 -- 172.21.15.37:0/1793501426 >> 172.21.15.37:0/1793501426 conn(0x562bf40e34e0 msgr2=0x562bf4496440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 -- 172.21.15.37:0/1793501426 shutdown_connections 2024-09-21T06:35:32.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.018+0000 7faa7b7db740 1 -- 172.21.15.37:0/1793501426 wait complete. 2024-09-21T06:35:32.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.019+0000 7faa7b7db740 1 -- start start 2024-09-21T06:35:32.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.019+0000 7faa7b7db740 1 Processor -- start 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562bf439f410 con 0x562bf40c64d0 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562bf43fcab0 con 0x562bf40ef390 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562bf4169740 con 0x562bf40ce6b0 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa7b7db740 10 client.0 _ll_register_callbacks cb 0x562bf437c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51556/0 (socket says 172.21.15.37:51556) 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 -- 172.21.15.37:0/1156039852 learned_addr learned my addr 172.21.15.37:0/1156039852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71400640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa72800640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 -- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 msgr2=0x562bf44eb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44eb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 -- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 msgr2=0x562bf44eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71e00640 1 -- 172.21.15.37:0/1156039852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf415a970 con 0x562bf40c64d0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa71400640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44eb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.021+0000 7faa72800640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa71e00640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7faa600b9490 tx=0x7faa600827a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17f85fad06f92f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600a9070 con 0x562bf40c64d0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7faa6006e170 con 0x562bf40c64d0 2024-09-21T06:35:32.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5c016070 con 0x562bf40c64d0 2024-09-21T06:35:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7faa6006e370 con 0x562bf40c64d0 2024-09-21T06:35:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa7b7db740 10 client.8595 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x562bf41386a0 con 0x562bf40c64d0 2024-09-21T06:35:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.022+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5c016070 con 0x562bf40c64d0 2024-09-21T06:35:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7faa6008c290 con 0x562bf40c64d0 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 10 client.8595 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 10 client.8595 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 20 client.8595 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf40adf60 0x562bf44ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 4 client.8595 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bf440b7c0 con 0x562bf40adf60 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa71400640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf40adf60 0x562bf44ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa71400640 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf40adf60 0x562bf44ea610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fbe0699aa5081cb server_cookie=b8681f791636ce52 in_seq=0 out_seq=0 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.023+0000 7faa4ea00640 10 client.8595 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.024+0000 7faa4ea00640 1 -- 172.21.15.37:0/1156039852 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bf440b7c0 con 0x562bf40adf60 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.024+0000 7faa4ea00640 10 client.8595 handle_command_reply: tid=0 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.024+0000 7faa7b7db740 1 client.8595 shutdown 2024-09-21T06:35:32.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.024+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 msgr2=0x562bf44bd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.024+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7faa600b9490 tx=0x7faa600827a0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 shutdown_connections 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf40adf60 0x562bf44ea610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44eb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 --2- 172.21.15.37:0/1156039852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bd130 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa4ea00640 0 client.8595 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 >> 172.21.15.37:0/1156039852 conn(0x562bf40e34e0 msgr2=0x562bf44b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 shutdown_connections 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 1 -- 172.21.15.37:0/1156039852 wait complete. 2024-09-21T06:35:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.025+0000 7faa7b7db740 20 client.8595 trim_cache size 0 max 16384 2024-09-21T06:35:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.045+0000 7faa7b7db740 1 Processor -- start 2024-09-21T06:35:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.045+0000 7faa7b7db740 1 -- start start 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44bd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c2027b0 con 0x562bf40c64d0 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c201970 con 0x562bf40ef390 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c202080 con 0x562bf40ce6b0 2024-09-21T06:35:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51558/0 (socket says 172.21.15.37:51558) 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 -- 172.21.15.37:0/3235567872 learned_addr learned my addr 172.21.15.37:0/3235567872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa72800640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71400640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44bd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 -- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 msgr2=0x562bf44bd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44bd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 -- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 msgr2=0x562bf44de2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44de2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa71e00640 1 -- 172.21.15.37:0/3235567872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf44d6bd0 con 0x562bf40c64d0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.047+0000 7faa72800640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.048+0000 7faa71e00640 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c310 tx=0x7faa6007ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c790955b2c3c1a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.048+0000 7faa59c00640 1 -- 172.21.15.37:0/3235567872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600a9070 con 0x562bf40c64d0 2024-09-21T06:35:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.048+0000 7faa59c00640 1 -- 172.21.15.37:0/3235567872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa60083490 con 0x562bf40c64d0 2024-09-21T06:35:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.048+0000 7faa59c00640 1 -- 172.21.15.37:0/3235567872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60082d90 con 0x562bf40c64d0 2024-09-21T06:35:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 -- 172.21.15.37:0/3235567872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 msgr2=0x562bf44bcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c310 tx=0x7faa6007ec30 comp rx=0 tx=0).stop 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 -- 172.21.15.37:0/3235567872 shutdown_connections 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf44de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40ce6b0 0x562bf44bd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 --2- 172.21.15.37:0/3235567872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40c64d0 0x562bf44bcb30 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 -- 172.21.15.37:0/3235567872 >> 172.21.15.37:0/3235567872 conn(0x562bf40e34e0 msgr2=0x562bf44e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 -- 172.21.15.37:0/3235567872 shutdown_connections 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.049+0000 7faa7b7db740 1 -- 172.21.15.37:0/3235567872 wait complete. 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.050+0000 7faa7b7db740 1 -- start start 2024-09-21T06:35:32.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.050+0000 7faa7b7db740 1 Processor -- start 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 0x562bf44e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf455a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c200a60 con 0x562bf40ce6b0 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562bf455afb0 con 0x562bf40ef390 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x562bf44eb830 con 0x562bf40c64d0 2024-09-21T06:35:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa7b7db740 10 client.0 _ll_register_callbacks cb 0x7faa60001750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51562/0 (socket says 172.21.15.37:51562) 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 -- 172.21.15.37:0/2302914652 learned_addr learned my addr 172.21.15.37:0/2302914652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71e00640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 0x562bf44e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa72800640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf455a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 -- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 msgr2=0x562bf44e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 0x562bf44e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 -- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 msgr2=0x562bf455a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf455a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71400640 1 -- 172.21.15.37:0/2302914652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf44af600 con 0x562bf40ce6b0 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.052+0000 7faa71e00640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 0x562bf44e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa71400640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7faa5c012930 tx=0x7faa5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb9d7f2053b4edc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c011ea0 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7faa5c02c880 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60083490 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7faa5c02ca80 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa7b7db740 10 client.8601 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.053+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x562bf44b8140 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa60083490 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7faa5c02cc20 con 0x562bf40ce6b0 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 10 client.8601 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 10 client.8601 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 20 client.8601 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf455ab60 0x562bf44ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 4 client.8601 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bf4407270 con 0x562bf455ab60 2024-09-21T06:35:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.054+0000 7faa71e00640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf455ab60 0x562bf44ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa71e00640 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf455ab60 0x562bf44ddcf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c55ec9848e9e34e server_cookie=f31c626dcc96894e in_seq=0 out_seq=0 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa4ea00640 10 client.8601 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa4ea00640 1 -- 172.21.15.37:0/2302914652 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562bf4407270 con 0x562bf455ab60 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa4ea00640 10 client.8601 handle_command_reply: tid=0 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 client.8601 shutdown 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 msgr2=0x562bf45595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7faa5c012930 tx=0x7faa5c0118d0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 shutdown_connections 2024-09-21T06:35:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x562bf455ab60 0x562bf44ddcf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562bf40ce6b0 0x562bf45595a0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bf40ef390 0x562bf455a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 --2- 172.21.15.37:0/2302914652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562bf40c64d0 0x562bf44e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa4ea00640 0 client.8601 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 >> 172.21.15.37:0/2302914652 conn(0x562bf40e34e0 msgr2=0x562bf44bcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 shutdown_connections 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.055+0000 7faa7b7db740 1 -- 172.21.15.37:0/2302914652 wait complete. 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa7b7db740 20 client.8601 trim_cache size 0 max 16384 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:32.061 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.062 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 msgr2=0x7faa5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 0x7faa5007b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0b8440 tx=0x7faa5c006b70 comp rx=0 tx=0).stop 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 msgr2=0x7faa6c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7faa6002b140 tx=0x7faa60004bc0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 shutdown_connections 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faa50078b90 0x7faa5007b0a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa6c152450 0x7faa6c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa6c155e70 0x7faa6c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 --2- 172.21.15.37:0/3409530351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa54003a50 0x7faa6c153150 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 >> 172.21.15.37:0/3409530351 conn(0x7faa6c0b46e0 msgr2=0x7faa6c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 shutdown_connections 2024-09-21T06:35:32.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.056+0000 7faa74800640 1 -- 172.21.15.37:0/3409530351 wait complete. 2024-09-21T06:35:32.080 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:32.080 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:32.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-21T06:35:32.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 -- 172.21.15.37:0/366142031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 msgr2=0x7fef2806f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 --2- 172.21.15.37:0/366142031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef2806f450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fef14009860 tx=0x7fef1402a540 comp rx=0 tx=0).stop 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 -- 172.21.15.37:0/366142031 shutdown_connections 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 --2- 172.21.15.37:0/366142031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef280b2ca0 0x7fef280b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 --2- 172.21.15.37:0/366142031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef2806f450 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 -- 172.21.15.37:0/366142031 >> 172.21.15.37:0/366142031 conn(0x7fef280cdca0 msgr2=0x7fef280ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 -- 172.21.15.37:0/366142031 shutdown_connections 2024-09-21T06:35:32.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.309+0000 7fef30200640 1 -- 172.21.15.37:0/366142031 wait complete. 2024-09-21T06:35:32.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.311+0000 7fef30200640 1 Processor -- start 2024-09-21T06:35:32.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.311+0000 7fef30200640 1 -- start start 2024-09-21T06:35:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef280c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef2815a270 0x7fef2815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef28160530 con 0x7fef280b2ca0 2024-09-21T06:35:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef281603b0 con 0x7fef280ae970 2024-09-21T06:35:32.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef30200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fef281606b0 con 0x7fef2815a270 2024-09-21T06:35:32.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-09-21T06:35:32.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.313+0000 7fef2ce00640 1 -- 172.21.15.37:0/3322787539 learned_addr learned my addr 172.21.15.37:0/3322787539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2e200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef2815a270 0x7fef2815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2d800640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef280c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 -- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef2815a270 msgr2=0x7fef2815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef2815a270 0x7fef2815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 -- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 msgr2=0x7fef280c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef280c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 -- 172.21.15.37:0/3322787539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef280c66f0 con 0x7fef280b2ca0 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2d800640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef280c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef2ce00640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fef14009980 tx=0x7fef14033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.314+0000 7fef25600640 1 -- 172.21.15.37:0/3322787539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14031d60 con 0x7fef280b2ca0 2024-09-21T06:35:32.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.315+0000 7fef25600640 1 -- 172.21.15.37:0/3322787539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef14033e10 con 0x7fef280b2ca0 2024-09-21T06:35:32.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.315+0000 7fef25600640 1 -- 172.21.15.37:0/3322787539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14005260 con 0x7fef280b2ca0 2024-09-21T06:35:32.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.315+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef280c69d0 con 0x7fef280b2ca0 2024-09-21T06:35:32.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.315+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2815cbb0 con 0x7fef280b2ca0 2024-09-21T06:35:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.317+0000 7fef25600640 1 -- 172.21.15.37:0/3322787539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef14005400 con 0x7fef280b2ca0 2024-09-21T06:35:32.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.318+0000 7fef25600640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.318+0000 7fef25600640 1 -- 172.21.15.37:0/3322787539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fef140bbf10 con 0x7fef280b2ca0 2024-09-21T06:35:32.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.318+0000 7fef2d800640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.319+0000 7fef3727a740 1 Processor -- start 2024-09-21T06:35:32.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.319+0000 7fef3727a740 1 -- start start 2024-09-21T06:35:32.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.319+0000 7fef2d800640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 0x7fef0807b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000f60 tx=0x7fef1c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d744ca71c0 con 0x55d7449a0390 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d744cba870 con 0x55d74497f6b0 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d744ca9ca0 con 0x55d7449774d0 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51580/0 (socket says 172.21.15.37:51580) 2024-09-21T06:35:32.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef2d800640 1 -- 172.21.15.37:0/1666408604 learned_addr learned my addr 172.21.15.37:0/1666408604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.320+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2e200640 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 -- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 msgr2=0x55d744d75630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d75630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 -- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 msgr2=0x55d744d75b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d75b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 -- 172.21.15.37:0/1666408604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d744a827d0 con 0x55d7449a0390 2024-09-21T06:35:32.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef2d800640 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005100 tx=0x7fef1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91589e9b5cc9cc15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef1a000640 1 -- 172.21.15.37:0/1666408604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0035f0 con 0x55d7449a0390 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef1a000640 1 -- 172.21.15.37:0/1666408604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef1c003880 con 0x55d7449a0390 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.321+0000 7fef1a000640 1 -- 172.21.15.37:0/1666408604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c003c30 con 0x55d7449a0390 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 -- 172.21.15.37:0/1666408604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 msgr2=0x55d744d92960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005100 tx=0x7fef1c011000 comp rx=0 tx=0).stop 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 -- 172.21.15.37:0/1666408604 shutdown_connections 2024-09-21T06:35:32.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d75b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d75630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 --2- 172.21.15.37:0/1666408604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449a0390 0x55d744d92960 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 -- 172.21.15.37:0/1666408604 >> 172.21.15.37:0/1666408604 conn(0x55d7449944e0 msgr2=0x55d744d4b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 -- 172.21.15.37:0/1666408604 shutdown_connections 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.322+0000 7fef3727a740 1 -- 172.21.15.37:0/1666408604 wait complete. 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.323+0000 7fef3727a740 1 -- start start 2024-09-21T06:35:32.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.324+0000 7fef3727a740 1 Processor -- start 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d8c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d4b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d744c50410 con 0x55d7449774d0 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d744cadab0 con 0x55d74497f6b0 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d744a1a740 con 0x55d7449a0390 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef3727a740 10 client.0 _ll_register_callbacks cb 0x55d744c2d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.325+0000 7fef2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51582/0 (socket says 172.21.15.37:51582) 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 -- 172.21.15.37:0/1717805902 learned_addr learned my addr 172.21.15.37:0/1717805902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2e200640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d4b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d8c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 -- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 msgr2=0x55d744d4b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d4b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 -- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 msgr2=0x55d744d8c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d8c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 -- 172.21.15.37:0/1717805902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d744a0b970 con 0x55d7449774d0 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d8c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2e200640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d4b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef2d800640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005040 tx=0x7fef1c001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdc7d7cc9f50888 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.326+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c021920 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fef1c021ac0 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1c003880 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fef1c021cc0 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 10 client.8610 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55d7449e96a0 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef1c003880 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fef1c029020 con 0x55d7449774d0 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 10 client.8610 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 10 client.8610 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:32.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 20 client.8610 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d74495ef60 0x55d744d8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 4 client.8610 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.327+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d744cbc7c0 con 0x55d74495ef60 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.328+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d74495ef60 0x55d744d8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.329+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d74495ef60 0x55d744d8aa00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67693eec3fa0a689 server_cookie=7fb736f82c18d895 in_seq=0 out_seq=0 2024-09-21T06:35:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.329+0000 7fef0ea00640 10 client.8610 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef0ea00640 1 -- 172.21.15.37:0/1717805902 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d744cbc7c0 con 0x55d74495ef60 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef0ea00640 10 client.8610 handle_command_reply: tid=0 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 client.8610 shutdown 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 msgr2=0x55d744d640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005040 tx=0x7fef1c001e20 comp rx=0 tx=0).stop 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 shutdown_connections 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d74495ef60 0x55d744d8aa00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d4b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d74497f6b0 0x55d744d8c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 --2- 172.21.15.37:0/1717805902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7449774d0 0x55d744d640e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef0ea00640 0 client.8610 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 >> 172.21.15.37:0/1717805902 conn(0x55d7449944e0 msgr2=0x55d744d65900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 shutdown_connections 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 1 -- 172.21.15.37:0/1717805902 wait complete. 2024-09-21T06:35:32.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.330+0000 7fef3727a740 20 client.8610 trim_cache size 0 max 16384 2024-09-21T06:35:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.349+0000 7fef3727a740 1 Processor -- start 2024-09-21T06:35:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.349+0000 7fef3727a740 1 -- start start 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d8d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449a0390 0x55d744e09d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef2815f330 con 0x55d74497f6b0 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef280d0bf0 con 0x55d7449a0390 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fef2815ec00 con 0x55d7449774d0 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51590/0 (socket says 172.21.15.37:51590) 2024-09-21T06:35:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 -- 172.21.15.37:0/4205674678 learned_addr learned my addr 172.21.15.37:0/4205674678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2d800640 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d8d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2e200640 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449a0390 0x55d744e09d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 -- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 msgr2=0x55d744d8d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d8d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 -- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449a0390 msgr2=0x55d744e09d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449a0390 0x55d744e09d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.351+0000 7fef2ce00640 1 -- 172.21.15.37:0/4205674678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d744d483b0 con 0x55d74497f6b0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.352+0000 7fef2ce00640 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fef14009860 tx=0x7fef1402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900bd2c3b047564f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.352+0000 7fef19600640 1 -- 172.21.15.37:0/4205674678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14048040 con 0x55d74497f6b0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.352+0000 7fef19600640 1 -- 172.21.15.37:0/4205674678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef14083b60 con 0x55d74497f6b0 2024-09-21T06:35:32.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.352+0000 7fef19600640 1 -- 172.21.15.37:0/4205674678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14088590 con 0x55d74497f6b0 2024-09-21T06:35:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 -- 172.21.15.37:0/4205674678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 msgr2=0x55d744e09800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fef14009860 tx=0x7fef1402a540 comp rx=0 tx=0).stop 2024-09-21T06:35:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 -- 172.21.15.37:0/4205674678 shutdown_connections 2024-09-21T06:35:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744e09800 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449a0390 0x55d744e09d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 --2- 172.21.15.37:0/4205674678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449774d0 0x55d744d8d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 -- 172.21.15.37:0/4205674678 >> 172.21.15.37:0/4205674678 conn(0x55d7449944e0 msgr2=0x55d744d64a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 -- 172.21.15.37:0/4205674678 shutdown_connections 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.353+0000 7fef3727a740 1 -- 172.21.15.37:0/4205674678 wait complete. 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.354+0000 7fef3727a740 1 -- start start 2024-09-21T06:35:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.354+0000 7fef3727a740 1 Processor -- start 2024-09-21T06:35:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449774d0 0x55d744e0c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d9c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef281f76e0 con 0x55d74497f6b0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d744e0f610 con 0x55d7449774d0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55d744d4c5c0 con 0x55d7449a0390 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef3727a740 10 client.0 _ll_register_callbacks cb 0x7fef14005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51604/0 (socket says 172.21.15.37:51604) 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 -- 172.21.15.37:0/1779072889 learned_addr learned my addr 172.21.15.37:0/1779072889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2e200640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d9c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2d800640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449774d0 0x55d744e0c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 -- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 msgr2=0x55d744d9c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d9c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 -- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449774d0 msgr2=0x55d744e0c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449774d0 0x55d744e0c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.356+0000 7fef2ce00640 1 -- 172.21.15.37:0/1779072889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d744d89c70 con 0x55d74497f6b0 2024-09-21T06:35:32.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef2ce00640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fef14080dd0 tx=0x7fef140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf0d26f634f56e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14048040 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fef14088130 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef14083b60 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fef14088330 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 10 client.8616 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55d744d88640 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef14083b60 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fef14083dc0 con 0x55d74497f6b0 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 10 client.8616 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 10 client.8616 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 20 client.8616 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d744e0f160 0x55d744e129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 4 client.8616 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.357+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d744cb8270 con 0x55d744e0f160 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.358+0000 7fef2d800640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d744e0f160 0x55d744e129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.358+0000 7fef2d800640 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d744e0f160 0x55d744e129c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cf963171dcc1b14 server_cookie=96d48e825e61662 in_seq=0 out_seq=0 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.358+0000 7fef0ea00640 10 client.8616 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef0ea00640 1 -- 172.21.15.37:0/1779072889 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d744cb8270 con 0x55d744e0f160 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef0ea00640 10 client.8616 handle_command_reply: tid=0 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 client.8616 shutdown 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 msgr2=0x55d744d9c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fef14080dd0 tx=0x7fef140045e0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 shutdown_connections 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55d744e0f160 0x55d744e129c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d74497f6b0 0x55d744d9c0b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7449a0390 0x55d744d9c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 --2- 172.21.15.37:0/1779072889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7449774d0 0x55d744e0c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef0ea00640 0 client.8616 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 >> 172.21.15.37:0/1779072889 conn(0x55d7449944e0 msgr2=0x55d744d8fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 shutdown_connections 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 1 -- 172.21.15.37:0/1779072889 wait complete. 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.359+0000 7fef3727a740 20 client.8616 trim_cache size 0 max 16384 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:32.364 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.365 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 msgr2=0x7fef0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 0x7fef0807b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000f60 tx=0x7fef1c009040 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 msgr2=0x7fef2815b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fef14009980 tx=0x7fef14033c20 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 shutdown_connections 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef280b2ca0 0x7fef2815b7c0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef2815a270 0x7fef2815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 --2- 172.21.15.37:0/3322787539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef280ae970 0x7fef280c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 >> 172.21.15.37:0/3322787539 conn(0x7fef280cdca0 msgr2=0x7fef280746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 shutdown_connections 2024-09-21T06:35:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.360+0000 7fef30200640 1 -- 172.21.15.37:0/3322787539 wait complete. 2024-09-21T06:35:32.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:32.385 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:32.385 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 -- 172.21.15.37:0/3739552920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 msgr2=0x7fc7e0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3739552920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e0074620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005fd0 tx=0x7fc7d402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 -- 172.21.15.37:0/3739552920 shutdown_connections 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3739552920 >> v2:172.21.15.150:3300/0 conn(0x7fc7e006e500 0x7fc7e00cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3739552920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e0074620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 -- 172.21.15.37:0/3739552920 >> 172.21.15.37:0/3739552920 conn(0x7fc7e00b1b90 msgr2=0x7fc7e00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 -- 172.21.15.37:0/3739552920 shutdown_connections 2024-09-21T06:35:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.610+0000 7fc7e7600640 1 -- 172.21.15.37:0/3739552920 wait complete. 2024-09-21T06:35:32.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.613+0000 7fc7e7600640 1 Processor -- start 2024-09-21T06:35:32.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.613+0000 7fc7e7600640 1 -- start start 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e0155e80 0x7fc7e015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00cf7d0 con 0x7fc7e0074b60 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00cf650 con 0x7fc7e0155e80 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e7600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00cf950 con 0x7fc7e0156a50 2024-09-21T06:35:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.614+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51606/0 (socket says 172.21.15.37:51606) 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 -- 172.21.15.37:0/3188092036 learned_addr learned my addr 172.21.15.37:0/3188092036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e5600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e0155e80 0x7fc7e015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 -- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 msgr2=0x7fc7e015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 -- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e0155e80 msgr2=0x7fc7e015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e0155e80 0x7fc7e015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 -- 172.21.15.37:0/3188092036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e015ca20 con 0x7fc7e0074b60 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.615+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.616+0000 7fc7dcc00640 1 -- 172.21.15.37:0/3188092036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d403e030 con 0x7fc7e0074b60 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.616+0000 7fc7dcc00640 1 -- 172.21.15.37:0/3188092036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7d4003ae0 con 0x7fc7e0074b60 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.616+0000 7fc7dcc00640 1 -- 172.21.15.37:0/3188092036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4035040 con 0x7fc7e0074b60 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.616+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e00c38b0 con 0x7fc7e0074b60 2024-09-21T06:35:32.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.616+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e0155ab0 con 0x7fc7e0074b60 2024-09-21T06:35:32.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.618+0000 7fc7dcc00640 1 -- 172.21.15.37:0/3188092036 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc7d4041030 con 0x7fc7e0074b60 2024-09-21T06:35:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.619+0000 7fc7dcc00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 0x7fc7c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.619+0000 7fc7dcc00640 1 -- 172.21.15.37:0/3188092036 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc7d40bb480 con 0x7fc7e0074b60 2024-09-21T06:35:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.620+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 0x7fc7c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.620+0000 7fc7ee7c8740 1 Processor -- start 2024-09-21T06:35:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.620+0000 7fc7ee7c8740 1 -- start start 2024-09-21T06:35:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.620+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 0x7fc7c007b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00af870 tx=0x7fc7cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b65687390 0x558b65a75550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 0x558b65a58220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b6598e1c0 con 0x558b656666b0 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b659a1870 con 0x558b6565e4d0 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.621+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558b65990ca0 con 0x558b65687390 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51620/0 (socket says 172.21.15.37:51620) 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 -- 172.21.15.37:0/2594324257 learned_addr learned my addr 172.21.15.37:0/2594324257 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b65687390 0x558b65a75550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 0x558b65a58220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 -- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b65687390 msgr2=0x558b65a75550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b65687390 0x558b65a75550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 -- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 msgr2=0x558b65a58220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 0x558b65a58220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 -- 172.21.15.37:0/2594324257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b657697d0 con 0x558b656666b0 2024-09-21T06:35:32.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 0x558b65a58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7e5600640 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009f80 tx=0x7fc7d80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4687f049f36cb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.622+0000 7fc7d1200640 1 -- 172.21.15.37:0/2594324257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8003c30 con 0x558b656666b0 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7d1200640 1 -- 172.21.15.37:0/2594324257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7d8003dd0 con 0x558b656666b0 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7d1200640 1 -- 172.21.15.37:0/2594324257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8004250 con 0x558b656666b0 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2594324257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 msgr2=0x558b65a58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009f80 tx=0x7fc7d80118d0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2594324257 shutdown_connections 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a58760 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.623+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b6565e4d0 0x558b65a58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.624+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2594324257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b65687390 0x558b65a75550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.624+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2594324257 >> 172.21.15.37:0/2594324257 conn(0x558b6567b4e0 msgr2=0x558b65a2e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.624+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2594324257 shutdown_connections 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.624+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2594324257 wait complete. 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.625+0000 7fc7ee7c8740 1 -- start start 2024-09-21T06:35:32.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.625+0000 7fc7ee7c8740 1 Processor -- start 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65a2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b65937410 con 0x558b6565e4d0 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b65994ab0 con 0x558b65687390 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558b65701740 con 0x558b656666b0 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.626+0000 7fc7ee7c8740 10 client.0 _ll_register_callbacks cb 0x558b65914360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51622/0 (socket says 172.21.15.37:51622) 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1240525171 learned_addr learned my addr 172.21.15.37:0/1240525171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65a2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e5600640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a46590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 msgr2=0x558b65a2d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65a2d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 msgr2=0x558b65a46590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a46590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1240525171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b656f2970 con 0x558b6565e4d0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e5600640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40802b0 tx=0x7fc7d4082870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204ff3c340d9e3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.627+0000 7fc7c5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4048040 con 0x558b6565e4d0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d8003dd0 con 0x558b6565e4d0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc7d40832a0 con 0x558b6565e4d0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fc7d40a93b0 con 0x558b6565e4d0 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 10 client.8625 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x558b656d06a0 con 0x558b6565e4d0 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7d8003dd0 con 0x558b6565e4d0 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fc7d408c520 con 0x558b6565e4d0 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 10 client.8625 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 10 client.8625 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 20 client.8625 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65645f60 0x558b65a855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 4 client.8625 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.628+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b659a37c0 con 0x558b65645f60 2024-09-21T06:35:32.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.629+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65645f60 0x558b65a855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.629+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65645f60 0x558b65a855d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454942a9844b4f2a server_cookie=a00065daf43ef1f8 in_seq=0 out_seq=0 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.629+0000 7fc7c5600640 10 client.8625 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.630+0000 7fc7c5600640 1 -- 172.21.15.37:0/1240525171 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b659a37c0 con 0x558b65645f60 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.630+0000 7fc7c5600640 10 client.8625 handle_command_reply: tid=0 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.630+0000 7fc7ee7c8740 1 client.8625 shutdown 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 msgr2=0x558b65a6e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40802b0 tx=0x7fc7d4082870 comp rx=0 tx=0).stop 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 shutdown_connections 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65645f60 0x558b65a855d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65a2d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7c5600640 0 client.8625 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:32.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1240525171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65a6e980 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 >> 172.21.15.37:0/1240525171 conn(0x558b6567b4e0 msgr2=0x558b65a479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 shutdown_connections 2024-09-21T06:35:32.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1240525171 wait complete. 2024-09-21T06:35:32.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.631+0000 7fc7ee7c8740 20 client.8625 trim_cache size 0 max 16384 2024-09-21T06:35:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.650+0000 7fc7ee7c8740 1 Processor -- start 2024-09-21T06:35:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.650+0000 7fc7ee7c8740 1 -- start start 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 0x558b65a45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a8a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0202680 con 0x558b656666b0 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0201820 con 0x558b65687390 2024-09-21T06:35:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e0201f50 con 0x558b6565e4d0 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.651+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51624/0 (socket says 172.21.15.37:51624) 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 -- 172.21.15.37:0/2031582206 learned_addr learned my addr 172.21.15.37:0/2031582206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 -- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 msgr2=0x558b65a45f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 0x558b65a45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7e5600640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a8a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 0x558b65a45f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 -- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 msgr2=0x558b65a8a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a8a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 -- 172.21.15.37:0/2031582206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b65a851c0 con 0x558b656666b0 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 0x558b65a45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7e5600640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a8a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000f60 tx=0x7fc7cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2532dd6cc8033051 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.652+0000 7fc7c7e00640 1 -- 172.21.15.37:0/2031582206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc02f070 con 0x558b656666b0 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7c7e00640 1 -- 172.21.15.37:0/2031582206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7cc01b040 con 0x558b656666b0 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7c7e00640 1 -- 172.21.15.37:0/2031582206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc023760 con 0x558b656666b0 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2031582206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 msgr2=0x558b65a464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000f60 tx=0x7fc7cc012000 comp rx=0 tx=0).stop 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2031582206 shutdown_connections 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b656666b0 0x558b65a464d0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65a8a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/2031582206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b6565e4d0 0x558b65a45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2031582206 >> 172.21.15.37:0/2031582206 conn(0x558b6567b4e0 msgr2=0x558b65a47150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2031582206 shutdown_connections 2024-09-21T06:35:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.653+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/2031582206 wait complete. 2024-09-21T06:35:32.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.654+0000 7fc7ee7c8740 1 -- start start 2024-09-21T06:35:32.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.655+0000 7fc7ee7c8740 1 Processor -- start 2024-09-21T06:35:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65af6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65af66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b65a2fa90 con 0x558b6565e4d0 2024-09-21T06:35:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0200890 con 0x558b65687390 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558b65af2e20 con 0x558b656666b0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7ee7c8740 10 client.0 _ll_register_callbacks cb 0x7fc7d407f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51632/0 (socket says 172.21.15.37:51632) 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1019693779 learned_addr learned my addr 172.21.15.37:0/1019693779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.656+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65af6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e5600640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65af66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 msgr2=0x558b65af6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65af6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 msgr2=0x558b65af66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65af66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 -- 172.21.15.37:0/1019693779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b65a73500 con 0x558b6565e4d0 2024-09-21T06:35:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7e4c00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4034040 tx=0x7fc7d4082a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6916aeb30d249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7c5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4048070 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc7d40832a0 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7cc01b040 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.657+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fc7d4087a70 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 10 client.8631 fetch_fsmap learned FSMap version 116 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x558b65a73500 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7cc01b040 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 116) ==== 3558+0+0 (secure 0 0 0) 0x7fc7d40a5040 con 0x558b6565e4d0 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 10 client.8631 fetch_fsmap finished waiting for FSMap version 116 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 10 client.8631 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 20 client.8631 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65af2750 0x558b65a27830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 4 client.8631 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b65a70e40 con 0x558b65af2750 2024-09-21T06:35:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.658+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65af2750 0x558b65a27830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7dfe00640 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65af2750 0x558b65a27830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a454ad6ad541429 server_cookie=9fbb3c9efc275259 in_seq=0 out_seq=0 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7c5600640 10 client.8631 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7c5600640 1 -- 172.21.15.37:0/1019693779 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x558b65a70e40 con 0x558b65af2750 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7c5600640 10 client.8631 handle_command_reply: tid=0 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 client.8631 shutdown 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 msgr2=0x558b65ae91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4034040 tx=0x7fc7d4082a20 comp rx=0 tx=0).stop 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 shutdown_connections 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558b65af2750 0x558b65a27830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b65687390 0x558b65af66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558b656666b0 0x558b65af6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 --2- 172.21.15.37:0/1019693779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b6565e4d0 0x558b65ae91d0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 >> 172.21.15.37:0/1019693779 conn(0x558b6567b4e0 msgr2=0x558b65a76c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.659+0000 7fc7c5600640 0 client.8631 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 shutdown_connections 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7ee7c8740 1 -- 172.21.15.37:0/1019693779 wait complete. 2024-09-21T06:35:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7ee7c8740 20 client.8631 trim_cache size 0 max 16384 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:35:31.440721+0000", 2024-09-21T06:35:32.666 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:32.667 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 msgr2=0x7fc7c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 0x7fc7c007b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00af870 tx=0x7fc7cc009040 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 msgr2=0x7fc7e00c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40035f0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 shutdown_connections 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc7c0078b90 0x7fc7c007b0a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e0155e80 0x7fc7e015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7e0156a50 0x7fc7e015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 --2- 172.21.15.37:0/3188092036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7e0074b60 0x7fc7e00c3370 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.660+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 >> 172.21.15.37:0/3188092036 conn(0x7fc7e00b1b90 msgr2=0x7fc7e00afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.661+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 shutdown_connections 2024-09-21T06:35:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.661+0000 7fc7e7600640 1 -- 172.21.15.37:0/3188092036 wait complete. 2024-09-21T06:35:32.687 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:32.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 -- 172.21.15.37:0/2213436924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc003a50 msgr2=0x7f71bc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 --2- 172.21.15.37:0/2213436924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc003a50 0x7f71bc003ec0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f71c800c4a0 tx=0x7f71bc005830 comp rx=0 tx=0).stop 2024-09-21T06:35:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 -- 172.21.15.37:0/2213436924 shutdown_connections 2024-09-21T06:35:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 --2- 172.21.15.37:0/2213436924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc003a50 0x7f71bc003ec0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 -- 172.21.15.37:0/2213436924 >> 172.21.15.37:0/2213436924 conn(0x7f71d806f940 msgr2=0x7f71d80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 -- 172.21.15.37:0/2213436924 shutdown_connections 2024-09-21T06:35:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.913+0000 7f71de000640 1 -- 172.21.15.37:0/2213436924 wait complete. 2024-09-21T06:35:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.915+0000 7f71de000640 1 Processor -- start 2024-09-21T06:35:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.915+0000 7f71de000640 1 -- start start 2024-09-21T06:35:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 0x7f71d8156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 0x7f71d814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80b4da0 con 0x7f71d814dd70 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80cd060 con 0x7f71bc003a50 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71de000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80b4f20 con 0x7f71d814d420 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71dd600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71dd600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51660/0 (socket says 172.21.15.37:51660) 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71dd600640 1 -- 172.21.15.37:0/1730323643 learned_addr learned my addr 172.21.15.37:0/1730323643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71cfe00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 0x7f71d814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.917+0000 7f71dcc00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 0x7f71d8156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 -- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 msgr2=0x7f71d814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 0x7f71d814d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 -- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 msgr2=0x7f71d8156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 0x7f71d8156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 -- 172.21.15.37:0/1730323643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81559f0 con 0x7f71d814dd70 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71cfe00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 0x7f71d814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dcc00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 0x7f71d8156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71dd600640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f71d815dc60 tx=0x7f71d00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0011870 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d0017070 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d814aae0 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0011dc0 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.918+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d8072c00 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.919+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d80cddc0 con 0x7f71d814dd70 2024-09-21T06:35:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.921+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71d001c460 con 0x7f71d814dd70 2024-09-21T06:35:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.922+0000 7f71d5600640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 0x7f71b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.922+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f71d009b7d0 con 0x7f71d814dd70 2024-09-21T06:35:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.922+0000 7f71dcc00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 0x7f71b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.923+0000 7f71dcc00640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 0x7f71b407b0d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f71c8033000 tx=0x7f71d815d220 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:32.926+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71d00639e0 con 0x7f71d814dd70 2024-09-21T06:35:33.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.164+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71d814e180 con 0x7f71d814dd70 2024-09-21T06:35:33.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.165+0000 7f71d5600640 1 -- 172.21.15.37:0/1730323643 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5278 (secure 0 0 0) 0x7f71d00632e0 con 0x7f71d814dd70 2024-09-21T06:35:33.167 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:33.167 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:30.743657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:33.167 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 116 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 msgr2=0x7f71b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 0x7f71b407b0d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f71c8033000 tx=0x7f71d815d220 comp rx=0 tx=0).stop 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 msgr2=0x7f71d8073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f71d815dc60 tx=0x7f71d00019b0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 shutdown_connections 2024-09-21T06:35:33.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f71b4078bc0 0x7f71b407b0d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d814dd70 0x7f71d8073400 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d814d420 0x7f71d814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 --2- 172.21.15.37:0/1730323643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71bc003a50 0x7f71d8156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 >> 172.21.15.37:0/1730323643 conn(0x7f71d806f940 msgr2=0x7f71d80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 shutdown_connections 2024-09-21T06:35:33.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.168+0000 7f71de000640 1 -- 172.21.15.37:0/1730323643 wait complete. 2024-09-21T06:35:33.193 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:33.193 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 -- 172.21.15.37:0/2809345717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 msgr2=0x7f5bd0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 --2- 172.21.15.37:0/2809345717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bd0004c10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0cd8c0 tx=0x7f5bd0005830 comp rx=0 tx=0).stop 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 -- 172.21.15.37:0/2809345717 shutdown_connections 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 --2- 172.21.15.37:0/2809345717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bd0004c10 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 -- 172.21.15.37:0/2809345717 >> 172.21.15.37:0/2809345717 conn(0x7f5bec0cca40 msgr2=0x7f5bec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 -- 172.21.15.37:0/2809345717 shutdown_connections 2024-09-21T06:35:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.417+0000 7f5bf1400640 1 -- 172.21.15.37:0/2809345717 wait complete. 2024-09-21T06:35:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.420+0000 7f5bf1400640 1 Processor -- start 2024-09-21T06:35:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.420+0000 7f5bf1400640 1 -- start start 2024-09-21T06:35:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bec154900 0x7f5bec0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 0x7f5bec0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0cc0b0 con 0x7f5bd0004800 2024-09-21T06:35:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0cbf30 con 0x7f5bec0ca9d0 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.421+0000 7f5bf1400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec0cc230 con 0x7f5bec154900 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51674/0 (socket says 172.21.15.37:51674) 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 -- 172.21.15.37:0/1301759624 learned_addr learned my addr 172.21.15.37:0/1301759624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5be3e00640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bec154900 0x7f5bec0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5beb200640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 0x7f5bec0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 -- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bec154900 msgr2=0x7f5bec0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bec154900 0x7f5bec0b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 -- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 msgr2=0x7f5bec0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 0x7f5bec0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 -- 172.21.15.37:0/1301759624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec0b3a70 con 0x7f5bd0004800 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5beb200640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 0x7f5bec0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.422+0000 7f5bea800640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e5c0 tx=0x7f5bdc00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.423+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc015070 con 0x7f5bd0004800 2024-09-21T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.423+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bdc010980 con 0x7f5bd0004800 2024-09-21T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.423+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec09fc30 con 0x7f5bd0004800 2024-09-21T06:35:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.423+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc010dd0 con 0x7f5bd0004800 2024-09-21T06:35:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.423+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec15ed60 con 0x7f5bd0004800 2024-09-21T06:35:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.424+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec06dfb0 con 0x7f5bd0004800 2024-09-21T06:35:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.425+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5bdc018070 con 0x7f5bd0004800 2024-09-21T06:35:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.426+0000 7f5be2a00640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 0x7f5bc807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.426+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5bdc09b1d0 con 0x7f5bd0004800 2024-09-21T06:35:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.427+0000 7f5be3e00640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 0x7f5bc807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.427+0000 7f5be3e00640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 0x7f5bc807b0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4032000 tx=0x7f5bd4001360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.431+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bdc062e20 con 0x7f5bd0004800 2024-09-21T06:35:33.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.704+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5bec15a070 con 0x7f5bd0004800 2024-09-21T06:35:33.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.705+0000 7f5be2a00640 1 -- 172.21.15.37:0/1301759624 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1621 (secure 0 0 0) 0x7f5bdc067850 con 0x7f5bd0004800 2024-09-21T06:35:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:33.707 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":63,"data_bytes":668443,"bytes_used":267206656,"bytes_avail":772826906624,"bytes_total":773094113280,"write_bytes_sec":5203,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-09-21T06:34:45.201307+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 msgr2=0x7f5bc807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 0x7f5bc807b0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4032000 tx=0x7f5bd4001360 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 msgr2=0x7f5bec0b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e5c0 tx=0x7f5bdc00ea80 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 shutdown_connections 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5bc8078bc0 0x7f5bc807b0d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bec0ca9d0 0x7f5bec0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bec154900 0x7f5bec0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 --2- 172.21.15.37:0/1301759624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bd0004800 0x7f5bec0b0fc0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.707+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 >> 172.21.15.37:0/1301759624 conn(0x7f5bec0cca40 msgr2=0x7f5bec15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.708+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 shutdown_connections 2024-09-21T06:35:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:33.708+0000 7f5bf1400640 1 -- 172.21.15.37:0/1301759624 wait complete. 2024-09-21T06:35:33.732 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-21T06:35:34.733 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:34.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 -- 172.21.15.37:0/1381354292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 msgr2=0x7f4f60073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 --2- 172.21.15.37:0/1381354292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60073dd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4f50005fd0 tx=0x7f4f5002a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 -- 172.21.15.37:0/1381354292 shutdown_connections 2024-09-21T06:35:34.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 --2- 172.21.15.37:0/1381354292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60073dd0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 -- 172.21.15.37:0/1381354292 >> 172.21.15.37:0/1381354292 conn(0x7f4f600cca40 msgr2=0x7f4f600cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:34.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 -- 172.21.15.37:0/1381354292 shutdown_connections 2024-09-21T06:35:34.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.967+0000 7f4f65c00640 1 -- 172.21.15.37:0/1381354292 wait complete. 2024-09-21T06:35:34.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.969+0000 7f4f65c00640 1 Processor -- start 2024-09-21T06:35:34.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.969+0000 7f4f65c00640 1 -- start start 2024-09-21T06:35:34.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f600cbc10 0x7f4f6009a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 0x7f4f6009cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6006f670 con 0x7f4f60074cb0 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6006f4f0 con 0x7f4f600cbc10 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f65c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6006f7f0 con 0x7f4f600b3820 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37302/0 (socket says 172.21.15.37:37302) 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 -- 172.21.15.37:0/2922599338 learned_addr learned my addr 172.21.15.37:0/2922599338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f57e00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f600cbc10 0x7f4f6009a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5f400640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 0x7f4f6009cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 -- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 msgr2=0x7f4f6009cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 0x7f4f6009cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 -- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f600cbc10 msgr2=0x7f4f6009a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f600cbc10 0x7f4f6009a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5ea00640 1 -- 172.21.15.37:0/2922599338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f601f7220 con 0x7f4f60074cb0 2024-09-21T06:35:34.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.971+0000 7f4f5f400640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 0x7f4f6009cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f5ea00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5003e030 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f50003ca0 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6009d6b0 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50035040 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.972+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f600b3020 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.973+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f600b2e10 con 0x7f4f60074cb0 2024-09-21T06:35:34.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.974+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f50041030 con 0x7f4f60074cb0 2024-09-21T06:35:34.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.976+0000 7f4f56a00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 0x7f4f3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.976+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4f500bb530 con 0x7f4f60074cb0 2024-09-21T06:35:34.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.976+0000 7f4f57e00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 0x7f4f3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.977+0000 7f4f57e00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 0x7f4f3c07b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4f6015c730 tx=0x7f4f48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:34.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:34.981+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f500837d0 con 0x7f4f60074cb0 2024-09-21T06:35:35.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.216+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f60073dd0 con 0x7f4f60074cb0 2024-09-21T06:35:35.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.217+0000 7f4f56a00640 1 -- 172.21.15.37:0/2922599338 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5278 (secure 0 0 0) 0x7f4f500830d0 con 0x7f4f60074cb0 2024-09-21T06:35:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:35.220 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:30.743657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:35.220 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 116 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 msgr2=0x7f4f3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 0x7f4f3c07b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4f6015c730 tx=0x7f4f48009040 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 msgr2=0x7f4f60158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f500038a0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 shutdown_connections 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f3c078b90 0x7f4f3c07b0a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f600b3820 0x7f4f6009cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f600cbc10 0x7f4f6009a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 --2- 172.21.15.37:0/2922599338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f60074cb0 0x7f4f60158080 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.219+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 >> 172.21.15.37:0/2922599338 conn(0x7f4f600cca40 msgr2=0x7f4f6006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.220+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 shutdown_connections 2024-09-21T06:35:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.220+0000 7f4f65c00640 1 -- 172.21.15.37:0/2922599338 wait complete. 2024-09-21T06:35:35.244 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:35.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.468+0000 7f9d73e00640 1 -- 172.21.15.37:0/2073577039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d74152530 msgr2=0x7f9d740b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.468+0000 7f9d73e00640 1 --2- 172.21.15.37:0/2073577039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d74152530 0x7f9d740b3490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005fd0 tx=0x7f9d5c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 -- 172.21.15.37:0/2073577039 shutdown_connections 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 --2- 172.21.15.37:0/2073577039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d74152530 0x7f9d740b3490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 --2- 172.21.15.37:0/2073577039 >> v2:172.21.15.150:3300/0 conn(0x7f9d740b39d0 0x7f9d7406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 -- 172.21.15.37:0/2073577039 >> 172.21.15.37:0/2073577039 conn(0x7f9d7406f940 msgr2=0x7f9d740b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 -- 172.21.15.37:0/2073577039 shutdown_connections 2024-09-21T06:35:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.469+0000 7f9d73e00640 1 -- 172.21.15.37:0/2073577039 wait complete. 2024-09-21T06:35:35.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.471+0000 7f9d73e00640 1 Processor -- start 2024-09-21T06:35:35.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.471+0000 7f9d73e00640 1 -- start start 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 0x7f9d74140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d74152530 0x7f9d741f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d740cf5b0 con 0x7f9d74073dd0 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d740cf430 con 0x7f9d74152530 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.472+0000 7f9d73e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f9d740cf730 con 0x7f9d740b39d0 2024-09-21T06:35:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37316/0 (socket says 172.21.15.37:37316) 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 -- 172.21.15.37:0/774341214 learned_addr learned my addr 172.21.15.37:0/774341214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d73400640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d74152530 0x7f9d741f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d6be00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 0x7f9d74140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 -- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 msgr2=0x7f9d74140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 0x7f9d74140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 -- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d74152530 msgr2=0x7f9d741f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d74152530 0x7f9d741f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d72a00640 1 -- 172.21.15.37:0/774341214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d741f3840 con 0x7f9d74073dd0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.473+0000 7f9d6be00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 0x7f9d74140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d72a00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c460 tx=0x7f9d6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64017070 con 0x7f9d74073dd0 2024-09-21T06:35:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d640053e0 con 0x7f9d74073dd0 2024-09-21T06:35:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64003850 con 0x7f9d74073dd0 2024-09-21T06:35:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d741f7100 con 0x7f9d74073dd0 2024-09-21T06:35:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.474+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d741f76c0 con 0x7f9d74073dd0 2024-09-21T06:35:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.475+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d740cfb50 con 0x7f9d74073dd0 2024-09-21T06:35:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.476+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d6401a070 con 0x7f9d74073dd0 2024-09-21T06:35:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.477+0000 7f9d6aa00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 0x7f9d5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.477+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f9d6409a2f0 con 0x7f9d74073dd0 2024-09-21T06:35:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.478+0000 7f9d6be00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 0x7f9d5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.479+0000 7f9d6be00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 0x7f9d5007b0a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c033000 tx=0x7f9d5c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.482+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d741f3840 con 0x7f9d74073dd0 2024-09-21T06:35:35.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.762+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9d7415c8e0 con 0x7f9d74073dd0 2024-09-21T06:35:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.763+0000 7f9d6aa00640 1 -- 172.21.15.37:0/774341214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1857 (secure 0 0 0) 0x7f9d64062510 con 0x7f9d74073dd0 2024-09-21T06:35:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:35.765 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":483,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":267272192,"bytes_avail":772826841088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9895,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":33,"modified":"2024-09-21T06:35:33.215429+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 msgr2=0x7f9d5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 0x7f9d5007b0a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c033000 tx=0x7f9d5c004730 comp rx=0 tx=0).stop 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 msgr2=0x7f9d741f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c460 tx=0x7f9d6400c920 comp rx=0 tx=0).stop 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 shutdown_connections 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9d50078b90 0x7f9d5007b0a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d74152530 0x7f9d741f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d740b39d0 0x7f9d74140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 --2- 172.21.15.37:0/774341214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d74073dd0 0x7f9d741f68b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:35.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.765+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 >> 172.21.15.37:0/774341214 conn(0x7f9d7406f940 msgr2=0x7f9d7415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:35.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.766+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 shutdown_connections 2024-09-21T06:35:35.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:35.766+0000 7f9d73e00640 1 -- 172.21.15.37:0/774341214 wait complete. 2024-09-21T06:35:35.789 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:36.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 -- 172.21.15.37:0/1567306294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c06e500 msgr2=0x7f7a5c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 --2- 172.21.15.37:0/1567306294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c06e500 0x7f7a5c0cdb90 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005fd0 tx=0x7f7a5402ba30 comp rx=0 tx=0).stop 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 -- 172.21.15.37:0/1567306294 shutdown_connections 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 --2- 172.21.15.37:0/1567306294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c06e500 0x7f7a5c0cdb90 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 -- 172.21.15.37:0/1567306294 >> 172.21.15.37:0/1567306294 conn(0x7f7a5c0cca40 msgr2=0x7f7a5c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 -- 172.21.15.37:0/1567306294 shutdown_connections 2024-09-21T06:35:37.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.026+0000 7f7a63400640 1 -- 172.21.15.37:0/1567306294 wait complete. 2024-09-21T06:35:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.029+0000 7f7a63400640 1 Processor -- start 2024-09-21T06:35:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.029+0000 7f7a63400640 1 -- start start 2024-09-21T06:35:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a5c070030 0x7f7a5c09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a5c074d90 0x7f7a5c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c071090 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0ce0d0 con 0x7f7a5c074d90 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.030+0000 7f7a63400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c09c240 con 0x7f7a5c070030 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37330/0 (socket says 172.21.15.37:37330) 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 -- 172.21.15.37:0/4175814017 learned_addr learned my addr 172.21.15.37:0/4175814017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a60a00640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a5c070030 0x7f7a5c09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a5be00640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a5c074d90 0x7f7a5c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 -- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a5c070030 msgr2=0x7f7a5c09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a5c070030 0x7f7a5c09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 -- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a5c074d90 msgr2=0x7f7a5c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a5c074d90 0x7f7a5c0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.031+0000 7f7a61400640 1 -- 172.21.15.37:0/4175814017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c09b1a0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a61400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7a54006010 tx=0x7f7a54003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54034070 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a540044d0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54039450 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c09b480 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.032+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c09ba40 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.033+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c0ce5f0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.034+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a54037070 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.035+0000 7f7a58c00640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 0x7f7a3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.036+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7a540bc7c0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.036+0000 7f7a60a00640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 0x7f7a3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.037+0000 7f7a60a00640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 0x7f7a3c07b0d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7a500015e0 tx=0x7f7a50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.040+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a540849d0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.275+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5c09af30 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.276+0000 7f7a58c00640 1 -- 172.21.15.37:0/4175814017 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5278 (secure 0 0 0) 0x7f7a540842d0 con 0x7f7a5c1f6c10 2024-09-21T06:35:37.278 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:37.279 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:30.743657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 116 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 msgr2=0x7f7a3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 0x7f7a3c07b0d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7a500015e0 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 msgr2=0x7f7a5c1f7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7a54006010 tx=0x7f7a54003ef0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 shutdown_connections 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7a3c078bc0 0x7f7a3c07b0d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a5c1f6c10 0x7f7a5c1f7020 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a5c074d90 0x7f7a5c0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 --2- 172.21.15.37:0/4175814017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a5c070030 0x7f7a5c09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 >> 172.21.15.37:0/4175814017 conn(0x7f7a5c0cca40 msgr2=0x7f7a5c159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.278+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 shutdown_connections 2024-09-21T06:35:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.279+0000 7f7a63400640 1 -- 172.21.15.37:0/4175814017 wait complete. 2024-09-21T06:35:37.306 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:37.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2542429626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a015dfe0 msgr2=0x7fb5a0160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2542429626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a015dfe0 0x7fb5a0160a40 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb590009830 tx=0x7fb59002a480 comp rx=0 tx=0).stop 2024-09-21T06:35:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2542429626 shutdown_connections 2024-09-21T06:35:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2542429626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a00ccfe0 0x7fb5a006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2542429626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a015dfe0 0x7fb5a0160a40 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2542429626 >> 172.21.15.37:0/2542429626 conn(0x7fb5a006f940 msgr2=0x7fb5a00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2542429626 shutdown_connections 2024-09-21T06:35:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.531+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2542429626 wait complete. 2024-09-21T06:35:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.533+0000 7fb5a4e00640 1 Processor -- start 2024-09-21T06:35:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.533+0000 7fb5a4e00640 1 -- start start 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 0x7fb5a0142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a015dfe0 0x7fb5a013c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a00b4b80 con 0x7fb5a013da50 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a00b4a00 con 0x7fb5a015dfe0 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb5a4e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a00b4d00 con 0x7fb5a006e160 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59f400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 0x7fb5a0142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59f400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 0x7fb5a0142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:47446/0 (socket says 172.21.15.37:47446) 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59f400640 1 -- 172.21.15.37:0/2482221827 learned_addr learned my addr 172.21.15.37:0/2482221827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37342/0 (socket says 172.21.15.37:37342) 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.535+0000 7fb59ea00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a015dfe0 0x7fb5a013c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 -- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 msgr2=0x7fb5a0142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 0x7fb5a0142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 -- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a015dfe0 msgr2=0x7fb5a013c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a015dfe0 0x7fb5a013c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 -- 172.21.15.37:0/2482221827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a0074480 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb59fe00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb59800e990 tx=0x7fb59800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598004d20 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb59800ca60 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598005020 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.536+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a01f5210 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.537+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a01f57d0 con 0x7fb5a013da50 2024-09-21T06:35:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.537+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a00b51b0 con 0x7fb5a013da50 2024-09-21T06:35:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.539+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5980051c0 con 0x7fb5a013da50 2024-09-21T06:35:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.540+0000 7fb597400640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 0x7fb58007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.540+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fb59809b6d0 con 0x7fb5a013da50 2024-09-21T06:35:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.540+0000 7fb59f400640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 0x7fb58007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.541+0000 7fb59f400640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 0x7fb58007b0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb590005eb0 tx=0x7fb590004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.544+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb598016030 con 0x7fb5a013da50 2024-09-21T06:35:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.817+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb5a00cc0e0 con 0x7fb5a013da50 2024-09-21T06:35:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.818+0000 7fb597400640 1 -- 172.21.15.37:0/2482221827 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1857 (secure 0 0 0) 0x7fb598063320 con 0x7fb5a013da50 2024-09-21T06:35:37.820 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:37.820 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":267272192,"bytes_avail":772826841088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":116,"btime":"2024-09-21T06:35:30:828865+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":33,"modified":"2024-09-21T06:35:33.215429+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 msgr2=0x7fb58007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 0x7fb58007b0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb590005eb0 tx=0x7fb590004730 comp rx=0 tx=0).stop 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 msgr2=0x7fb5a013cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb59800e990 tx=0x7fb59800ed90 comp rx=0 tx=0).stop 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 shutdown_connections 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb580078b90 0x7fb58007b0a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a013da50 0x7fb5a013cd50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a015dfe0 0x7fb5a013c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 --2- 172.21.15.37:0/2482221827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5a006e160 0x7fb5a0142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.820+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 >> 172.21.15.37:0/2482221827 conn(0x7fb5a006f940 msgr2=0x7fb5a015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.821+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 shutdown_connections 2024-09-21T06:35:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:37.821+0000 7fb5a4e00640 1 -- 172.21.15.37:0/2482221827 wait complete. 2024-09-21T06:35:37.845 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:38.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:38.868 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:35:38.866+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 -- 172.21.15.37:0/2340356687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 msgr2=0x7ff354073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 --2- 172.21.15.37:0/2340356687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354073dd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff35415d980 tx=0x7ff34802a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 -- 172.21.15.37:0/2340356687 shutdown_connections 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 --2- 172.21.15.37:0/2340356687 >> v2:172.21.15.150:3300/0 conn(0x7ff35406e500 0x7ff3540cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 --2- 172.21.15.37:0/2340356687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354073dd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 -- 172.21.15.37:0/2340356687 >> 172.21.15.37:0/2340356687 conn(0x7ff3540cca40 msgr2=0x7ff3540cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.082+0000 7ff35be00640 1 -- 172.21.15.37:0/2340356687 shutdown_connections 2024-09-21T06:35:39.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.083+0000 7ff35be00640 1 -- 172.21.15.37:0/2340356687 wait complete. 2024-09-21T06:35:39.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.085+0000 7ff35be00640 1 Processor -- start 2024-09-21T06:35:39.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.085+0000 7ff35be00640 1 -- start start 2024-09-21T06:35:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.086+0000 7ff35be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.086+0000 7ff35be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 0x7ff35409d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff35be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3540b2e10 0x7ff35414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff35be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff35406f360 con 0x7ff354074cb0 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff35be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff35406f1e0 con 0x7ff3540b2e10 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff35be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff35406f4e0 con 0x7ff3540cbc10 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37360/0 (socket says 172.21.15.37:37360) 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 -- 172.21.15.37:0/521190580 learned_addr learned my addr 172.21.15.37:0/521190580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359e00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3540b2e10 0x7ff35414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff358a00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 0x7ff35409d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 -- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 msgr2=0x7ff35409d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 0x7ff35409d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 -- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3540b2e10 msgr2=0x7ff35414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3540b2e10 0x7ff35414ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff359400640 1 -- 172.21.15.37:0/521190580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540b5450 con 0x7ff354074cb0 2024-09-21T06:35:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.087+0000 7ff358a00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 0x7ff35409d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff359400640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff3480099c0 tx=0x7ff3480039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34803e030 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3480047a0 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348035040 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3540b5730 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.088+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3540b5cf0 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.089+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35406f990 con 0x7ff354074cb0 2024-09-21T06:35:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.091+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff348041030 con 0x7ff354074cb0 2024-09-21T06:35:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.092+0000 7ff351600640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 0x7ff33407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.092+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7ff3480bb640 con 0x7ff354074cb0 2024-09-21T06:35:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.092+0000 7ff358a00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 0x7ff33407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.093+0000 7ff358a00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 0x7ff33407b0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff3540a0560 tx=0x7ff340009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.097+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff348083200 con 0x7ff354074cb0 2024-09-21T06:35:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.329+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff35409ddd0 con 0x7ff354074cb0 2024-09-21T06:35:39.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.330+0000 7ff351600640 1 -- 172.21.15.37:0/521190580 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5279 (secure 0 0 0) 0x7ff348087c30 con 0x7ff354074cb0 2024-09-21T06:35:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:39.332 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":117,"btime":"2024-09-21T06:35:38:867554+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:38.343067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[7079,8172,8175]},"id":7}]} 2024-09-21T06:35:39.333 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 117 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 msgr2=0x7ff33407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 0x7ff33407b0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff3540a0560 tx=0x7ff340009040 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 msgr2=0x7ff354150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff3480099c0 tx=0x7ff3480039e0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 shutdown_connections 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff334078bc0 0x7ff33407b0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3540b2e10 0x7ff35414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3540cbc10 0x7ff35409d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 --2- 172.21.15.37:0/521190580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354074cb0 0x7ff354150b50 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 >> 172.21.15.37:0/521190580 conn(0x7ff3540cca40 msgr2=0x7ff35415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 shutdown_connections 2024-09-21T06:35:39.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.332+0000 7ff35be00640 1 -- 172.21.15.37:0/521190580 wait complete. 2024-09-21T06:35:39.361 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:39.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:39.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 -- 172.21.15.37:0/4019832685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f9006e330 msgr2=0x7f7f900cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 --2- 172.21.15.37:0/4019832685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f9006e330 0x7f7f900cdb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f88005fd0 tx=0x7f7f8802a8a0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 -- 172.21.15.37:0/4019832685 shutdown_connections 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 --2- 172.21.15.37:0/4019832685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90070030 0x7f7f9006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 --2- 172.21.15.37:0/4019832685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f9006e330 0x7f7f900cdb90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 -- 172.21.15.37:0/4019832685 >> 172.21.15.37:0/4019832685 conn(0x7f7f900cca40 msgr2=0x7f7f900cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 -- 172.21.15.37:0/4019832685 shutdown_connections 2024-09-21T06:35:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.627+0000 7f7f97200640 1 -- 172.21.15.37:0/4019832685 wait complete. 2024-09-21T06:35:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.630+0000 7f7f97200640 1 Processor -- start 2024-09-21T06:35:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.630+0000 7f7f97200640 1 -- start start 2024-09-21T06:35:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f90070030 0x7f7f900b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90074d90 0x7f7f9006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90071000 con 0x7f7f9006e330 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90070e80 con 0x7f7f90070030 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.631+0000 7f7f97200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f7f900ce0d0 con 0x7f7f90074d90 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37364/0 (socket says 172.21.15.37:37364) 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 -- 172.21.15.37:0/3406237722 learned_addr learned my addr 172.21.15.37:0/3406237722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f95200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90074d90 0x7f7f9006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f87e00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f90070030 0x7f7f900b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 -- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90074d90 msgr2=0x7f7f9006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90074d90 0x7f7f9006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 -- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f90070030 msgr2=0x7f7f900b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f90070030 0x7f7f900b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 -- 172.21.15.37:0/3406237722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f900b4000 con 0x7f7f9006e330 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.632+0000 7f7f8fe00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000c430 tx=0x7f7f8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.633+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80017070 con 0x7f7f9006e330 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.633+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f800053e0 con 0x7f7f9006e330 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.633+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f900b42e0 con 0x7f7f9006e330 2024-09-21T06:35:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.633+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003850 con 0x7f7f9006e330 2024-09-21T06:35:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.633+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f900b48a0 con 0x7f7f9006e330 2024-09-21T06:35:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.634+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f900ce5a0 con 0x7f7f9006e330 2024-09-21T06:35:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.635+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f8001a070 con 0x7f7f9006e330 2024-09-21T06:35:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.636+0000 7f7f8cc00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 0x7f7f6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.636+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7f8009a1a0 con 0x7f7f9006e330 2024-09-21T06:35:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.637+0000 7f7f87e00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 0x7f7f6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.637+0000 7f7f87e00640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 0x7f7f6c07b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7f901f7150 tx=0x7f7f78009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.640+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f80061df0 con 0x7f7f9006e330 2024-09-21T06:35:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.919+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7f9015c990 con 0x7f7f9006e330 2024-09-21T06:35:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.919+0000 7f7f8cc00640 1 -- 172.21.15.37:0/3406237722 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7f7f80066820 con 0x7f7f9006e330 2024-09-21T06:35:39.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:39.922 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":267280384,"bytes_avail":772826832896,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7251,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-09-21T06:35:39.217213+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 msgr2=0x7f7f6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 0x7f7f6c07b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7f901f7150 tx=0x7f7f78009260 comp rx=0 tx=0).stop 2024-09-21T06:35:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 msgr2=0x7f7f900c0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000c430 tx=0x7f7f8000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 shutdown_connections 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7f6c078bc0 0x7f7f6c07b0d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f90074d90 0x7f7f9006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f90070030 0x7f7f900b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 --2- 172.21.15.37:0/3406237722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f9006e330 0x7f7f900c0c70 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 >> 172.21.15.37:0/3406237722 conn(0x7f7f900cca40 msgr2=0x7f7f900cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 shutdown_connections 2024-09-21T06:35:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:39.922+0000 7f7f97200640 1 -- 172.21.15.37:0/3406237722 wait complete. 2024-09-21T06:35:39.946 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:39.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:39.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub abort 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 -- 172.21.15.37:0/3232382484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4750005280 msgr2=0x7f475000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 --2- 172.21.15.37:0/3232382484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4750005280 0x7f475000b9c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f476c146180 tx=0x7f4750005830 comp rx=0 tx=0).stop 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 -- 172.21.15.37:0/3232382484 shutdown_connections 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 --2- 172.21.15.37:0/3232382484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4750005280 0x7f475000b9c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 -- 172.21.15.37:0/3232382484 >> 172.21.15.37:0/3232382484 conn(0x7f476c06f920 msgr2=0x7f476c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 -- 172.21.15.37:0/3232382484 shutdown_connections 2024-09-21T06:35:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.176+0000 7f476be00640 1 -- 172.21.15.37:0/3232382484 wait complete. 2024-09-21T06:35:40.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.179+0000 7f476be00640 1 Processor -- start 2024-09-21T06:35:40.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.179+0000 7f476be00640 1 -- start start 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 0x7f476c1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f476c1f95d0 0x7f476c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f476c0ce150 con 0x7f476c13bbd0 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f476c0cdfd0 con 0x7f4750005280 2024-09-21T06:35:40.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.180+0000 7f476be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f476c0cfce0 con 0x7f476c1f95d0 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 0x7f476c1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f4763e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f476c1f95d0 0x7f476c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37384/0 (socket says 172.21.15.37:37384) 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 -- 172.21.15.37:0/1507377453 learned_addr learned my addr 172.21.15.37:0/1507377453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 0x7f476c1f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:54126/0 (socket says 172.21.15.37:54126) 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 -- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f476c1f95d0 msgr2=0x7f476c15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f476c1f95d0 0x7f476c15b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 -- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 msgr2=0x7f476c1f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 0x7f476c1f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 -- 172.21.15.37:0/1507377453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1349e0 con 0x7f476c13bbd0 2024-09-21T06:35:40.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.181+0000 7f476b400640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f476400c700 tx=0x7f4764003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.182+0000 7f4762a00640 1 -- 172.21.15.37:0/1507377453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476402b8f0 con 0x7f476c13bbd0 2024-09-21T06:35:40.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.182+0000 7f4762a00640 1 -- 172.21.15.37:0/1507377453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f476403a070 con 0x7f476c13bbd0 2024-09-21T06:35:40.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.182+0000 7f4762a00640 1 -- 172.21.15.37:0/1507377453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476402be20 con 0x7f476c13bbd0 2024-09-21T06:35:40.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.182+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c13c1f0 con 0x7f476c13bbd0 2024-09-21T06:35:40.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.182+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c13c7b0 con 0x7f476c13bbd0 2024-09-21T06:35:40.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.184+0000 7f4762a00640 1 -- 172.21.15.37:0/1507377453 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47640060d0 con 0x7f476c13bbd0 2024-09-21T06:35:40.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.185+0000 7f4762a00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 0x7f474807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.185+0000 7f4762a00640 1 -- 172.21.15.37:0/1507377453 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f47640baf80 con 0x7f476c13bbd0 2024-09-21T06:35:40.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.186+0000 7f4778a68740 1 Processor -- start 2024-09-21T06:35:40.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.186+0000 7f476aa00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 0x7f474807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.186+0000 7f4778a68740 1 -- start start 2024-09-21T06:35:40.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.186+0000 7f476aa00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 0x7f474807b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f475c009bd0 tx=0x7f475c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 0x558ee6f66860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 0x558ee6f49530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558ee6e32c10 con 0x558ee6b566b0 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ee6e919f0 con 0x558ee6b77390 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558ee6e80e20 con 0x558ee6b4e4d0 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f476b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f476b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37388/0 (socket says 172.21.15.37:37388) 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.187+0000 7f476b400640 1 -- 172.21.15.37:0/2876506309 learned_addr learned my addr 172.21.15.37:0/2876506309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476aa00640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 0x558ee6f66860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f4763e00640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 0x558ee6f49530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 -- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 msgr2=0x558ee6f49530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 0x558ee6f49530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 -- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 msgr2=0x558ee6f66860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 0x558ee6f66860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 -- 172.21.15.37:0/2876506309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee6c597d0 con 0x558ee6b566b0 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476aa00640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 0x558ee6f66860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f4763e00640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 0x558ee6f49530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:40.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f476b400640 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4764082a70 tx=0x7f47640829b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d6dc46dad7cbba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f475b400640 1 -- 172.21.15.37:0/2876506309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764045040 con 0x558ee6b566b0 2024-09-21T06:35:40.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f475b400640 1 -- 172.21.15.37:0/2876506309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f476402f030 con 0x558ee6b566b0 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.188+0000 7f475b400640 1 -- 172.21.15.37:0/2876506309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764081f40 con 0x558ee6b566b0 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 -- 172.21.15.37:0/2876506309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 msgr2=0x558ee6f49a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4764082a70 tx=0x7f47640829b0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 -- 172.21.15.37:0/2876506309 shutdown_connections 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f49a70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b4e4d0 0x558ee6f49530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 --2- 172.21.15.37:0/2876506309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b77390 0x558ee6f66860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 -- 172.21.15.37:0/2876506309 >> 172.21.15.37:0/2876506309 conn(0x558ee6b6b4e0 msgr2=0x558ee6f1f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 -- 172.21.15.37:0/2876506309 shutdown_connections 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.189+0000 7f4778a68740 1 -- 172.21.15.37:0/2876506309 wait complete. 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.190+0000 7f4778a68740 1 -- start start 2024-09-21T06:35:40.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.190+0000 7f4778a68740 1 Processor -- start 2024-09-21T06:35:40.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f37a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558ee6e26790 con 0x558ee6b566b0 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ee6e84c30 con 0x558ee6b4e4d0 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558ee6bf1740 con 0x558ee6b77390 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4778a68740 10 client.0 _ll_register_callbacks cb 0x558ee6e36260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37394/0 (socket says 172.21.15.37:37394) 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f4763e00640 1 -- 172.21.15.37:0/255021314 learned_addr learned my addr 172.21.15.37:0/255021314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f476b400640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f37a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.192+0000 7f476aa00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 -- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 msgr2=0x558ee6f37a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f37a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 -- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 msgr2=0x558ee6f733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 -- 172.21.15.37:0/255021314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee6bc06a0 con 0x558ee6b566b0 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f476aa00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4763e00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4754003ad0 tx=0x7f4754003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00d83fa4e7417b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f474fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754011630 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f475400cb00 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476402f030 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.193+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f4754023050 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 10 client.8694 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x558ee6cb9f60 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f476402f030 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f4754026020 con 0x558ee6b566b0 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 10 client.8694 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 10 client.8694 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 20 client.8694 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6b35f60 0x558ee6f1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 4 client.8694 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:40.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee6e93940 con 0x558ee6b35f60 2024-09-21T06:35:40.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.194+0000 7f476aa00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6b35f60 0x558ee6f1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.195+0000 7f476aa00640 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6b35f60 0x558ee6f1f290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32a114bd105a8f0 server_cookie=674d82820631f1c9 in_seq=0 out_seq=0 2024-09-21T06:35:40.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.195+0000 7f474fe00640 10 client.8694 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.195+0000 7f474fe00640 1 -- 172.21.15.37:0/255021314 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ee6e93940 con 0x558ee6b35f60 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.195+0000 7f474fe00640 10 client.8694 handle_command_reply: tid=0 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 client.8694 shutdown 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 msgr2=0x558ee6f62e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4754003ad0 tx=0x7f4754003f90 comp rx=0 tx=0).stop 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 shutdown_connections 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6b35f60 0x558ee6f1f290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f62e30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f37a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 --2- 172.21.15.37:0/255021314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 >> 172.21.15.37:0/255021314 conn(0x558ee6b6b4e0 msgr2=0x558ee6f49530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f474fe00640 0 client.8694 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 shutdown_connections 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 1 -- 172.21.15.37:0/255021314 wait complete. 2024-09-21T06:35:40.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.196+0000 7f4778a68740 20 client.8694 trim_cache size 0 max 16384 2024-09-21T06:35:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.214+0000 7f4778a68740 1 Processor -- start 2024-09-21T06:35:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.215+0000 7f4778a68740 1 -- start start 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f5d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f476c202820 con 0x558ee6b566b0 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f476c2019c0 con 0x558ee6b4e4d0 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f476c2020f0 con 0x558ee6b77390 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37404/0 (socket says 172.21.15.37:37404) 2024-09-21T06:35:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.216+0000 7f4763e00640 1 -- 172.21.15.37:0/759230237 learned_addr learned my addr 172.21.15.37:0/759230237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f476b400640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f5d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f476aa00640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 -- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 msgr2=0x558ee6f5d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f5d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 -- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 msgr2=0x558ee6f19cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f19cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 -- 172.21.15.37:0/759230237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee6c57ad0 con 0x558ee6b566b0 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f476aa00640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f4763e00640 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f4754011640 tx=0x7f4754011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21793a3f35d9b8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f475aa00640 1 -- 172.21.15.37:0/759230237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754015070 con 0x558ee6b566b0 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f475aa00640 1 -- 172.21.15.37:0/759230237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f475402c070 con 0x558ee6b566b0 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.217+0000 7f475aa00640 1 -- 172.21.15.37:0/759230237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754011ad0 con 0x558ee6b566b0 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 -- 172.21.15.37:0/759230237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 msgr2=0x558ee6f1a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f4754011640 tx=0x7f4754011670 comp rx=0 tx=0).stop 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 -- 172.21.15.37:0/759230237 shutdown_connections 2024-09-21T06:35:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f1a200 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f5d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 --2- 172.21.15.37:0/759230237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6f19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 -- 172.21.15.37:0/759230237 >> 172.21.15.37:0/759230237 conn(0x558ee6b6b4e0 msgr2=0x558ee6f64040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 -- 172.21.15.37:0/759230237 shutdown_connections 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.218+0000 7f4778a68740 1 -- 172.21.15.37:0/759230237 wait complete. 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.219+0000 7f4778a68740 1 -- start start 2024-09-21T06:35:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.219+0000 7f4778a68740 1 Processor -- start 2024-09-21T06:35:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6fe5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f72ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f476c073750 con 0x558ee6b566b0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ee6e89eb0 con 0x558ee6b4e4d0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x558ee6f76f80 con 0x558ee6b77390 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4778a68740 10 client.0 _ll_register_callbacks cb 0x7f4754003730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37408/0 (socket says 172.21.15.37:37408) 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 -- 172.21.15.37:0/2823864777 learned_addr learned my addr 172.21.15.37:0/2823864777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f476b400640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f72ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f476aa00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6fe5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 -- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 msgr2=0x558ee6f72ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f72ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 -- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 msgr2=0x558ee6fe5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6fe5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f4763e00640 1 -- 172.21.15.37:0/2823864777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee6f76d70 con 0x558ee6b566b0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.221+0000 7f476b400640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f72ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f4763e00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f4754001160 tx=0x7f47540036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8724519f26464b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f474fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754015050 con 0x558ee6b566b0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f4754030450 con 0x558ee6b566b0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475402c070 con 0x558ee6b566b0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f475403d050 con 0x558ee6b566b0 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f4778a68740 10 client.8700 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.222+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x558ee6f627f0 con 0x558ee6b566b0 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f475402c070 con 0x558ee6b566b0 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f4754040020 con 0x558ee6b566b0 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 10 client.8700 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 10 client.8700 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 20 client.8700 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6f7c410 0x558ee6f85e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 4 client.8700 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x558ee6e8f3f0 con 0x558ee6f7c410 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.223+0000 7f476aa00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6f7c410 0x558ee6f85e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f476aa00640 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6f7c410 0x558ee6f85e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1489d84179825e27 server_cookie=420ecbe32a9d62f9 in_seq=0 out_seq=0 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f474fe00640 10 client.8700 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f474fe00640 1 -- 172.21.15.37:0/2823864777 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ee6e8f3f0 con 0x558ee6f7c410 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f474fe00640 10 client.8700 handle_command_reply: tid=0 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 client.8700 shutdown 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 msgr2=0x558ee6f72980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f4754001160 tx=0x7f47540036a0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 shutdown_connections 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x558ee6f7c410 0x558ee6f85e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ee6b566b0 0x558ee6f72980 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558ee6b77390 0x558ee6f72ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 --2- 172.21.15.37:0/2823864777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ee6b4e4d0 0x558ee6fe5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f474fe00640 0 client.8700 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 >> 172.21.15.37:0/2823864777 conn(0x558ee6b6b4e0 msgr2=0x558ee6f61dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 shutdown_connections 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.224+0000 7f4778a68740 1 -- 172.21.15.37:0/2823864777 wait complete. 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f4778a68740 20 client.8700 trim_cache size 0 max 16384 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:40.232 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 msgr2=0x7f474807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 0x7f474807b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f475c009bd0 tx=0x7f475c005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 msgr2=0x7f476c15b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f476400c700 tx=0x7f4764003c00 comp rx=0 tx=0).stop 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 shutdown_connections 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4748078b90 0x7f474807b0a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f476c13bbd0 0x7f476c15b980 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f476c1f95d0 0x7f476c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 --2- 172.21.15.37:0/1507377453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750005280 0x7f476c1f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 >> 172.21.15.37:0/1507377453 conn(0x7f476c06f920 msgr2=0x7f476c0b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 shutdown_connections 2024-09-21T06:35:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.225+0000 7f476be00640 1 -- 172.21.15.37:0/1507377453 wait complete. 2024-09-21T06:35:40.252 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:40.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 -- 172.21.15.37:0/3824225468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 msgr2=0x7f61140cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 --2- 172.21.15.37:0/3824225468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140cf310 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f610c005fd0 tx=0x7f610c02aaa0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 -- 172.21.15.37:0/3824225468 shutdown_connections 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 --2- 172.21.15.37:0/3824225468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140cf310 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 --2- 172.21.15.37:0/3824225468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f611406ddf0 0x7f611406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 -- 172.21.15.37:0/3824225468 >> 172.21.15.37:0/3824225468 conn(0x7f61140b1b90 msgr2=0x7f61140b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 -- 172.21.15.37:0/3824225468 shutdown_connections 2024-09-21T06:35:40.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.480+0000 7f611a600640 1 -- 172.21.15.37:0/3824225468 wait complete. 2024-09-21T06:35:40.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.483+0000 7f611a600640 1 Processor -- start 2024-09-21T06:35:40.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.483+0000 7f611a600640 1 -- start start 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f611406ddf0 0x7f61140b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6114156a50 0x7f6114158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61140cf850 con 0x7f61140cef00 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61141610a0 con 0x7f611406ddf0 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f611a600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f61140cf9d0 con 0x7f6114156a50 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f610be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f610be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37436/0 (socket says 172.21.15.37:37436) 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.484+0000 7f610be00640 1 -- 172.21.15.37:0/3643315528 learned_addr learned my addr 172.21.15.37:0/3643315528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f6113400640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f611406ddf0 0x7f61140b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f6113e00640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6114156a50 0x7f6114158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 -- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6114156a50 msgr2=0x7f6114158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6114156a50 0x7f6114158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 -- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f611406ddf0 msgr2=0x7f61140b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f611406ddf0 0x7f61140b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 -- 172.21.15.37:0/3643315528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61140b5da0 con 0x7f61140cef00 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610be00640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009fd0 tx=0x7f60fc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.485+0000 7f610b400640 1 -- 172.21.15.37:0/3643315528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00ccc0 con 0x7f61140cef00 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.486+0000 7f610b400640 1 -- 172.21.15.37:0/3643315528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60fc00ee70 con 0x7f61140cef00 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.486+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6114159490 con 0x7f61140cef00 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.486+0000 7f610b400640 1 -- 172.21.15.37:0/3643315528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00ce60 con 0x7f61140cef00 2024-09-21T06:35:40.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.486+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6114159a10 con 0x7f61140cef00 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.489+0000 7f610b400640 1 -- 172.21.15.37:0/3643315528 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60fc004280 con 0x7f61140cef00 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.490+0000 7f610b400640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 0x7f60f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.491+0000 7f61216c8740 1 Processor -- start 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.491+0000 7f610b400640 1 -- 172.21.15.37:0/3643315528 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f60fc09b7d0 con 0x7f61140cef00 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.491+0000 7f61216c8740 1 -- start start 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.491+0000 7f6113400640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 0x7f60f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113400640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 0x7f60f007b0a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6104001720 tx=0x7f6104009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 0x55703fbb9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7aa4d0 0x55703fbc3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703fada1c0 con 0x55703f7b26b0 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55703faed870 con 0x55703f7d3390 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55703fadcca0 con 0x55703f7aa4d0 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 0x55703fbb9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f610be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7aa4d0 0x55703fbc3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 0x55703fbb9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:54188/0 (socket says 172.21.15.37:54188) 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37448/0 (socket says 172.21.15.37:37448) 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.492+0000 7f6113400640 1 -- 172.21.15.37:0/4217574907 learned_addr learned my addr 172.21.15.37:0/4217574907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 -- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7aa4d0 msgr2=0x55703fbc3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7aa4d0 0x55703fbc3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 -- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 msgr2=0x55703fbb9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 0x55703fbb9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 -- 172.21.15.37:0/4217574907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703f8b57d0 con 0x55703f7b26b0 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6113e00640 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f610c032000 tx=0x7f610c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b76bde5b6eaac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.493+0000 7f6103e00640 1 -- 172.21.15.37:0/4217574907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c02ecf0 con 0x55703f7b26b0 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f6103e00640 1 -- 172.21.15.37:0/4217574907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f610c045070 con 0x55703f7b26b0 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f6103e00640 1 -- 172.21.15.37:0/4217574907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c003bd0 con 0x55703f7b26b0 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 -- 172.21.15.37:0/4217574907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 msgr2=0x55703fbc3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f610c032000 tx=0x7f610c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 -- 172.21.15.37:0/4217574907 shutdown_connections 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7b26b0 0x55703fbc3890 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7aa4d0 0x55703fbc3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 --2- 172.21.15.37:0/4217574907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7d3390 0x55703fbb9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 -- 172.21.15.37:0/4217574907 >> 172.21.15.37:0/4217574907 conn(0x55703f7c74e0 msgr2=0x55703fb7e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.494+0000 7f61216c8740 1 -- 172.21.15.37:0/4217574907 shutdown_connections 2024-09-21T06:35:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.495+0000 7f61216c8740 1 -- 172.21.15.37:0/4217574907 wait complete. 2024-09-21T06:35:40.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.495+0000 7f61216c8740 1 -- start start 2024-09-21T06:35:40.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.496+0000 7f61216c8740 1 Processor -- start 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbc2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703fa83410 con 0x55703f7d3390 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55703fae0ab0 con 0x55703f7aa4d0 2024-09-21T06:35:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55703f84d740 con 0x55703f7b26b0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f61216c8740 10 client.0 _ll_register_callbacks cb 0x55703fa60360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.497+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbd7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.37:54200/0 (socket says 172.21.15.37:54200) 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113400640 1 -- 172.21.15.37:0/111466989 learned_addr learned my addr 172.21.15.37:0/111466989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37464/0 (socket says 172.21.15.37:37464) 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f610be00640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbc2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 -- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 msgr2=0x55703fbc2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbc2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 -- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 msgr2=0x55703fbd7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbd7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 -- 172.21.15.37:0/111466989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703f83e970 con 0x55703f7d3390 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6113e00640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f610c009bf0 tx=0x7f610c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f98f9d7ca47276 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.498+0000 7f6100c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004240 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f610c053cb0 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c045070 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f610c009800 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 10 client.8715 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55703f81c6a0 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f610c045070 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f610c004b40 con 0x55703f7d3390 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 10 client.8715 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 10 client.8715 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 20 client.8715 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703f791f60 0x55703fba97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 4 client.8715 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.499+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55703faef7c0 con 0x55703f791f60 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.500+0000 7f6113400640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703f791f60 0x55703fba97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.500+0000 7f6113400640 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703f791f60 0x55703fba97f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de90b14c6090158 server_cookie=cd77b2982500cb2f in_seq=0 out_seq=0 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.500+0000 7f6100c00640 10 client.8715 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f6100c00640 1 -- 172.21.15.37:0/111466989 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55703faef7c0 con 0x55703f791f60 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f6100c00640 10 client.8715 handle_command_reply: tid=0 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 client.8715 shutdown 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 msgr2=0x55703fbaab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f610c009bf0 tx=0x7f610c003ba0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 shutdown_connections 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703f791f60 0x55703fba97f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbaab40 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbc2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 --2- 172.21.15.37:0/111466989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbd7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f6100c00640 0 client.8715 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.501+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 >> 172.21.15.37:0/111466989 conn(0x55703f7c74e0 msgr2=0x55703fb987b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.502+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 shutdown_connections 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.502+0000 7f61216c8740 1 -- 172.21.15.37:0/111466989 wait complete. 2024-09-21T06:35:40.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.502+0000 7f61216c8740 20 client.8715 trim_cache size 0 max 16384 2024-09-21T06:35:40.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.520+0000 7f61216c8740 1 Processor -- start 2024-09-21T06:35:40.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.520+0000 7f61216c8740 1 -- start start 2024-09-21T06:35:40.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61141fe610 con 0x55703f7d3390 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61141fd9a0 con 0x55703f7aa4d0 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f61141fdee0 con 0x55703f7b26b0 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37468/0 (socket says 172.21.15.37:37468) 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 -- 172.21.15.37:0/2423105602 learned_addr learned my addr 172.21.15.37:0/2423105602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113400640 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f610be00640 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 -- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 msgr2=0x55703fbead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 -- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 msgr2=0x55703fbea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbea800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.522+0000 7f6113e00640 1 -- 172.21.15.37:0/2423105602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703fbac240 con 0x55703f7d3390 2024-09-21T06:35:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.523+0000 7f6113e00640 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f610c004a80 tx=0x7f610c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d406b3c1b7efa60e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.523+0000 7f6103400640 1 -- 172.21.15.37:0/2423105602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004200 con 0x55703f7d3390 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.523+0000 7f6103400640 1 -- 172.21.15.37:0/2423105602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f610c061070 con 0x55703f7d3390 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.523+0000 7f6103400640 1 -- 172.21.15.37:0/2423105602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004710 con 0x55703f7d3390 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 -- 172.21.15.37:0/2423105602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 msgr2=0x55703fb7e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f610c004a80 tx=0x7f610c0021f0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 -- 172.21.15.37:0/2423105602 shutdown_connections 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fb7e4f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 --2- 172.21.15.37:0/2423105602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbea800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 -- 172.21.15.37:0/2423105602 >> 172.21.15.37:0/2423105602 conn(0x55703f7c74e0 msgr2=0x55703fbbebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 -- 172.21.15.37:0/2423105602 shutdown_connections 2024-09-21T06:35:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.524+0000 7f61216c8740 1 -- 172.21.15.37:0/2423105602 wait complete. 2024-09-21T06:35:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.525+0000 7f61216c8740 1 -- start start 2024-09-21T06:35:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.525+0000 7f61216c8740 1 Processor -- start 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbe53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbe0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61140cb900 con 0x55703f7d3390 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55703fbdfda0 con 0x55703f7aa4d0 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.526+0000 7f61216c8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55703fc49ac0 con 0x55703f7b26b0 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f61216c8740 10 client.0 _ll_register_callbacks cb 0x7f6104017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37470/0 (socket says 172.21.15.37:37470) 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 -- 172.21.15.37:0/3292791308 learned_addr learned my addr 172.21.15.37:0/3292791308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f610be00640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbe0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113400640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbe53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 -- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 msgr2=0x55703fbe0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbe0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 -- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 msgr2=0x55703fbe53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbe53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 -- 172.21.15.37:0/3292791308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703fc49da0 con 0x55703f7d3390 2024-09-21T06:35:40.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6113e00640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f610c0099a0 tx=0x7f610c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eb43c4b9b0c4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.527+0000 7f6100c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c009360 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7f610c066d00 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c061070 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f610c05c480 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 10 client.8721 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55703fbc1f30 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f610c061070 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f610c05c620 con 0x55703f7d3390 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 10 client.8721 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 10 client.8721 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 20 client.8721 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703fbeade0 0x55703fba97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 4 client.8721 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.528+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55703faeb270 con 0x55703fbeade0 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f6113400640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703fbeade0 0x55703fba97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f6113400640 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703fbeade0 0x55703fba97f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61b1d8b1bf4d2443 server_cookie=3ff37fa096b30c70 in_seq=0 out_seq=0 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f6100c00640 10 client.8721 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f6100c00640 1 -- 172.21.15.37:0/3292791308 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55703faeb270 con 0x55703fbeade0 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f6100c00640 10 client.8721 handle_command_reply: tid=0 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.529+0000 7f61216c8740 1 client.8721 shutdown 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 msgr2=0x55703fbea800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f610c0099a0 tx=0x7f610c004220 comp rx=0 tx=0).stop 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 shutdown_connections 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55703fbeade0 0x55703fba97f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703f7d3390 0x55703fbea800 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55703f7b26b0 0x55703fbe0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f6100c00640 0 client.8721 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 --2- 172.21.15.37:0/3292791308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55703f7aa4d0 0x55703fbe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 >> 172.21.15.37:0/3292791308 conn(0x55703f7c74e0 msgr2=0x55703fbaa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 shutdown_connections 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 1 -- 172.21.15.37:0/3292791308 wait complete. 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.530+0000 7f61216c8740 20 client.8721 trim_cache size 0 max 16384 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:40.535 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:40.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:40.536 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:40.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:40.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:40.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 msgr2=0x7f60f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 0x7f60f007b0a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6104001720 tx=0x7f6104009040 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 msgr2=0x7f61140bccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009fd0 tx=0x7f60fc00ec20 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 shutdown_connections 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f60f0078b90 0x7f60f007b0a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61140cef00 0x7f61140bccf0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6114156a50 0x7f6114158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 --2- 172.21.15.37:0/3643315528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f611406ddf0 0x7f61140b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 >> 172.21.15.37:0/3643315528 conn(0x7f61140b1b90 msgr2=0x7f61140cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 shutdown_connections 2024-09-21T06:35:40.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.531+0000 7f611a600640 1 -- 172.21.15.37:0/3643315528 wait complete. 2024-09-21T06:35:40.556 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:40.556 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:40.556 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-21T06:35:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.783+0000 7fc698800640 1 -- 172.21.15.37:0/3183659685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc69006e330 msgr2=0x7fc6900cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.783+0000 7fc698800640 1 --2- 172.21.15.37:0/3183659685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc69006e330 0x7fc6900cdc70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fc6900cca40 tx=0x7fc68c02aa30 comp rx=0 tx=0).stop 2024-09-21T06:35:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 -- 172.21.15.37:0/3183659685 shutdown_connections 2024-09-21T06:35:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 --2- 172.21.15.37:0/3183659685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc69006e330 0x7fc6900cdc70 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 --2- 172.21.15.37:0/3183659685 >> v2:172.21.15.150:3300/0 conn(0x7fc690074b60 0x7fc69006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 --2- 172.21.15.37:0/3183659685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 0x7fc690074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 -- 172.21.15.37:0/3183659685 >> 172.21.15.37:0/3183659685 conn(0x7fc6900b1b90 msgr2=0x7fc6900b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 -- 172.21.15.37:0/3183659685 shutdown_connections 2024-09-21T06:35:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.784+0000 7fc698800640 1 -- 172.21.15.37:0/3183659685 wait complete. 2024-09-21T06:35:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.786+0000 7fc698800640 1 Processor -- start 2024-09-21T06:35:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.786+0000 7fc698800640 1 -- start start 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc69006e330 0x7fc6901f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 0x7fc6900c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6900ce330 con 0x7fc690074b60 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6900ce1b0 con 0x7fc69006e330 2024-09-21T06:35:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc698800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc6900ce4b0 con 0x7fc690156a50 2024-09-21T06:35:40.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37498/0 (socket says 172.21.15.37:37498) 2024-09-21T06:35:40.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 -- 172.21.15.37:0/3719665993 learned_addr learned my addr 172.21.15.37:0/3719665993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695e00640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc69006e330 0x7fc6901f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc696800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 0x7fc6900c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 -- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 msgr2=0x7fc6900c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 0x7fc6900c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 -- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc69006e330 msgr2=0x7fc6901f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc69006e330 0x7fc6901f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.788+0000 7fc695400640 1 -- 172.21.15.37:0/3719665993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6900b59c0 con 0x7fc690074b60 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc695400640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc68000e710 tx=0x7fc68000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc68a000640 1 -- 172.21.15.37:0/3719665993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68000cab0 con 0x7fc690074b60 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc68a000640 1 -- 172.21.15.37:0/3719665993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc68000cd40 con 0x7fc690074b60 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6901f9250 con 0x7fc690074b60 2024-09-21T06:35:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc68a000640 1 -- 172.21.15.37:0/3719665993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680010690 con 0x7fc690074b60 2024-09-21T06:35:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.789+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6901f6690 con 0x7fc690074b60 2024-09-21T06:35:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.792+0000 7fc68a000640 1 -- 172.21.15.37:0/3719665993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc680010830 con 0x7fc690074b60 2024-09-21T06:35:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.792+0000 7fc68a000640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 0x7fc67407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.793+0000 7fc68a000640 1 -- 172.21.15.37:0/3719665993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7fc68009b8f0 con 0x7fc690074b60 2024-09-21T06:35:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.793+0000 7fc69f9c1740 1 Processor -- start 2024-09-21T06:35:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.793+0000 7fc695e00640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 0x7fc67407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.793+0000 7fc69f9c1740 1 -- start start 2024-09-21T06:35:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.794+0000 7fc695e00640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 0x7fc67407b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc684009b30 tx=0x7fc684005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.794+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1787550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.794+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e176a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0e16a01c0 con 0x55b0e13786b0 2024-09-21T06:35:40.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0e16b3870 con 0x55b0e13704d0 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0e16a2ca0 con 0x55b0e1399390 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37510/0 (socket says 172.21.15.37:37510) 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 -- 172.21.15.37:0/103080237 learned_addr learned my addr 172.21.15.37:0/103080237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc695e00640 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1787550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc695400640 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e176a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 -- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 msgr2=0x55b0e1787550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1787550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 -- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 msgr2=0x55b0e176a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e176a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.795+0000 7fc696800640 1 -- 172.21.15.37:0/103080237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e147b7d0 con 0x55b0e13786b0 2024-09-21T06:35:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.796+0000 7fc696800640 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009930 tx=0x7fc68c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25e32b3e32fda3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.796+0000 7fc67e600640 1 -- 172.21.15.37:0/103080237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c033030 con 0x55b0e13786b0 2024-09-21T06:35:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.796+0000 7fc67e600640 1 -- 172.21.15.37:0/103080237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc68c02fb80 con 0x55b0e13786b0 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.796+0000 7fc67e600640 1 -- 172.21.15.37:0/103080237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003970 con 0x55b0e13786b0 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 -- 172.21.15.37:0/103080237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 msgr2=0x55b0e176a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009930 tx=0x7fc68c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 -- 172.21.15.37:0/103080237 shutdown_connections 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e176a760 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e176a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/103080237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1787550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 -- 172.21.15.37:0/103080237 >> 172.21.15.37:0/103080237 conn(0x55b0e138d4e0 msgr2=0x55b0e1740440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 -- 172.21.15.37:0/103080237 shutdown_connections 2024-09-21T06:35:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.797+0000 7fc69f9c1740 1 -- 172.21.15.37:0/103080237 wait complete. 2024-09-21T06:35:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.798+0000 7fc69f9c1740 1 -- start start 2024-09-21T06:35:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.798+0000 7fc69f9c1740 1 Processor -- start 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e1757d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1780c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1649410 con 0x55b0e13786b0 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0e16a6ab0 con 0x55b0e13704d0 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0e1413740 con 0x55b0e1399390 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc69f9c1740 10 client.0 _ll_register_callbacks cb 0x55b0e1626360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37520/0 (socket says 172.21.15.37:37520) 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 -- 172.21.15.37:0/384645978 learned_addr learned my addr 172.21.15.37:0/384645978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695e00640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e1757d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc696800640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1780c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 -- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 msgr2=0x55b0e1780c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1780c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 -- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 msgr2=0x55b0e1757d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e1757d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.800+0000 7fc695400640 1 -- 172.21.15.37:0/384645978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e1404970 con 0x55b0e13786b0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc695400640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc6800606b0 tx=0x7fc680062a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428186be8165b962 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc672a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680062bb0 con 0x55b0e13786b0 2024-09-21T06:35:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7fc68007adf0 con 0x55b0e13786b0 2024-09-21T06:35:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c02fb80 con 0x55b0e13786b0 2024-09-21T06:35:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fc680062f50 con 0x55b0e13786b0 2024-09-21T06:35:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc69f9c1740 10 client.8733 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55b0e13e26a0 con 0x55b0e13786b0 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.801+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc68c02fb80 con 0x55b0e13786b0 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fc68005ef90 con 0x55b0e13786b0 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 10 client.8733 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 10 client.8733 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 20 client.8733 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e1357f60 0x55b0e1794bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 4 client.8733 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e16b57c0 con 0x55b0e1357f60 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.802+0000 7fc695e00640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e1357f60 0x55b0e1794bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.803+0000 7fc695e00640 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e1357f60 0x55b0e1794bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e5385eca1ae7546 server_cookie=bff200b652546a55 in_seq=0 out_seq=0 2024-09-21T06:35:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.803+0000 7fc672a00640 10 client.8733 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.804+0000 7fc672a00640 1 -- 172.21.15.37:0/384645978 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0e16b57c0 con 0x55b0e1357f60 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.804+0000 7fc672a00640 10 client.8733 handle_command_reply: tid=0 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.804+0000 7fc69f9c1740 1 client.8733 shutdown 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.804+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 msgr2=0x55b0e17988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.804+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc6800606b0 tx=0x7fc680062a70 comp rx=0 tx=0).stop 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 shutdown_connections 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e1357f60 0x55b0e1794bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13786b0 0x55b0e17988e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1780c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/384645978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e1757d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc672a00640 0 client.8733 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 >> 172.21.15.37:0/384645978 conn(0x55b0e138d4e0 msgr2=0x55b0e17885b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 shutdown_connections 2024-09-21T06:35:40.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 1 -- 172.21.15.37:0/384645978 wait complete. 2024-09-21T06:35:40.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.805+0000 7fc69f9c1740 20 client.8733 trim_cache size 0 max 16384 2024-09-21T06:35:40.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.824+0000 7fc69f9c1740 1 Processor -- start 2024-09-21T06:35:40.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.824+0000 7fc69f9c1740 1 -- start start 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e173aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e13786b0 0x55b0e1741340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc690202940 con 0x55b0e1399390 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc690201ae0 con 0x55b0e13704d0 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc690202210 con 0x55b0e13786b0 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc696800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc696800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37526/0 (socket says 172.21.15.37:37526) 2024-09-21T06:35:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.825+0000 7fc696800640 1 -- 172.21.15.37:0/820571418 learned_addr learned my addr 172.21.15.37:0/820571418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc695400640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e13786b0 0x55b0e1741340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc695e00640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e173aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 -- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e13786b0 msgr2=0x55b0e1741340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e13786b0 0x55b0e1741340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 -- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 msgr2=0x55b0e173aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e173aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 -- 172.21.15.37:0/820571418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e177e030 con 0x55b0e1399390 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc695e00640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e173aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:40.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc696800640 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009960 tx=0x7fc68c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9aac0d84c0df7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc67dc00640 1 -- 172.21.15.37:0/820571418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c033070 con 0x55b0e1399390 2024-09-21T06:35:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc67dc00640 1 -- 172.21.15.37:0/820571418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc68c009360 con 0x55b0e1399390 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.826+0000 7fc67dc00640 1 -- 172.21.15.37:0/820571418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c02fbc0 con 0x55b0e1399390 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 -- 172.21.15.37:0/820571418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 msgr2=0x55b0e1741880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009960 tx=0x7fc68c004a40 comp rx=0 tx=0).stop 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 -- 172.21.15.37:0/820571418 shutdown_connections 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e1399390 0x55b0e1741880 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e13786b0 0x55b0e1741340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/820571418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13704d0 0x55b0e173aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 -- 172.21.15.37:0/820571418 >> 172.21.15.37:0/820571418 conn(0x55b0e138d4e0 msgr2=0x55b0e1740810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 -- 172.21.15.37:0/820571418 shutdown_connections 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.827+0000 7fc69f9c1740 1 -- 172.21.15.37:0/820571418 wait complete. 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.828+0000 7fc69f9c1740 1 -- start start 2024-09-21T06:35:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.828+0000 7fc69f9c1740 1 Processor -- start 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 0x55b0e17885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1788af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc69009fbb0 con 0x55b0e13704d0 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0e17946b0 con 0x55b0e13786b0 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55b0e177c680 con 0x55b0e1399390 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc69f9c1740 10 client.0 _ll_register_callbacks cb 0x7fc68005f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc695e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc695e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37536/0 (socket says 172.21.15.37:37536) 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.830+0000 7fc695e00640 1 -- 172.21.15.37:0/2465941205 learned_addr learned my addr 172.21.15.37:0/2465941205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc696800640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1788af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695400640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 0x55b0e17885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 -- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 msgr2=0x55b0e1788af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1788af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 -- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 msgr2=0x55b0e17885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 0x55b0e17885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 -- 172.21.15.37:0/2465941205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e177c1b0 con 0x55b0e13704d0 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc696800640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1788af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695400640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 0x55b0e17885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc695e00640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc684011810 tx=0x7fc684004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca44d2232f700715 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc672a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:40.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684003e40 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c009360 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.831+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6217+0+0 (secure 0 0 0) 0x7fc684009040 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fc684005050 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 10 client.8739 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55b0e173eca0 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc68c009360 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fc6840051f0 con 0x55b0e13704d0 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 10 client.8739 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 10 client.8739 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 20 client.8739 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e17988a0 0x55b0e17567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 4 client.8739 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.832+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0e16b1270 con 0x55b0e17988a0 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc695400640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e17988a0 0x55b0e17567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc695400640 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e17988a0 0x55b0e17567f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c70418969fc77ba2 server_cookie=92fa5bce995b977f in_seq=0 out_seq=0 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc672a00640 10 client.8739 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc672a00640 1 -- 172.21.15.37:0/2465941205 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b0e16b1270 con 0x55b0e17988a0 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc672a00640 10 client.8739 handle_command_reply: tid=0 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.833+0000 7fc69f9c1740 1 client.8739 shutdown 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 msgr2=0x55b0e1804580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc684011810 tx=0x7fc684004730 comp rx=0 tx=0).stop 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 shutdown_connections 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x55b0e17988a0 0x55b0e17567f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b0e1399390 0x55b0e1788af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0e13786b0 0x55b0e17885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 --2- 172.21.15.37:0/2465941205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0e13704d0 0x55b0e1804580 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc672a00640 0 client.8739 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 >> 172.21.15.37:0/2465941205 conn(0x55b0e138d4e0 msgr2=0x55b0e1735f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 shutdown_connections 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 1 -- 172.21.15.37:0/2465941205 wait complete. 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc69f9c1740 20 client.8739 trim_cache size 0 max 16384 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:40.839 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:40.840 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.834+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 msgr2=0x7fc67407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 0x7fc67407b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc684009b30 tx=0x7fc684005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 msgr2=0x7fc6900b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc68000e710 tx=0x7fc68000ebd0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 shutdown_connections 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc674078b90 0x7fc67407b0a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc690074b60 0x7fc6900b50e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc690156a50 0x7fc6900c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 --2- 172.21.15.37:0/3719665993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc69006e330 0x7fc6901f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 >> 172.21.15.37:0/3719665993 conn(0x7fc6900b1b90 msgr2=0x7fc69006d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 shutdown_connections 2024-09-21T06:35:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:40.835+0000 7fc698800640 1 -- 172.21.15.37:0/3719665993 wait complete. 2024-09-21T06:35:40.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:40.862 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:40.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T06:35:41.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 -- 172.21.15.37:0/1410368530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41456e0 msgr2=0x7f15f40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 --2- 172.21.15.37:0/1410368530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41456e0 0x7f15f40b4f30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005fd0 tx=0x7f15e402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 -- 172.21.15.37:0/1410368530 shutdown_connections 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 --2- 172.21.15.37:0/1410368530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41456e0 0x7f15f40b4f30 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 -- 172.21.15.37:0/1410368530 >> 172.21.15.37:0/1410368530 conn(0x7f15f406f920 msgr2=0x7f15f40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 -- 172.21.15.37:0/1410368530 shutdown_connections 2024-09-21T06:35:41.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.092+0000 7f15f9a00640 1 -- 172.21.15.37:0/1410368530 wait complete. 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.094+0000 7f15f9a00640 1 Processor -- start 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.094+0000 7f15f9a00640 1 -- start start 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 0x7f15f4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 0x7f15f413f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40ce230 con 0x7f15f41429b0 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40ce0b0 con 0x7f15f40cc8d0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f15f40ce3b0 con 0x7f15f41456e0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37544/0 (socket says 172.21.15.37:37544) 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15ebe00640 1 -- 172.21.15.37:0/2031328146 learned_addr learned my addr 172.21.15.37:0/2031328146 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 0x7f15f4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15f9000640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 0x7f15f413f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.096+0000 7f15ebe00640 1 -- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 msgr2=0x7f15f413f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 0x7f15f413f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15ebe00640 1 -- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 msgr2=0x7f15f4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 0x7f15f4140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15ebe00640 1 -- 172.21.15.37:0/2031328146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f41408c0 con 0x7f15f41429b0 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f9000640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 0x7f15f413f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 0x7f15f4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b3f0 tx=0x7f15dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f0c00640 1 -- 172.21.15.37:0/2031328146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc015020 con 0x7f15f41429b0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f0c00640 1 -- 172.21.15.37:0/2031328146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc011920 con 0x7f15f41429b0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f0c00640 1 -- 172.21.15.37:0/2031328146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc011d70 con 0x7f15f41429b0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f41f9570 con 0x7f15f41429b0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.097+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f41f57f0 con 0x7f15f41429b0 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.100+0000 7f15f0c00640 1 -- 172.21.15.37:0/2031328146 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15dc018020 con 0x7f15f41429b0 2024-09-21T06:35:41.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.101+0000 7f15f0c00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 0x7f15d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.101+0000 7f15f0c00640 1 -- 172.21.15.37:0/2031328146 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f15dc09acd0 con 0x7f15f41429b0 2024-09-21T06:35:41.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.101+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 0x7f15d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.102+0000 7f1602012740 1 Processor -- start 2024-09-21T06:35:41.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.102+0000 7f1602012740 1 -- start start 2024-09-21T06:35:41.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.102+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 0x7f15d007b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15e4033000 tx=0x7f15e4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667e5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f6666fe1c0 con 0x55f6663ce4d0 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f666711870 con 0x55f6663f7390 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f666700ca0 con 0x55f6663d66b0 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.103+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37558/0 (socket says 172.21.15.37:37558) 2024-09-21T06:35:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 -- 172.21.15.37:0/3016656368 learned_addr learned my addr 172.21.15.37:0/3016656368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15f9000640 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15f3e00640 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 -- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 msgr2=0x55f6667c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 -- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 msgr2=0x55f6667e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 -- 172.21.15.37:0/3016656368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6664d97d0 con 0x55f6663ce4d0 2024-09-21T06:35:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f15dc062700 tx=0x7f15dc062010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe13ca39924ecc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15e8c00640 1 -- 172.21.15.37:0/3016656368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc067250 con 0x55f6663ce4d0 2024-09-21T06:35:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15e8c00640 1 -- 172.21.15.37:0/3016656368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc06c190 con 0x55f6663ce4d0 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.104+0000 7f15e8c00640 1 -- 172.21.15.37:0/3016656368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc061c10 con 0x55f6663ce4d0 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 -- 172.21.15.37:0/3016656368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 msgr2=0x55f6667c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f15dc062700 tx=0x7f15dc062010 comp rx=0 tx=0).stop 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 -- 172.21.15.37:0/3016656368 shutdown_connections 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667c8400 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 --2- 172.21.15.37:0/3016656368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 -- 172.21.15.37:0/3016656368 >> 172.21.15.37:0/3016656368 conn(0x55f6663eb4e0 msgr2=0x55f66679e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 -- 172.21.15.37:0/3016656368 shutdown_connections 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.105+0000 7f1602012740 1 -- 172.21.15.37:0/3016656368 wait complete. 2024-09-21T06:35:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.106+0000 7f1602012740 1 -- start start 2024-09-21T06:35:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.107+0000 7f1602012740 1 Processor -- start 2024-09-21T06:35:41.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f6666a7410 con 0x55f6663ce4d0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f666704ab0 con 0x55f6663f7390 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f666471740 con 0x55f6663d66b0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.108+0000 7f1602012740 10 client.0 _ll_register_callbacks cb 0x55f666684360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37568/0 (socket says 172.21.15.37:37568) 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 -- 172.21.15.37:0/3446929164 learned_addr learned my addr 172.21.15.37:0/3446929164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f9000640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 -- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 msgr2=0x55f6667da0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667da0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 -- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 msgr2=0x55f6667dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 -- 172.21.15.37:0/3446929164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f666462970 con 0x55f6663ce4d0 2024-09-21T06:35:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f9000640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15f3e00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f15e4000c00 tx=0x7f15e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d904af86213354f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.109+0000 7f15e1600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e404d020 con 0x55f6663ce4d0 2024-09-21T06:35:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f15e40049a0 con 0x55f6663ce4d0 2024-09-21T06:35:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc06c190 con 0x55f6663ce4d0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f15e4057050 con 0x55f6663ce4d0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f1602012740 10 client.8754 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55f6664406a0 con 0x55f6663ce4d0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc06c190 con 0x55f6663ce4d0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.110+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f15e405a020 con 0x55f6663ce4d0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 10 client.8754 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 10 client.8754 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 20 client.8754 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6663b5f60 0x55f6667f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 4 client.8754 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6667137c0 con 0x55f6663b5f60 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6663b5f60 0x55f6667f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f15ebe00640 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6663b5f60 0x55f6667f5be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821c931f3b8db4f7 server_cookie=4225946259cc54c6 in_seq=0 out_seq=0 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.111+0000 7f15e1600640 10 client.8754 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f15e1600640 1 -- 172.21.15.37:0/3446929164 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6667137c0 con 0x55f6663b5f60 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f15e1600640 10 client.8754 handle_command_reply: tid=0 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 client.8754 shutdown 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 msgr2=0x55f6667f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f15e4000c00 tx=0x7f15e4009260 comp rx=0 tx=0).stop 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 shutdown_connections 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6663b5f60 0x55f6667f5be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663f7390 0x55f6667dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f6667da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 --2- 172.21.15.37:0/3446929164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663ce4d0 0x55f6667f1df0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f15e1600640 0 client.8754 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.112+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 >> 172.21.15.37:0/3446929164 conn(0x55f6663eb4e0 msgr2=0x55f66679f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.113+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 shutdown_connections 2024-09-21T06:35:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.113+0000 7f1602012740 1 -- 172.21.15.37:0/3446929164 wait complete. 2024-09-21T06:35:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.113+0000 7f1602012740 20 client.8754 trim_cache size 0 max 16384 2024-09-21T06:35:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.131+0000 7f1602012740 1 Processor -- start 2024-09-21T06:35:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.132+0000 7f1602012740 1 -- start start 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f666806980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 0x55f6667e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15f42027e0 con 0x55f6663d66b0 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4201980 con 0x55f6663ce4d0 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f15f42020b0 con 0x55f6663f7390 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37576/0 (socket says 172.21.15.37:37576) 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15ebe00640 1 -- 172.21.15.37:0/2654328411 learned_addr learned my addr 172.21.15.37:0/2654328411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15f9000640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 0x55f6667e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f666806980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15ebe00640 1 -- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 msgr2=0x55f6667e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.133+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 0x55f6667e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15ebe00640 1 -- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 msgr2=0x55f666806980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f666806980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15ebe00640 1 -- 172.21.15.37:0/2654328411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f666706f50 con 0x55f6663d66b0 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15f9000640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 0x55f6667e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15f3e00640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f666806980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15ebe00640 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f15dc05e820 tx=0x7f15dc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ea8bdbe95f5b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15e3e00640 1 -- 172.21.15.37:0/2654328411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0619d0 con 0x55f6663d66b0 2024-09-21T06:35:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15e3e00640 1 -- 172.21.15.37:0/2654328411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc061c60 con 0x55f6663d66b0 2024-09-21T06:35:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.134+0000 7f15e3e00640 1 -- 172.21.15.37:0/2654328411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc05e300 con 0x55f6663d66b0 2024-09-21T06:35:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 -- 172.21.15.37:0/2654328411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 msgr2=0x55f6667e39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f15dc05e820 tx=0x7f15dc004680 comp rx=0 tx=0).stop 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 -- 172.21.15.37:0/2654328411 shutdown_connections 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663d66b0 0x55f6667e39d0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663f7390 0x55f6667e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 --2- 172.21.15.37:0/2654328411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f666806980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 -- 172.21.15.37:0/2654328411 >> 172.21.15.37:0/2654328411 conn(0x55f6663eb4e0 msgr2=0x55f66679f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 -- 172.21.15.37:0/2654328411 shutdown_connections 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.135+0000 7f1602012740 1 -- 172.21.15.37:0/2654328411 wait complete. 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.136+0000 7f1602012740 1 -- start start 2024-09-21T06:35:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.136+0000 7f1602012740 1 Processor -- start 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.137+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f6668592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f666862820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f6667f1b20 con 0x55f6663f7390 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41e70c0 con 0x55f6663ce4d0 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55f6667dabe0 con 0x55f6663d66b0 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f1602012740 10 client.0 _ll_register_callbacks cb 0x7f15e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37582/0 (socket says 172.21.15.37:37582) 2024-09-21T06:35:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 -- 172.21.15.37:0/1441935724 learned_addr learned my addr 172.21.15.37:0/1441935724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15ebe00640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f666862820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f3e00640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f6668592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 -- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 msgr2=0x55f666862820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f666862820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 -- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 msgr2=0x55f6668592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f6668592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f9000640 1 -- 172.21.15.37:0/1441935724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6667e4170 con 0x55f6663f7390 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.138+0000 7f15f3e00640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f6668592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15f9000640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f15ec007fd0 tx=0x7f15ec011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9180a043e1338073 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15e1600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec003b80 con 0x55f6663f7390 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc061c60 con 0x55f6663f7390 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f15ec010040 con 0x55f6663f7390 2024-09-21T06:35:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f15ec021050 con 0x55f6663f7390 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f1602012740 10 client.8760 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55f6667fc810 con 0x55f6663f7390 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.139+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc061c60 con 0x55f6663f7390 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f15ec024020 con 0x55f6663f7390 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 10 client.8760 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 10 client.8760 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 20 client.8760 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6668632a0 0x55f6667d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 4 client.8760 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6667dd590 con 0x55f6668632a0 2024-09-21T06:35:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f15f3e00640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6668632a0 0x55f6667d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f15e1600640 10 client.8760 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.140+0000 7f15f3e00640 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6668632a0 0x55f6667d9640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10047d826d86614e server_cookie=97051317403ef1bf in_seq=0 out_seq=0 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f15e1600640 1 -- 172.21.15.37:0/1441935724 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x55f6667dd590 con 0x55f6668632a0 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f15e1600640 10 client.8760 handle_command_reply: tid=0 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 client.8760 shutdown 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 msgr2=0x55f666862d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f15ec007fd0 tx=0x7f15ec011910 comp rx=0 tx=0).stop 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 shutdown_connections 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x55f6668632a0 0x55f6667d9640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6663f7390 0x55f666862d60 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f6663d66b0 0x55f666862820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 --2- 172.21.15.37:0/1441935724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6663ce4d0 0x55f6668592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 >> 172.21.15.37:0/1441935724 conn(0x55f6663eb4e0 msgr2=0x55f6667db8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f15e1600640 0 client.8760 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 shutdown_connections 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 1 -- 172.21.15.37:0/1441935724 wait complete. 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.141+0000 7f1602012740 20 client.8760 trim_cache size 0 max 16384 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:41.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:35:31.440721+0000", 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.148 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 msgr2=0x7f15d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 0x7f15d007b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15e4033000 tx=0x7f15e4004730 comp rx=0 tx=0).stop 2024-09-21T06:35:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 msgr2=0x7f15f413edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b3f0 tx=0x7f15dc00b8b0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 shutdown_connections 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f15d0078b90 0x7f15d007b0a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15f41429b0 0x7f15f413edc0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15f41456e0 0x7f15f413f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 --2- 172.21.15.37:0/2031328146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15f40cc8d0 0x7f15f4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 >> 172.21.15.37:0/2031328146 conn(0x7f15f406f920 msgr2=0x7f15f40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 shutdown_connections 2024-09-21T06:35:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.142+0000 7f15f9a00640 1 -- 172.21.15.37:0/2031328146 wait complete. 2024-09-21T06:35:41.166 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:41.166 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:41.166 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-21T06:35:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.394+0000 7f803d200640 1 -- 172.21.15.37:0/3607587831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f803806ddf0 msgr2=0x7f803806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.394+0000 7f803d200640 1 --2- 172.21.15.37:0/3607587831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f803806ddf0 0x7f803806e200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8020009860 tx=0x7f802002a530 comp rx=0 tx=0).stop 2024-09-21T06:35:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.395+0000 7f803d200640 1 -- 172.21.15.37:0/3607587831 shutdown_connections 2024-09-21T06:35:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.395+0000 7f803d200640 1 --2- 172.21.15.37:0/3607587831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f803806ddf0 0x7f803806e200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.395+0000 7f803d200640 1 -- 172.21.15.37:0/3607587831 >> 172.21.15.37:0/3607587831 conn(0x7f80380b1b90 msgr2=0x7f80380b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.395+0000 7f803d200640 1 -- 172.21.15.37:0/3607587831 shutdown_connections 2024-09-21T06:35:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.395+0000 7f803d200640 1 -- 172.21.15.37:0/3607587831 wait complete. 2024-09-21T06:35:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.397+0000 7f803d200640 1 Processor -- start 2024-09-21T06:35:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.397+0000 7f803d200640 1 -- start start 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803806ddf0 0x7f80380cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80380cef00 0x7f80380b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8038160fd0 con 0x7f8038156a50 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8038160e50 con 0x7f803806ddf0 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.398+0000 7f803d200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f80380cfa20 con 0x7f80380cef00 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37592/0 (socket says 172.21.15.37:37592) 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 -- 172.21.15.37:0/1790105557 learned_addr learned my addr 172.21.15.37:0/1790105557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f802fe00640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80380cef00 0x7f80380b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8035e00640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803806ddf0 0x7f80380cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 -- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80380cef00 msgr2=0x7f80380b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80380cef00 0x7f80380b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 -- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803806ddf0 msgr2=0x7f80380cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803806ddf0 0x7f80380cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.399+0000 7f8036800640 1 -- 172.21.15.37:0/1790105557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803809f8a0 con 0x7f8038156a50 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f8036800640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f803000c730 tx=0x7f803000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f802e000640 1 -- 172.21.15.37:0/1790105557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030003b70 con 0x7f8038156a50 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f802e000640 1 -- 172.21.15.37:0/1790105557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030003d10 con 0x7f8038156a50 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f802e000640 1 -- 172.21.15.37:0/1790105557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803000aab0 con 0x7f8038156a50 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803809fb80 con 0x7f8038156a50 2024-09-21T06:35:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.400+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80380a0140 con 0x7f8038156a50 2024-09-21T06:35:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.403+0000 7f802e000640 1 -- 172.21.15.37:0/1790105557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f803000ac50 con 0x7f8038156a50 2024-09-21T06:35:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.403+0000 7f802e000640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 0x7f801407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.404+0000 7f802e000640 1 -- 172.21.15.37:0/1790105557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f803009aae0 con 0x7f8038156a50 2024-09-21T06:35:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.404+0000 7f80442af740 1 Processor -- start 2024-09-21T06:35:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.404+0000 7f8035e00640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 0x7f801407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.404+0000 7f80442af740 1 -- start start 2024-09-21T06:35:41.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.405+0000 7f8035e00640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 0x7f801407b0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f8028004600 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a99e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4a981220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb4a8b71c0 con 0x55cb4a58f6b0 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb4a8ca870 con 0x55cb4a5b0390 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cb4a8b9ca0 con 0x55cb4a5874d0 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37602/0 (socket says 172.21.15.37:37602) 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 -- 172.21.15.37:0/989624603 learned_addr learned my addr 172.21.15.37:0/989624603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8035e00640 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a99e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f802fe00640 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4a981220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 -- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 msgr2=0x55cb4a981220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4a981220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 -- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 msgr2=0x55cb4a99e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a99e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.406+0000 7f8036800640 1 -- 172.21.15.37:0/989624603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4a6927d0 con 0x55cb4a58f6b0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.407+0000 7f8036800640 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8030062450 tx=0x7f8030061ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23015993cf3b0657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.407+0000 7f8026a00640 1 -- 172.21.15.37:0/989624603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030088020 con 0x55cb4a58f6b0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.407+0000 7f8026a00640 1 -- 172.21.15.37:0/989624603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030062750 con 0x55cb4a58f6b0 2024-09-21T06:35:41.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.407+0000 7f8026a00640 1 -- 172.21.15.37:0/989624603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030067180 con 0x55cb4a58f6b0 2024-09-21T06:35:41.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 -- 172.21.15.37:0/989624603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 msgr2=0x55cb4a981760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8030062450 tx=0x7f8030061ec0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 -- 172.21.15.37:0/989624603 shutdown_connections 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a981760 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4a981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 --2- 172.21.15.37:0/989624603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a99e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 -- 172.21.15.37:0/989624603 >> 172.21.15.37:0/989624603 conn(0x55cb4a5a44e0 msgr2=0x55cb4a957440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 -- 172.21.15.37:0/989624603 shutdown_connections 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.408+0000 7f80442af740 1 -- 172.21.15.37:0/989624603 wait complete. 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.409+0000 7f80442af740 1 -- start start 2024-09-21T06:35:41.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.409+0000 7f80442af740 1 Processor -- start 2024-09-21T06:35:41.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a9af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a9946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb4a860410 con 0x55cb4a5874d0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb4a8bdab0 con 0x55cb4a5b0390 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cb4a62a740 con 0x55cb4a58f6b0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f80442af740 10 client.0 _ll_register_callbacks cb 0x55cb4a83d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37610/0 (socket says 172.21.15.37:37610) 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 -- 172.21.15.37:0/3056883566 learned_addr learned my addr 172.21.15.37:0/3056883566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f802fe00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a9af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8036800640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a9946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 -- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 msgr2=0x55cb4a9af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a9af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 -- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 msgr2=0x55cb4a9946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a9946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8035e00640 1 -- 172.21.15.37:0/3056883566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4a61b970 con 0x55cb4a5874d0 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.411+0000 7f8036800640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a9946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f8035e00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8028014a30 tx=0x7f8028004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec86001af0e90df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f801b400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802801b070 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8030062750 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f8028015b90 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f802802e050 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f80442af740 10 client.8772 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.412+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55cb4a5f96a0 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030062750 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f8028031020 con 0x55cb4a5874d0 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 10 client.8772 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 10 client.8772 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 20 client.8772 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a56ef60 0x55cb4a992d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 4 client.8772 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:41.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4a8cc7c0 con 0x55cb4a56ef60 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.413+0000 7f802fe00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a56ef60 0x55cb4a992d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.414+0000 7f802fe00640 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a56ef60 0x55cb4a992d50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f6496ca8c3dd989 server_cookie=536bbd89bf23c40f in_seq=0 out_seq=0 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.414+0000 7f801b400640 10 client.8772 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.414+0000 7f801b400640 1 -- 172.21.15.37:0/3056883566 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb4a8cc7c0 con 0x55cb4a56ef60 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.414+0000 7f801b400640 10 client.8772 handle_command_reply: tid=0 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.414+0000 7f80442af740 1 client.8772 shutdown 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 msgr2=0x55cb4a96f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8028014a30 tx=0x7f8028004670 comp rx=0 tx=0).stop 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 shutdown_connections 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a56ef60 0x55cb4a992d50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a9946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a9af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 --2- 172.21.15.37:0/3056883566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5874d0 0x55cb4a96f460 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f801b400640 0 client.8772 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:41.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 >> 172.21.15.37:0/3056883566 conn(0x55cb4a5a44e0 msgr2=0x55cb4a9709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 shutdown_connections 2024-09-21T06:35:41.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 1 -- 172.21.15.37:0/3056883566 wait complete. 2024-09-21T06:35:41.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.415+0000 7f80442af740 20 client.8772 trim_cache size 0 max 16384 2024-09-21T06:35:41.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.434+0000 7f80442af740 1 Processor -- start 2024-09-21T06:35:41.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.434+0000 7f80442af740 1 -- start start 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5874d0 0x55cb4a9bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a998980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8038202960 con 0x55cb4a5b0390 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8038201b00 con 0x55cb4a5874d0 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.435+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8038202230 con 0x55cb4a58f6b0 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37616/0 (socket says 172.21.15.37:37616) 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 -- 172.21.15.37:0/3817604948 learned_addr learned my addr 172.21.15.37:0/3817604948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8035e00640 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5874d0 0x55cb4a9bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f802fe00640 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a998980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 -- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 msgr2=0x55cb4a998980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a998980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 -- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5874d0 msgr2=0x55cb4a9bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5874d0 0x55cb4a9bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 -- 172.21.15.37:0/3817604948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4a953d50 con 0x55cb4a5b0390 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.436+0000 7f8036800640 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f803006bfa0 tx=0x7f8030098cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7b9c7a90d1a142 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f8026000640 1 -- 172.21.15.37:0/3817604948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030088040 con 0x55cb4a5b0390 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f8026000640 1 -- 172.21.15.37:0/3817604948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030062750 con 0x55cb4a5b0390 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f8026000640 1 -- 172.21.15.37:0/3817604948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803007cdc0 con 0x55cb4a5b0390 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 -- 172.21.15.37:0/3817604948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 msgr2=0x55cb4a998ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f803006bfa0 tx=0x7f8030098cd0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 -- 172.21.15.37:0/3817604948 shutdown_connections 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a5b0390 0x55cb4a998ec0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a58f6b0 0x55cb4a998980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 --2- 172.21.15.37:0/3817604948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5874d0 0x55cb4a9bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.437+0000 7f80442af740 1 -- 172.21.15.37:0/3817604948 >> 172.21.15.37:0/3817604948 conn(0x55cb4a5a44e0 msgr2=0x55cb4a951ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.438+0000 7f80442af740 1 -- 172.21.15.37:0/3817604948 shutdown_connections 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.438+0000 7f80442af740 1 -- 172.21.15.37:0/3817604948 wait complete. 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.438+0000 7f80442af740 1 -- start start 2024-09-21T06:35:41.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.439+0000 7f80442af740 1 Processor -- start 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4aa123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a96f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80380b3fc0 con 0x55cb4a58f6b0 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb4a96fbf0 con 0x55cb4a5b0390 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55cb4a9aa7f0 con 0x55cb4a5874d0 2024-09-21T06:35:41.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f80442af740 10 client.0 _ll_register_callbacks cb 0x7f8028014680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f802fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.440+0000 7f802fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37624/0 (socket says 172.21.15.37:37624) 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 -- 172.21.15.37:0/257037657 learned_addr learned my addr 172.21.15.37:0/257037657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f8036800640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a96f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f8035e00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4aa123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 -- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 msgr2=0x55cb4aa123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4aa123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 -- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 msgr2=0x55cb4a96f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a96f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 -- 172.21.15.37:0/257037657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4a998fa0 con 0x55cb4a58f6b0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f8035e00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4aa123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f802fe00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f80200097b0 tx=0x7f80200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4808da095a2582a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f801b400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802004a040 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f8020005220 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.441+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8030062750 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f8020055050 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 10 client.8778 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55cb4a999280 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030062750 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f8020058020 con 0x55cb4a58f6b0 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 10 client.8778 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 10 client.8778 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 20 client.8778 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a99dad0 0x55cb4aa1ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 4 client.8778 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb4a8c8270 con 0x55cb4a99dad0 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.442+0000 7f8035e00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a99dad0 0x55cb4aa1ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f8035e00640 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a99dad0 0x55cb4aa1ee70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=934769d79dd04168 server_cookie=d7ca3e7dde73f842 in_seq=0 out_seq=0 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f801b400640 10 client.8778 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f801b400640 1 -- 172.21.15.37:0/257037657 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55cb4a8c8270 con 0x55cb4a99dad0 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f801b400640 10 client.8778 handle_command_reply: tid=0 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 client.8778 shutdown 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 msgr2=0x55cb4a96ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f80200097b0 tx=0x7f80200046c0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 shutdown_connections 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55cb4a99dad0 0x55cb4aa1ee70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb4a58f6b0 0x55cb4a96ef90 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb4a5b0390 0x55cb4a96f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f801b400640 0 client.8778 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 --2- 172.21.15.37:0/257037657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cb4a5874d0 0x55cb4aa123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.443+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 >> 172.21.15.37:0/257037657 conn(0x55cb4a5a44e0 msgr2=0x55cb4a9ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 shutdown_connections 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f80442af740 1 -- 172.21.15.37:0/257037657 wait complete. 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f80442af740 20 client.8778 trim_cache size 0 max 16384 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:41.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:41.450 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:41.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.450 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 msgr2=0x7f801407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 0x7f801407b0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f8028004600 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 msgr2=0x7f80381f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.444+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f803000c730 tx=0x7f803000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 shutdown_connections 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8014078bc0 0x7f801407b0d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8038156a50 0x7f80381f8b20 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80380cef00 0x7f80380b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 --2- 172.21.15.37:0/1790105557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803806ddf0 0x7f80380cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 >> 172.21.15.37:0/1790105557 conn(0x7f80380b1b90 msgr2=0x7f80380afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 shutdown_connections 2024-09-21T06:35:41.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.445+0000 7f803d200640 1 -- 172.21.15.37:0/1790105557 wait complete. 2024-09-21T06:35:41.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:41.472 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:41.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-21T06:35:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 -- 172.21.15.37:0/3996903025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 msgr2=0x7ff2b4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/3996903025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b4074620 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005fd0 tx=0x7ff2a402a540 comp rx=0 tx=0).stop 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 -- 172.21.15.37:0/3996903025 shutdown_connections 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/3996903025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b4074b60 0x7ff2b406ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/3996903025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b4074620 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 -- 172.21.15.37:0/3996903025 >> 172.21.15.37:0/3996903025 conn(0x7ff2b40b1b90 msgr2=0x7ff2b40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 -- 172.21.15.37:0/3996903025 shutdown_connections 2024-09-21T06:35:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.699+0000 7ff2b9e00640 1 -- 172.21.15.37:0/3996903025 wait complete. 2024-09-21T06:35:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.701+0000 7ff2b9e00640 1 Processor -- start 2024-09-21T06:35:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.702+0000 7ff2b9e00640 1 -- start start 2024-09-21T06:35:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b406e330 0x7ff2b40cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 0x7ff2b409fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b40ce500 con 0x7ff2b4156a50 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b40ce380 con 0x7ff2b4074b60 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b9e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b40ce680 con 0x7ff2b406e330 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37644/0 (socket says 172.21.15.37:37644) 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b3400640 1 -- 172.21.15.37:0/1927887964 learned_addr learned my addr 172.21.15.37:0/1927887964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2abe00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 0x7ff2b409fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.703+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b406e330 0x7ff2b40cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 -- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b406e330 msgr2=0x7ff2b40cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b406e330 0x7ff2b40cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 -- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 msgr2=0x7ff2b409fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 0x7ff2b409fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 -- 172.21.15.37:0/1927887964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41f8a90 con 0x7ff2b4156a50 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2abe00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 0x7ff2b409fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b970 tx=0x7ff2ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2ab400640 1 -- 172.21.15.37:0/1927887964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac019070 con 0x7ff2b4156a50 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2ab400640 1 -- 172.21.15.37:0/1927887964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2ac00cc30 con 0x7ff2b4156a50 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2ab400640 1 -- 172.21.15.37:0/1927887964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0146a0 con 0x7ff2b4156a50 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41f54a0 con 0x7ff2b4156a50 2024-09-21T06:35:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.704+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41f91e0 con 0x7ff2b4156a50 2024-09-21T06:35:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.707+0000 7ff2ab400640 1 -- 172.21.15.37:0/1927887964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2ac01c070 con 0x7ff2b4156a50 2024-09-21T06:35:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.708+0000 7ff2ab400640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 0x7ff29007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.708+0000 7ff2ab400640 1 -- 172.21.15.37:0/1927887964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7ff2ac09ac50 con 0x7ff2b4156a50 2024-09-21T06:35:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.708+0000 7ff2c0e71740 1 Processor -- start 2024-09-21T06:35:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.709+0000 7ff2c0e71740 1 -- start start 2024-09-21T06:35:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.709+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 0x7ff29007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 0x7ff29007b0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4033000 tx=0x7ff2a4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f4ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f09d6b0 0x555b3f48f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555b3f3c51c0 con 0x555b3f0954d0 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b3f3d8870 con 0x555b3f0be390 2024-09-21T06:35:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555b3f3c7ca0 con 0x555b3f09d6b0 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37648/0 (socket says 172.21.15.37:37648) 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.710+0000 7ff2abe00640 1 -- 172.21.15.37:0/720481383 learned_addr learned my addr 172.21.15.37:0/720481383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f4ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2b3400640 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f09d6b0 0x555b3f48f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 -- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f09d6b0 msgr2=0x555b3f48f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f09d6b0 0x555b3f48f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 -- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 msgr2=0x555b3f4ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f4ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 -- 172.21.15.37:0/720481383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3f1a07d0 con 0x555b3f0954d0 2024-09-21T06:35:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2abe00640 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00cc90 tx=0x7ff29c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fd241e44a9190f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2a3400640 1 -- 172.21.15.37:0/720481383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c014070 con 0x555b3f0954d0 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2a3400640 1 -- 172.21.15.37:0/720481383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff29c005310 con 0x555b3f0954d0 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.711+0000 7ff2a3400640 1 -- 172.21.15.37:0/720481383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0047a0 con 0x555b3f0954d0 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 -- 172.21.15.37:0/720481383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 msgr2=0x555b3f48f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00cc90 tx=0x7ff29c00e720 comp rx=0 tx=0).stop 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 -- 172.21.15.37:0/720481383 shutdown_connections 2024-09-21T06:35:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f48f220 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f09d6b0 0x555b3f48f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/720481383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f4ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 -- 172.21.15.37:0/720481383 >> 172.21.15.37:0/720481383 conn(0x555b3f0b24e0 msgr2=0x555b3f465440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 -- 172.21.15.37:0/720481383 shutdown_connections 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.712+0000 7ff2c0e71740 1 -- 172.21.15.37:0/720481383 wait complete. 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.713+0000 7ff2c0e71740 1 -- start start 2024-09-21T06:35:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.714+0000 7ff2c0e71740 1 Processor -- start 2024-09-21T06:35:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f465140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555b3f36e410 con 0x555b3f0be390 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b3f3cbab0 con 0x555b3f09d6b0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555b3f138740 con 0x555b3f0954d0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2c0e71740 10 client.0 _ll_register_callbacks cb 0x555b3f34b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2b3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2b3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37652/0 (socket says 172.21.15.37:37652) 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.715+0000 7ff2b3400640 1 -- 172.21.15.37:0/1063094136 learned_addr learned my addr 172.21.15.37:0/1063094136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f465140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2abe00640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 -- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 msgr2=0x555b3f465140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f465140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 -- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 msgr2=0x555b3f4875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 -- 172.21.15.37:0/1063094136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3f129970 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2b3400640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00bf60 tx=0x7ff2ac062da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32219205ab6e81d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff297e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac088070 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7ff2ac05e0f0 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff29c005310 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7ff2ac0671d0 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2c0e71740 10 client.8790 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.716+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x555b3f1076a0 con 0x555b3f0be390 2024-09-21T06:35:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff29c005310 con 0x555b3f0be390 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7ff2ac05e2f0 con 0x555b3f0be390 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 10 client.8790 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 10 client.8790 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 20 client.8790 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f07cf60 0x555b3f488990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 4 client.8790 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3f3da7c0 con 0x555b3f07cf60 2024-09-21T06:35:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.717+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f07cf60 0x555b3f488990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.718+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f07cf60 0x555b3f488990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a83f8f2928b60a1 server_cookie=1e159a48403212f8 in_seq=0 out_seq=0 2024-09-21T06:35:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.718+0000 7ff297e00640 10 client.8790 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.719+0000 7ff297e00640 1 -- 172.21.15.37:0/1063094136 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b3f3da7c0 con 0x555b3f07cf60 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.719+0000 7ff297e00640 10 client.8790 handle_command_reply: tid=0 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.719+0000 7ff2c0e71740 1 client.8790 shutdown 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 msgr2=0x555b3f4883c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00bf60 tx=0x7ff2ac062da0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 shutdown_connections 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f07cf60 0x555b3f488990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0be390 0x555b3f4883c0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/1063094136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f465140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff297e00640 0 client.8790 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 >> 172.21.15.37:0/1063094136 conn(0x555b3f0b24e0 msgr2=0x555b3f47e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 shutdown_connections 2024-09-21T06:35:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 1 -- 172.21.15.37:0/1063094136 wait complete. 2024-09-21T06:35:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.720+0000 7ff2c0e71740 20 client.8790 trim_cache size 0 max 16384 2024-09-21T06:35:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.739+0000 7ff2c0e71740 1 Processor -- start 2024-09-21T06:35:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.739+0000 7ff2c0e71740 1 -- start start 2024-09-21T06:35:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0be390 0x555b3f47e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4202870 con 0x555b3f0954d0 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4201a10 con 0x555b3f09d6b0 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b4202140 con 0x555b3f0be390 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2b2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2b2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37666/0 (socket says 172.21.15.37:37666) 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.740+0000 7ff2b2a00640 1 -- 172.21.15.37:0/859691722 learned_addr learned my addr 172.21.15.37:0/859691722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 -- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0be390 msgr2=0x555b3f47e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0be390 0x555b3f47e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 -- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 msgr2=0x555b3f4c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 -- 172.21.15.37:0/859691722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3f488430 con 0x555b3f0954d0 2024-09-21T06:35:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000c00 tx=0x7ff2a403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24d567e1da14519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2a2a00640 1 -- 172.21.15.37:0/859691722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4055070 con 0x555b3f0954d0 2024-09-21T06:35:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2a2a00640 1 -- 172.21.15.37:0/859691722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a4003a80 con 0x555b3f0954d0 2024-09-21T06:35:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.741+0000 7ff2a2a00640 1 -- 172.21.15.37:0/859691722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4035040 con 0x555b3f0954d0 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 -- 172.21.15.37:0/859691722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 msgr2=0x555b3f4b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000c00 tx=0x7ff2a403d000 comp rx=0 tx=0).stop 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 -- 172.21.15.37:0/859691722 shutdown_connections 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0be390 0x555b3f47e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f09d6b0 0x555b3f4c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/859691722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f0954d0 0x555b3f4b33a0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 -- 172.21.15.37:0/859691722 >> 172.21.15.37:0/859691722 conn(0x555b3f0b24e0 msgr2=0x555b3f460720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 -- 172.21.15.37:0/859691722 shutdown_connections 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.742+0000 7ff2c0e71740 1 -- 172.21.15.37:0/859691722 wait complete. 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.743+0000 7ff2c0e71740 1 -- start start 2024-09-21T06:35:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.743+0000 7ff2c0e71740 1 Processor -- start 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.744+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f523410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.744+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2c0e71740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f48f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4200c20 con 0x555b3f09d6b0 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b3f4657f0 con 0x555b3f0be390 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2c0e71740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x555b3f47e620 con 0x555b3f0954d0 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2c0e71740 10 client.0 _ll_register_callbacks cb 0x7ff2ac05edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37676/0 (socket says 172.21.15.37:37676) 2024-09-21T06:35:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 -- 172.21.15.37:0/3843491151 learned_addr learned my addr 172.21.15.37:0/3843491151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 -- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 msgr2=0x555b3f523410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f523410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 -- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 msgr2=0x555b3f48f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2b3400640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f48f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f48f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2abe00640 1 -- 172.21.15.37:0/3843491151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3f4668f0 con 0x555b3f09d6b0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.745+0000 7ff2b3400640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f48f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2abe00640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00cc60 tx=0x7ff29c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79248582b51a0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c014040 con 0x555b3f09d6b0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7ff29c004220 con 0x555b3f09d6b0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a4003a80 con 0x555b3f09d6b0 2024-09-21T06:35:41.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7ff29c02c050 con 0x555b3f09d6b0 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 10 client.8796 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x555b3f4b8770 con 0x555b3f09d6b0 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a4003a80 con 0x555b3f09d6b0 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7ff29c02f020 con 0x555b3f09d6b0 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 10 client.8796 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 10 client.8796 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 20 client.8796 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f488950 0x555b3f4b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 4 client.8796 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.746+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b3f3d6270 con 0x555b3f488950 2024-09-21T06:35:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.747+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f488950 0x555b3f4b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.747+0000 7ff2b2a00640 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f488950 0x555b3f4b7a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38c4e21e289fb5d9 server_cookie=4e3331a2cd35a2e in_seq=0 out_seq=0 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.747+0000 7ff297e00640 10 client.8796 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff297e00640 1 -- 172.21.15.37:0/3843491151 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555b3f3d6270 con 0x555b3f488950 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff297e00640 10 client.8796 handle_command_reply: tid=0 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 client.8796 shutdown 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 msgr2=0x555b3f47cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00cc60 tx=0x7ff29c00c9a0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 shutdown_connections 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x555b3f488950 0x555b3f4b7a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b3f09d6b0 0x555b3f47cf90 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b3f0be390 0x555b3f48f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff297e00640 0 client.8796 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 --2- 172.21.15.37:0/3843491151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555b3f0954d0 0x555b3f523410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 >> 172.21.15.37:0/3843491151 conn(0x555b3f0b24e0 msgr2=0x555b3f47ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 shutdown_connections 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 1 -- 172.21.15.37:0/3843491151 wait complete. 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.748+0000 7ff2c0e71740 20 client.8796 trim_cache size 0 max 16384 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:41.754 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:41.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:41.755 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 msgr2=0x7ff29007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 0x7ff29007b0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4033000 tx=0x7ff2a4004730 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 msgr2=0x7ff2b40a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b970 tx=0x7ff2ac00be30 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 shutdown_connections 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff290078bc0 0x7ff29007b0d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4156a50 0x7ff2b40a0380 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b4074b60 0x7ff2b409fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 --2- 172.21.15.37:0/1927887964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b406e330 0x7ff2b40cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 >> 172.21.15.37:0/1927887964 conn(0x7ff2b40b1b90 msgr2=0x7ff2b406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 shutdown_connections 2024-09-21T06:35:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:41.749+0000 7ff2b9e00640 1 -- 172.21.15.37:0/1927887964 wait complete. 2024-09-21T06:35:41.773 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:41.773 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:41.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T06:35:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 -- 172.21.15.37:0/2746598222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 msgr2=0x7f535c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 --2- 172.21.15.37:0/2746598222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c06e200 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f535c06fe40 tx=0x7f534402a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 -- 172.21.15.37:0/2746598222 shutdown_connections 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 --2- 172.21.15.37:0/2746598222 >> v2:172.21.15.150:3300/0 conn(0x7f535c0cef00 0x7f535c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 --2- 172.21.15.37:0/2746598222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c06e200 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 -- 172.21.15.37:0/2746598222 >> 172.21.15.37:0/2746598222 conn(0x7f535c0b1b90 msgr2=0x7f535c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 -- 172.21.15.37:0/2746598222 shutdown_connections 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.001+0000 7f5361e00640 1 -- 172.21.15.37:0/2746598222 wait complete. 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.003+0000 7f5361e00640 1 Processor -- start 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.004+0000 7f5361e00640 1 -- start start 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f535c0cef00 0x7f535c1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f535c156a50 0x7f535c1f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f535c0cf850 con 0x7f535c06ddf0 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f535c161060 con 0x7f535c0cef00 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f5361e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f535c0cf9d0 con 0x7f535c156a50 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.005+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37694/0 (socket says 172.21.15.37:37694) 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 -- 172.21.15.37:0/398096546 learned_addr learned my addr 172.21.15.37:0/398096546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f5352000640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f535c0cef00 0x7f535c1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535b400640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f535c156a50 0x7f535c1f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 -- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f535c156a50 msgr2=0x7f535c1f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f535c156a50 0x7f535c1f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 -- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f535c0cef00 msgr2=0x7f535c1f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f535c0cef00 0x7f535c1f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 -- 172.21.15.37:0/398096546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535c0c7390 con 0x7f535c06ddf0 2024-09-21T06:35:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f535aa00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c3c0 tx=0x7f534c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.006+0000 7f5353e00640 1 -- 172.21.15.37:0/398096546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c017070 con 0x7f535c06ddf0 2024-09-21T06:35:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.007+0000 7f5353e00640 1 -- 172.21.15.37:0/398096546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c0035f0 con 0x7f535c06ddf0 2024-09-21T06:35:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.007+0000 7f5353e00640 1 -- 172.21.15.37:0/398096546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c003a80 con 0x7f535c06ddf0 2024-09-21T06:35:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.007+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535c1f7800 con 0x7f535c06ddf0 2024-09-21T06:35:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.007+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535c15a070 con 0x7f535c06ddf0 2024-09-21T06:35:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.009+0000 7f5353e00640 1 -- 172.21.15.37:0/398096546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f534c01a070 con 0x7f535c06ddf0 2024-09-21T06:35:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.011+0000 7f5353e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 0x7f533807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.011+0000 7f5353e00640 1 -- 172.21.15.37:0/398096546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f534c09a2d0 con 0x7f535c06ddf0 2024-09-21T06:35:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.011+0000 7f5352000640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 0x7f533807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.011+0000 7f5368e62740 1 Processor -- start 2024-09-21T06:35:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.011+0000 7f5368e62740 1 -- start start 2024-09-21T06:35:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.012+0000 7f5352000640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 0x7f533807b0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5344033000 tx=0x7f5344004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9f24d0 0x561b2ddec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2ddec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561b2dd221c0 con 0x561b2da1b390 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b2dd35870 con 0x561b2d9f24d0 2024-09-21T06:35:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561b2dd24ca0 con 0x561b2d9fa6b0 2024-09-21T06:35:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37706/0 (socket says 172.21.15.37:37706) 2024-09-21T06:35:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 -- 172.21.15.37:0/424499566 learned_addr learned my addr 172.21.15.37:0/424499566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 -- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 msgr2=0x561b2ddec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2ddec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 -- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9f24d0 msgr2=0x561b2ddec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9f24d0 0x561b2ddec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.013+0000 7f535aa00640 1 -- 172.21.15.37:0/424499566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2dafd7d0 con 0x561b2da1b390 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.014+0000 7f535aa00640 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f534c05de10 tx=0x7f534c061630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2616ebf63d5fc17a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.014+0000 7f534b400640 1 -- 172.21.15.37:0/424499566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c09a0d0 con 0x561b2da1b390 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.014+0000 7f534b400640 1 -- 172.21.15.37:0/424499566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c0668e0 con 0x561b2da1b390 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.014+0000 7f534b400640 1 -- 172.21.15.37:0/424499566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c06b790 con 0x561b2da1b390 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 -- 172.21.15.37:0/424499566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 msgr2=0x561b2de09550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f534c05de10 tx=0x7f534c061630 comp rx=0 tx=0).stop 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 -- 172.21.15.37:0/424499566 shutdown_connections 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2ddec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9f24d0 0x561b2ddec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 --2- 172.21.15.37:0/424499566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2de09550 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 -- 172.21.15.37:0/424499566 >> 172.21.15.37:0/424499566 conn(0x561b2da0f4e0 msgr2=0x561b2ddc2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 -- 172.21.15.37:0/424499566 shutdown_connections 2024-09-21T06:35:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.015+0000 7f5368e62740 1 -- 172.21.15.37:0/424499566 wait complete. 2024-09-21T06:35:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.016+0000 7f5368e62740 1 -- start start 2024-09-21T06:35:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.016+0000 7f5368e62740 1 Processor -- start 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9f24d0 0x561b2de05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2ddfed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561b2dccb410 con 0x561b2da1b390 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b2dd28ab0 con 0x561b2d9fa6b0 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561b2da95740 con 0x561b2d9f24d0 2024-09-21T06:35:42.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f5368e62740 10 client.0 _ll_register_callbacks cb 0x561b2dca8360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:42.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37710/0 (socket says 172.21.15.37:37710) 2024-09-21T06:35:42.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 -- 172.21.15.37:0/3829024661 learned_addr learned my addr 172.21.15.37:0/3829024661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 -- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9f24d0 msgr2=0x561b2de05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9f24d0 0x561b2de05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 -- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 msgr2=0x561b2ddfed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2ddfed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.018+0000 7f535b400640 1 -- 172.21.15.37:0/3829024661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2da86970 con 0x561b2da1b390 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f535b400640 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f5354010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c684f16769fae62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f533fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354004d20 con 0x561b2da1b390 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f5354012720 con 0x561b2da1b390 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534c0668e0 con 0x561b2da1b390 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f5354023050 con 0x561b2da1b390 2024-09-21T06:35:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f5368e62740 10 client.8811 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.019+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x561b2da646a0 con 0x561b2da1b390 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c0668e0 con 0x561b2da1b390 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f5354026020 con 0x561b2da1b390 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 10 client.8811 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 10 client.8811 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 20 client.8811 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2d9d9f60 0x561b2de00420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 4 client.8811 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2dd377c0 con 0x561b2d9d9f60 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.020+0000 7f535aa00640 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2d9d9f60 0x561b2de00420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.021+0000 7f535aa00640 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2d9d9f60 0x561b2de00420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a50d18c7697d4c2 server_cookie=668cd96537f737a1 in_seq=0 out_seq=0 2024-09-21T06:35:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.021+0000 7f533fe00640 10 client.8811 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.021+0000 7f533fe00640 1 -- 172.21.15.37:0/3829024661 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b2dd377c0 con 0x561b2d9d9f60 2024-09-21T06:35:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.021+0000 7f533fe00640 10 client.8811 handle_command_reply: tid=0 2024-09-21T06:35:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.021+0000 7f5368e62740 1 client.8811 shutdown 2024-09-21T06:35:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 msgr2=0x561b2ddff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f5354010b00 comp rx=0 tx=0).stop 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 shutdown_connections 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2d9d9f60 0x561b2de00420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2da1b390 0x561b2ddff2c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2ddfed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 --2- 172.21.15.37:0/3829024661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9f24d0 0x561b2de05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f533fe00640 0 client.8811 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 >> 172.21.15.37:0/3829024661 conn(0x561b2da0f4e0 msgr2=0x561b2ddbb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 shutdown_connections 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 1 -- 172.21.15.37:0/3829024661 wait complete. 2024-09-21T06:35:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.022+0000 7f5368e62740 20 client.8811 trim_cache size 0 max 16384 2024-09-21T06:35:42.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.041+0000 7f5368e62740 1 Processor -- start 2024-09-21T06:35:42.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.041+0000 7f5368e62740 1 -- start start 2024-09-21T06:35:42.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2dde31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2da1b390 0x561b2dde36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f535c200720 con 0x561b2d9f24d0 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1ff8c0 con 0x561b2da1b390 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f535c1ffff0 con 0x561b2d9fa6b0 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.042+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37720/0 (socket says 172.21.15.37:37720) 2024-09-21T06:35:42.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 -- 172.21.15.37:0/812598426 learned_addr learned my addr 172.21.15.37:0/812598426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 -- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 msgr2=0x561b2dde31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2dde31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 -- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2da1b390 msgr2=0x561b2dde36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2da1b390 0x561b2dde36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 -- 172.21.15.37:0/812598426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2ddc2c90 con 0x561b2d9f24d0 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f535aa00640 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f534c061d90 tx=0x7f534c05e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca6dec1ec3a6b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f534aa00640 1 -- 172.21.15.37:0/812598426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c09a0b0 con 0x561b2d9f24d0 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f534aa00640 1 -- 172.21.15.37:0/812598426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c06b790 con 0x561b2d9f24d0 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.043+0000 7f534aa00640 1 -- 172.21.15.37:0/812598426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c061060 con 0x561b2d9f24d0 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 -- 172.21.15.37:0/812598426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 msgr2=0x561b2ddff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f534c061d90 tx=0x7f534c05e450 comp rx=0 tx=0).stop 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 -- 172.21.15.37:0/812598426 shutdown_connections 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2da1b390 0x561b2dde36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2d9fa6b0 0x561b2dde31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 --2- 172.21.15.37:0/812598426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2ddff210 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 -- 172.21.15.37:0/812598426 >> 172.21.15.37:0/812598426 conn(0x561b2da0f4e0 msgr2=0x561b2ddb7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 -- 172.21.15.37:0/812598426 shutdown_connections 2024-09-21T06:35:42.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.044+0000 7f5368e62740 1 -- 172.21.15.37:0/812598426 wait complete. 2024-09-21T06:35:42.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.045+0000 7f5368e62740 1 -- start start 2024-09-21T06:35:42.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.045+0000 7f5368e62740 1 Processor -- start 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2de03560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2da1b390 0x561b2de89a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f535c0ca1e0 con 0x561b2d9f24d0 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b2de08120 con 0x561b2d9fa6b0 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x561b2ddff210 con 0x561b2da1b390 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f5368e62740 10 client.0 _ll_register_callbacks cb 0x7f5354009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f535aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37734/0 (socket says 172.21.15.37:37734) 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.047+0000 7f535aa00640 1 -- 172.21.15.37:0/3586378604 learned_addr learned my addr 172.21.15.37:0/3586378604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f5352000640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2de03560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535b400640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2da1b390 0x561b2de89a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 -- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2da1b390 msgr2=0x561b2de89a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2da1b390 0x561b2de89a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 -- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 msgr2=0x561b2de03560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2de03560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 -- 172.21.15.37:0/3586378604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2de00fa0 con 0x561b2d9f24d0 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f5352000640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2de03560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f535aa00640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f534c061b00 tx=0x7f534c000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e9d68b7e63cc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f533fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c0668e0 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f534c069bd0 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534c06b790 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f534c069df0 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f5368e62740 10 client.8817 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.048+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x561b2ddff460 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c06b790 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f534c061060 con 0x561b2d9f24d0 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 10 client.8817 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 10 client.8817 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 20 client.8817 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:42.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2ddbbe70 0x561b2de89fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 4 client.8817 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b2dd33270 con 0x561b2ddbbe70 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.049+0000 7f5352000640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2ddbbe70 0x561b2de89fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5352000640 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2ddbbe70 0x561b2de89fc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2160148ad0497511 server_cookie=14618b7e4e9464a0 in_seq=0 out_seq=0 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f533fe00640 10 client.8817 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f533fe00640 1 -- 172.21.15.37:0/3586378604 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x561b2dd33270 con 0x561b2ddbbe70 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f533fe00640 10 client.8817 handle_command_reply: tid=0 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 client.8817 shutdown 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 msgr2=0x561b2de03020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f534c061b00 tx=0x7f534c000f60 comp rx=0 tx=0).stop 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 shutdown_connections 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x561b2ddbbe70 0x561b2de89fc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561b2da1b390 0x561b2de89a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2d9fa6b0 0x561b2de03560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 --2- 172.21.15.37:0/3586378604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561b2d9f24d0 0x561b2de03020 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f533fe00640 0 client.8817 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 >> 172.21.15.37:0/3586378604 conn(0x561b2da0f4e0 msgr2=0x561b2dde31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.050+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 shutdown_connections 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5368e62740 1 -- 172.21.15.37:0/3586378604 wait complete. 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5368e62740 20 client.8817 trim_cache size 0 max 16384 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-21T06:35:42.056 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:35:31.440721+0000", 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:35:42.057 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:35:42.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:42.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:42.058 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:42.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 msgr2=0x7f533807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 0x7f533807b0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5344033000 tx=0x7f5344004730 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 msgr2=0x7f535c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c3c0 tx=0x7f534c00c880 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 shutdown_connections 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5338078b90 0x7f533807b0a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f535c156a50 0x7f535c1f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f535c0cef00 0x7f535c1f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 --2- 172.21.15.37:0/398096546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f535c06ddf0 0x7f535c1554d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 >> 172.21.15.37:0/398096546 conn(0x7f535c0b1b90 msgr2=0x7f535c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.051+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 shutdown_connections 2024-09-21T06:35:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.052+0000 7f5361e00640 1 -- 172.21.15.37:0/398096546 wait complete. 2024-09-21T06:35:42.075 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:42.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.300+0000 7f4473200640 1 -- 172.21.15.37:0/12586056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4474073f20 msgr2=0x7f447415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.300+0000 7f4473200640 1 --2- 172.21.15.37:0/12586056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4474073f20 0x7f447415b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4464009860 tx=0x7f446402a4a0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.301+0000 7f4473200640 1 -- 172.21.15.37:0/12586056 shutdown_connections 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.301+0000 7f4473200640 1 --2- 172.21.15.37:0/12586056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4474073f20 0x7f447415b2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.301+0000 7f4473200640 1 -- 172.21.15.37:0/12586056 >> 172.21.15.37:0/12586056 conn(0x7f4474073400 msgr2=0x7f4474073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.301+0000 7f4473200640 1 -- 172.21.15.37:0/12586056 shutdown_connections 2024-09-21T06:35:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.301+0000 7f4473200640 1 -- 172.21.15.37:0/12586056 wait complete. 2024-09-21T06:35:42.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.303+0000 7f4473200640 1 Processor -- start 2024-09-21T06:35:42.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.303+0000 7f4473200640 1 -- start start 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447406ddf0 0x7f44741569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44740cb0c0 0x7f44740b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44740cbf70 con 0x7f4474073f20 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44740cbdf0 con 0x7f447406ddf0 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4473200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f44740cc0f0 con 0x7f44740cb0c0 2024-09-21T06:35:42.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37746/0 (socket says 172.21.15.37:37746) 2024-09-21T06:35:42.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 -- 172.21.15.37:0/2311069389 learned_addr learned my addr 172.21.15.37:0/2311069389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4471e00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447406ddf0 0x7f44741569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f4472800640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44740cb0c0 0x7f44740b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 -- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44740cb0c0 msgr2=0x7f44740b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44740cb0c0 0x7f44740b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 -- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447406ddf0 msgr2=0x7f44741569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.305+0000 7f446be00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447406ddf0 0x7f44741569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f446be00640 1 -- 172.21.15.37:0/2311069389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740c50d0 con 0x7f4474073f20 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f446be00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c430 tx=0x7f445c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c017070 con 0x7f4474073f20 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f445c0053e0 con 0x7f4474073f20 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44740c4b10 con 0x7f4474073f20 2024-09-21T06:35:42.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c003850 con 0x7f4474073f20 2024-09-21T06:35:42.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.306+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44740c5d10 con 0x7f4474073f20 2024-09-21T06:35:42.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.307+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447415b2f0 con 0x7f4474073f20 2024-09-21T06:35:42.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.309+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f445c01a070 con 0x7f4474073f20 2024-09-21T06:35:42.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.310+0000 7f446a000640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.310+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f445c09a330 con 0x7f4474073f20 2024-09-21T06:35:42.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.310+0000 7f4471e00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.311+0000 7f4471e00640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 0x7f445007b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4464034000 tx=0x7f4464004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.314+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f445c061e80 con 0x7f4474073f20 2024-09-21T06:35:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.553+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4474073af0 con 0x7f4474073f20 2024-09-21T06:35:42.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.554+0000 7f446a000640 1 -- 172.21.15.37:0/2311069389 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f445c0668b0 con 0x7f4474073f20 2024-09-21T06:35:42.556 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:42.556 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:42.557 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 msgr2=0x7f445007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 0x7f445007b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4464034000 tx=0x7f4464004730 comp rx=0 tx=0).stop 2024-09-21T06:35:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 msgr2=0x7f44740c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c430 tx=0x7f445c00c8f0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 shutdown_connections 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4474073f20 0x7f44740c41c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44740cb0c0 0x7f44740b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 --2- 172.21.15.37:0/2311069389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447406ddf0 0x7f44741569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 >> 172.21.15.37:0/2311069389 conn(0x7f4474073400 msgr2=0x7f44740b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 shutdown_connections 2024-09-21T06:35:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.557+0000 7f4473200640 1 -- 172.21.15.37:0/2311069389 wait complete. 2024-09-21T06:35:42.584 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:42.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.810+0000 7f5d12400640 1 -- 172.21.15.37:0/4081733334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c074cb0 msgr2=0x7f5d0c073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.810+0000 7f5d12400640 1 --2- 172.21.15.37:0/4081733334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c073dd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009830 tx=0x7f5cfc02a5c0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.811+0000 7f5d12400640 1 -- 172.21.15.37:0/4081733334 shutdown_connections 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.811+0000 7f5d12400640 1 --2- 172.21.15.37:0/4081733334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c073dd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.811+0000 7f5d12400640 1 -- 172.21.15.37:0/4081733334 >> 172.21.15.37:0/4081733334 conn(0x7f5d0c0cca40 msgr2=0x7f5d0c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.811+0000 7f5d12400640 1 -- 172.21.15.37:0/4081733334 shutdown_connections 2024-09-21T06:35:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.811+0000 7f5d12400640 1 -- 172.21.15.37:0/4081733334 wait complete. 2024-09-21T06:35:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.813+0000 7f5d12400640 1 Processor -- start 2024-09-21T06:35:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.813+0000 7f5d12400640 1 -- start start 2024-09-21T06:35:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cbc10 0x7f5d0c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c0c8b00 0x7f5d0c09ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c06f670 con 0x7f5d0c074cb0 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c06f4f0 con 0x7f5d0c0c8b00 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.814+0000 7f5d12400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c06f7f0 con 0x7f5d0c0cbc10 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37758/0 (socket says 172.21.15.37:37758) 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 -- 172.21.15.37:0/4142186480 learned_addr learned my addr 172.21.15.37:0/4142186480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0aa00640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cbc10 0x7f5d0c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0be00640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c0c8b00 0x7f5d0c09ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 -- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cbc10 msgr2=0x7f5d0c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cbc10 0x7f5d0c0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 -- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c0c8b00 msgr2=0x7f5d0c09ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c0c8b00 0x7f5d0c09ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.815+0000 7f5d0b400640 1 -- 172.21.15.37:0/4142186480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c1f89e0 con 0x7f5d0c074cb0 2024-09-21T06:35:42.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d0b400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009950 tx=0x7f5cfc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc031d60 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cfc033e10 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc033310 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c1f8cc0 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.816+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1f9280 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.817+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c073dd0 con 0x7f5d0c074cb0 2024-09-21T06:35:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.818+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5cfc0334b0 con 0x7f5d0c074cb0 2024-09-21T06:35:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.819+0000 7f5d03400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 0x7f5cec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.819+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f5cfc0bc240 con 0x7f5d0c074cb0 2024-09-21T06:35:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.820+0000 7f5d0aa00640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 0x7f5cec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.821+0000 7f5d0aa00640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 0x7f5cec07b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0c8220 tx=0x7f5cf8007cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:42.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:42.824+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cfc03f030 con 0x7f5d0c074cb0 2024-09-21T06:35:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.098+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5d0c0cb610 con 0x7f5d0c074cb0 2024-09-21T06:35:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.099+0000 7f5d03400640 1 -- 172.21.15.37:0/4142186480 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1991 (secure 0 0 0) 0x7f5cfc083e20 con 0x7f5d0c074cb0 2024-09-21T06:35:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:43.102 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":491,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":267280384,"bytes_avail":772826832896,"bytes_total":773094113280,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-09-21T06:35:39.217213+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.101+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 msgr2=0x7f5cec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.101+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 0x7f5cec07b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0c8220 tx=0x7f5cf8007cc0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 msgr2=0x7f5d0c0a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009950 tx=0x7f5cfc033c20 comp rx=0 tx=0).stop 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 shutdown_connections 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5cec078b90 0x7f5cec07b0a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c0c8b00 0x7f5d0c09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cbc10 0x7f5d0c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 --2- 172.21.15.37:0/4142186480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d0c074cb0 0x7f5d0c0a0c60 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 >> 172.21.15.37:0/4142186480 conn(0x7f5d0c0cca40 msgr2=0x7f5d0c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 shutdown_connections 2024-09-21T06:35:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.102+0000 7f5d12400640 1 -- 172.21.15.37:0/4142186480 wait complete. 2024-09-21T06:35:43.130 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:43.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:43.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-09-21T06:35:43.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 -- 172.21.15.37:0/572918373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 msgr2=0x7fbe240b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 --2- 172.21.15.37:0/572918373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe240b3740 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbe14005fd0 tx=0x7fbe1402a670 comp rx=0 tx=0).stop 2024-09-21T06:35:43.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 -- 172.21.15.37:0/572918373 shutdown_connections 2024-09-21T06:35:43.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 --2- 172.21.15.37:0/572918373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe240b3740 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 -- 172.21.15.37:0/572918373 >> 172.21.15.37:0/572918373 conn(0x7fbe240735d0 msgr2=0x7fbe240739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 -- 172.21.15.37:0/572918373 shutdown_connections 2024-09-21T06:35:43.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.356+0000 7fbe29e00640 1 -- 172.21.15.37:0/572918373 wait complete. 2024-09-21T06:35:43.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.358+0000 7fbe29e00640 1 Processor -- start 2024-09-21T06:35:43.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.358+0000 7fbe29e00640 1 -- start start 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.359+0000 7fbe29e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe240740f0 0x7fbe241f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe29e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe29e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 0x7fbe2415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe29e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240cff50 con 0x7fbe2415c740 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe29e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240cfdd0 con 0x7fbe240740f0 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe29e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbe240d00d0 con 0x7fbe241f93b0 2024-09-21T06:35:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37776/0 (socket says 172.21.15.37:37776) 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 -- 172.21.15.37:0/1675789406 learned_addr learned my addr 172.21.15.37:0/1675789406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe240740f0 0x7fbe241f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe23400640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 0x7fbe2415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 -- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 msgr2=0x7fbe2415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 0x7fbe2415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 -- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe240740f0 msgr2=0x7fbe241f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe240740f0 0x7fbe241f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe1be00640 1 -- 172.21.15.37:0/1675789406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe240c7c20 con 0x7fbe2415c740 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.360+0000 7fbe23400640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 0x7fbe2415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe1be00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c3d0 tx=0x7fbe0c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe1b400640 1 -- 172.21.15.37:0/1675789406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c017070 con 0x7fbe2415c740 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe1b400640 1 -- 172.21.15.37:0/1675789406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe0c0053c0 con 0x7fbe2415c740 2024-09-21T06:35:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe240c7f00 con 0x7fbe2415c740 2024-09-21T06:35:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe1b400640 1 -- 172.21.15.37:0/1675789406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c003850 con 0x7fbe2415c740 2024-09-21T06:35:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.361+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe240c84c0 con 0x7fbe2415c740 2024-09-21T06:35:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.364+0000 7fbe1b400640 1 -- 172.21.15.37:0/1675789406 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe0c01a070 con 0x7fbe2415c740 2024-09-21T06:35:43.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.364+0000 7fbe1b400640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.365+0000 7fbe1b400640 1 -- 172.21.15.37:0/1675789406 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fbe0c09a180 con 0x7fbe2415c740 2024-09-21T06:35:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.365+0000 7fbe30e3a740 1 Processor -- start 2024-09-21T06:35:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.365+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.365+0000 7fbe30e3a740 1 -- start start 2024-09-21T06:35:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.366+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 0x7fbe0007b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe14033000 tx=0x7fbe14004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x5640873c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 0x5640873c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640872f71c0 con 0x564086ff0390 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56408730a870 con 0x564086fcf6b0 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5640872f9ca0 con 0x564086fc74d0 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe22a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe22a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37784/0 (socket says 172.21.15.37:37784) 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe22a00640 1 -- 172.21.15.37:0/1611684367 learned_addr learned my addr 172.21.15.37:0/1611684367 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe1be00640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x5640873c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.367+0000 7fbe23400640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 0x5640873c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 -- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 msgr2=0x5640873c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x5640873c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 -- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 msgr2=0x5640873c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 0x5640873c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 -- 172.21.15.37:0/1611684367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640870d27d0 con 0x564086ff0390 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe23400640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 0x5640873c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe22a00640 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbe1402ac60 tx=0x7fbe1403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d519de69d64e771 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe13400640 1 -- 172.21.15.37:0/1611684367 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14035040 con 0x564086ff0390 2024-09-21T06:35:43.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe13400640 1 -- 172.21.15.37:0/1611684367 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1404d070 con 0x564086ff0390 2024-09-21T06:35:43.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.368+0000 7fbe13400640 1 -- 172.21.15.37:0/1611684367 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14003da0 con 0x564086ff0390 2024-09-21T06:35:43.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 -- 172.21.15.37:0/1611684367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 msgr2=0x5640873de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbe1402ac60 tx=0x7fbe1403d000 comp rx=0 tx=0).stop 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 -- 172.21.15.37:0/1611684367 shutdown_connections 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fcf6b0 0x5640873c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x5640873c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/1611684367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873de550 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 -- 172.21.15.37:0/1611684367 >> 172.21.15.37:0/1611684367 conn(0x564086fe44e0 msgr2=0x564087397440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 -- 172.21.15.37:0/1611684367 shutdown_connections 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.369+0000 7fbe30e3a740 1 -- 172.21.15.37:0/1611684367 wait complete. 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.370+0000 7fbe30e3a740 1 -- start start 2024-09-21T06:35:43.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.370+0000 7fbe30e3a740 1 Processor -- start 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fc74d0 0x5640873b9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x5640873aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640872a0410 con 0x564086ff0390 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5640872fdab0 con 0x564086fc74d0 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x56408706a740 con 0x564086fcf6b0 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe30e3a740 10 client.0 _ll_register_callbacks cb 0x56408727d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe23400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe23400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37792/0 (socket says 172.21.15.37:37792) 2024-09-21T06:35:43.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.372+0000 7fbe23400640 1 -- 172.21.15.37:0/881139990 learned_addr learned my addr 172.21.15.37:0/881139990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe1be00640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x5640873aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe22a00640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fc74d0 0x5640873b9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 -- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 msgr2=0x5640873aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x5640873aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 -- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fc74d0 msgr2=0x5640873b9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fc74d0 0x5640873b9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 -- 172.21.15.37:0/881139990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408705b970 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe23400640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0012e0 tx=0x7fbe1c005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5ba995fa481c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe07e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c013570 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fbe1c013710 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.373+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1404d070 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c021050 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 10 client.8844 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x5640870396a0 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1404d070 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fbe1c024020 con 0x564086ff0390 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 10 client.8844 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 10 client.8844 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 20 client.8844 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x564086faef60 0x5640873bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 4 client.8844 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408730c7c0 con 0x564086faef60 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.374+0000 7fbe22a00640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x564086faef60 0x5640873bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.375+0000 7fbe22a00640 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x564086faef60 0x5640873bc760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383575569a7feb75 server_cookie=7797fa3dd6fdd2d7 in_seq=0 out_seq=0 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.375+0000 7fbe07e00640 10 client.8844 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe07e00640 1 -- 172.21.15.37:0/881139990 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56408730c7c0 con 0x564086faef60 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe07e00640 10 client.8844 handle_command_reply: tid=0 2024-09-21T06:35:43.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 client.8844 shutdown 2024-09-21T06:35:43.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 msgr2=0x5640873d3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0012e0 tx=0x7fbe1c005920 comp rx=0 tx=0).stop 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 shutdown_connections 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x564086faef60 0x5640873bc760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086ff0390 0x5640873d3c40 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x5640873aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/881139990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086fc74d0 0x5640873b9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 >> 172.21.15.37:0/881139990 conn(0x564086fe44e0 msgr2=0x564087397540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe07e00640 0 client.8844 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 shutdown_connections 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.376+0000 7fbe30e3a740 1 -- 172.21.15.37:0/881139990 wait complete. 2024-09-21T06:35:43.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.377+0000 7fbe30e3a740 20 client.8844 trim_cache size 0 max 16384 2024-09-21T06:35:43.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.395+0000 7fbe30e3a740 1 Processor -- start 2024-09-21T06:35:43.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.395+0000 7fbe30e3a740 1 -- start start 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x564087398860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24202710 con 0x564086fc74d0 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe242018b0 con 0x564086ff0390 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbe24201fe0 con 0x564086fcf6b0 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe22a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.396+0000 7fbe22a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37802/0 (socket says 172.21.15.37:37802) 2024-09-21T06:35:43.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 -- 172.21.15.37:0/3144783326 learned_addr learned my addr 172.21.15.37:0/3144783326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 -- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 msgr2=0x564087398860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x564087398860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 -- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 msgr2=0x5640873c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 -- 172.21.15.37:0/3144783326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640873ad2f0 con 0x564086fc74d0 2024-09-21T06:35:43.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe22a00640 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fbe140099f0 tx=0x7fbe140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0be39924e86be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe12a00640 1 -- 172.21.15.37:0/3144783326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14040070 con 0x564086fc74d0 2024-09-21T06:35:43.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe12a00640 1 -- 172.21.15.37:0/3144783326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1404d040 con 0x564086fc74d0 2024-09-21T06:35:43.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.397+0000 7fbe12a00640 1 -- 172.21.15.37:0/3144783326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14061040 con 0x564086fc74d0 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 -- 172.21.15.37:0/3144783326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 msgr2=0x564087396130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fbe140099f0 tx=0x7fbe140046c0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 -- 172.21.15.37:0/3144783326 shutdown_connections 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fcf6b0 0x564087398860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/3144783326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fc74d0 0x564087396130 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 -- 172.21.15.37:0/3144783326 >> 172.21.15.37:0/3144783326 conn(0x564086fe44e0 msgr2=0x5640873decf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 -- 172.21.15.37:0/3144783326 shutdown_connections 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.398+0000 7fbe30e3a740 1 -- 172.21.15.37:0/3144783326 wait complete. 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.399+0000 7fbe30e3a740 1 -- start start 2024-09-21T06:35:43.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.399+0000 7fbe30e3a740 1 Processor -- start 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x56408745b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640873be190 con 0x564086fcf6b0 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2415db10 con 0x564086ff0390 2024-09-21T06:35:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5640873ad8b0 con 0x564086fc74d0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe30e3a740 10 client.0 _ll_register_callbacks cb 0x7fbe1c003b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37812/0 (socket says 172.21.15.37:37812) 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe1be00640 1 -- 172.21.15.37:0/399674750 learned_addr learned my addr 172.21.15.37:0/399674750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe23400640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.401+0000 7fbe22a00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x56408745b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 -- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 msgr2=0x56408745b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x56408745b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 -- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 msgr2=0x5640873972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 -- 172.21.15.37:0/399674750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640873fe620 con 0x564086fcf6b0 2024-09-21T06:35:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe1be00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c098440 tx=0x7fbe0c061ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c348797c0920eaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe07e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c077e60 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fbe0c069b10 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1404d040 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c06b6d0 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe30e3a740 10 client.8850 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.402+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x5640873ad5f0 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1404d040 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fbe0c05d740 con 0x564086fcf6b0 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 10 client.8850 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 10 client.8850 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 20 client.8850 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56408738d1f0 0x5640873fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 4 client.8850 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-21T06:35:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x564087308270 con 0x56408738d1f0 2024-09-21T06:35:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.403+0000 7fbe22a00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56408738d1f0 0x5640873fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe07e00640 10 client.8850 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe22a00640 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56408738d1f0 0x5640873fd450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d496d544a392ae server_cookie=6a6f8f20da7ef7f6 in_seq=0 out_seq=0 2024-09-21T06:35:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe07e00640 1 -- 172.21.15.37:0/399674750 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564087308270 con 0x56408738d1f0 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe07e00640 10 client.8850 handle_command_reply: tid=0 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 client.8850 shutdown 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 msgr2=0x564087396da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c098440 tx=0x7fbe0c061ce0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 shutdown_connections 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x56408738d1f0 0x5640873fd450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564086fcf6b0 0x564087396da0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564086ff0390 0x5640873972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 --2- 172.21.15.37:0/399674750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564086fc74d0 0x56408745b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 >> 172.21.15.37:0/399674750 conn(0x564086fe44e0 msgr2=0x5640873c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.404+0000 7fbe07e00640 0 client.8850 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 shutdown_connections 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe30e3a740 1 -- 172.21.15.37:0/399674750 wait complete. 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe30e3a740 20 client.8850 trim_cache size 0 max 16384 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-09-21T06:35:43.411 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 msgr2=0x7fbe0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 0x7fbe0007b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe14033000 tx=0x7fbe14004730 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 msgr2=0x7fbe2415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c3d0 tx=0x7fbe0c00c890 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 shutdown_connections 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe2415c740 0x7fbe2415e910 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe241f93b0 0x7fbe2415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 --2- 172.21.15.37:0/1675789406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe240740f0 0x7fbe241f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.405+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 >> 172.21.15.37:0/1675789406 conn(0x7fbe240735d0 msgr2=0x7fbe2415bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.406+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 shutdown_connections 2024-09-21T06:35:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.406+0000 7fbe29e00640 1 -- 172.21.15.37:0/1675789406 wait complete. 2024-09-21T06:35:43.433 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:43.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-21T06:35:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 -- 172.21.15.37:0/4021896246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4162890 msgr2=0x7fafb4154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 --2- 172.21.15.37:0/4021896246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4162890 0x7fafb4154f20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 -- 172.21.15.37:0/4021896246 shutdown_connections 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 --2- 172.21.15.37:0/4021896246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4162890 0x7fafb4154f20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 --2- 172.21.15.37:0/4021896246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb40d2760 0x7fafb4162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 -- 172.21.15.37:0/4021896246 >> 172.21.15.37:0/4021896246 conn(0x7fafb40d4360 msgr2=0x7fafb40d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 -- 172.21.15.37:0/4021896246 shutdown_connections 2024-09-21T06:35:43.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.660+0000 7fafb3e00640 1 -- 172.21.15.37:0/4021896246 wait complete. 2024-09-21T06:35:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.662+0000 7fafb3e00640 1 Processor -- start 2024-09-21T06:35:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.662+0000 7fafb3e00640 1 -- start start 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafb40d2760 0x7fafb415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb4162890 0x7fafb4156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fafb415f290 con 0x7fafb4155460 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40cfee0 con 0x7fafb40d2760 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fafb415f410 con 0x7fafb4162890 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37828/0 (socket says 172.21.15.37:37828) 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafabe00640 1 -- 172.21.15.37:0/3804059690 learned_addr learned my addr 172.21.15.37:0/3804059690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb3400640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb4162890 0x7fafb4156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafb2a00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafb40d2760 0x7fafb415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafabe00640 1 -- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb4162890 msgr2=0x7fafb4156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.664+0000 7fafabe00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb4162890 0x7fafb4156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafabe00640 1 -- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafb40d2760 msgr2=0x7fafb415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafabe00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafb40d2760 0x7fafb415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafabe00640 1 -- 172.21.15.37:0/3804059690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb414a920 con 0x7fafb4155460 2024-09-21T06:35:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafabe00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7faf9c034040 tx=0x7faf9c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafaaa00640 1 -- 172.21.15.37:0/3804059690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c03e030 con 0x7fafb4155460 2024-09-21T06:35:43.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafaaa00640 1 -- 172.21.15.37:0/3804059690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c003ae0 con 0x7fafb4155460 2024-09-21T06:35:43.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafaaa00640 1 -- 172.21.15.37:0/3804059690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c035040 con 0x7fafb4155460 2024-09-21T06:35:43.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb415db70 con 0x7fafb4155460 2024-09-21T06:35:43.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.665+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb40cc230 con 0x7fafb4155460 2024-09-21T06:35:43.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.668+0000 7fafaaa00640 1 -- 172.21.15.37:0/3804059690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf9c041030 con 0x7fafb4155460 2024-09-21T06:35:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.669+0000 7fafaaa00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 0x7faf9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.669+0000 7fafaaa00640 1 -- 172.21.15.37:0/3804059690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7faf9c0bb4d0 con 0x7fafb4155460 2024-09-21T06:35:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.669+0000 7fafc088e740 1 Processor -- start 2024-09-21T06:35:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.669+0000 7fafc088e740 1 -- start start 2024-09-21T06:35:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.669+0000 7fafb2a00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 0x7faf9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.670+0000 7fafb2a00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 0x7faf9007b0a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009b40 tx=0x7fafa4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05189400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc05189940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bc050bf1c0 con 0x55bc04db8390 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bc050d2870 con 0x55bc04d8f4d0 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55bc050c1ca0 con 0x55bc04d976b0 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37842/0 (socket says 172.21.15.37:37842) 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 -- 172.21.15.37:0/345931822 learned_addr learned my addr 172.21.15.37:0/345931822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb3400640 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc05189940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafabe00640 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05189400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 -- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 msgr2=0x55bc05189940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc05189940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 -- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 msgr2=0x55bc05189400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05189400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.671+0000 7fafb2a00640 1 -- 172.21.15.37:0/345931822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc04e9a7d0 con 0x55bc04db8390 2024-09-21T06:35:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.672+0000 7fafb2a00640 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fafa4011f60 tx=0x7fafa4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cc39db4ca684a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.672+0000 7fafa3400640 1 -- 172.21.15.37:0/345931822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004220 con 0x55bc04db8390 2024-09-21T06:35:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.672+0000 7fafa3400640 1 -- 172.21.15.37:0/345931822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa40035f0 con 0x55bc04db8390 2024-09-21T06:35:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.672+0000 7fafa3400640 1 -- 172.21.15.37:0/345931822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40039e0 con 0x55bc04db8390 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 -- 172.21.15.37:0/345931822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 msgr2=0x55bc051a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fafa4011f60 tx=0x7fafa4014000 comp rx=0 tx=0).stop 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 -- 172.21.15.37:0/345931822 shutdown_connections 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc05189940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05189400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 --2- 172.21.15.37:0/345931822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04db8390 0x55bc051a6730 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 -- 172.21.15.37:0/345931822 >> 172.21.15.37:0/345931822 conn(0x55bc04dac4e0 msgr2=0x55bc0515f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 -- 172.21.15.37:0/345931822 shutdown_connections 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.673+0000 7fafc088e740 1 -- 172.21.15.37:0/345931822 wait complete. 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.674+0000 7fafc088e740 1 -- start start 2024-09-21T06:35:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.674+0000 7fafc088e740 1 Processor -- start 2024-09-21T06:35:43.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc051b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04db8390 0x55bc051a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bc05068410 con 0x55bc04d8f4d0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bc050c5ab0 con 0x55bc04db8390 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55bc04e32740 con 0x55bc04d976b0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafc088e740 10 client.0 _ll_register_callbacks cb 0x55bc05045360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37848/0 (socket says 172.21.15.37:37848) 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 -- 172.21.15.37:0/4173193891 learned_addr learned my addr 172.21.15.37:0/4173193891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafabe00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc051b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb3400640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04db8390 0x55bc051a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 -- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 msgr2=0x55bc051b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc051b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 -- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04db8390 msgr2=0x55bc051a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04db8390 0x55bc051a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.676+0000 7fafb2a00640 1 -- 172.21.15.37:0/4173193891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc04e23970 con 0x55bc04d8f4d0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7fafb2a00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009920 tx=0x7fafa4020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad04d2f1fb598e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7faf97e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4037060 con 0x55bc04d8f4d0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa40035f0 con 0x55bc04d8f4d0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fafa4032470 con 0x55bc04d8f4d0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fafa403d050 con 0x55bc04d8f4d0 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7fafc088e740 10 client.8865 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:43.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.677+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55bc04e016a0 con 0x55bc04d8f4d0 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa40035f0 con 0x55bc04d8f4d0 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fafa4040020 con 0x55bc04d8f4d0 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 10 client.8865 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 10 client.8865 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 20 client.8865 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d76f60 0x55bc0519af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 4 client.8865 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc050d47c0 con 0x55bc04d76f60 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.678+0000 7fafabe00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d76f60 0x55bc0519af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.679+0000 7fafabe00640 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d76f60 0x55bc0519af80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c02ccc1079757c4 server_cookie=3f215677cee32abe in_seq=0 out_seq=0 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.679+0000 7faf97e00640 10 client.8865 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.679+0000 7faf97e00640 1 -- 172.21.15.37:0/4173193891 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc050d47c0 con 0x55bc04d76f60 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.679+0000 7faf97e00640 10 client.8865 handle_command_reply: tid=0 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 client.8865 shutdown 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 msgr2=0x55bc051b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009920 tx=0x7fafa4020ad0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 shutdown_connections 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d76f60 0x55bc0519af80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04db8390 0x55bc051a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04d976b0 0x55bc051b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7faf97e00640 0 client.8865 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:43.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 --2- 172.21.15.37:0/4173193891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b7600 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 >> 172.21.15.37:0/4173193891 conn(0x55bc04dac4e0 msgr2=0x55bc05178bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 shutdown_connections 2024-09-21T06:35:43.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 1 -- 172.21.15.37:0/4173193891 wait complete. 2024-09-21T06:35:43.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.680+0000 7fafc088e740 20 client.8865 trim_cache size 0 max 16384 2024-09-21T06:35:43.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.699+0000 7fafc088e740 1 Processor -- start 2024-09-21T06:35:43.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.699+0000 7fafc088e740 1 -- start start 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc051b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 0x55bc051a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bc0515aeb0 con 0x55bc04d8f4d0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4202540 con 0x55bc04d976b0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.700+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fafb42016e0 con 0x55bc04dac4e0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37856/0 (socket says 172.21.15.37:37856) 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 -- 172.21.15.37:0/2844521223 learned_addr learned my addr 172.21.15.37:0/2844521223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafb2a00640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc051b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafb3400640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 0x55bc051a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 -- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 msgr2=0x55bc051b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc051b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 -- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 msgr2=0x55bc051a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 0x55bc051a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafabe00640 1 -- 172.21.15.37:0/2844521223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc050c7f50 con 0x55bc04d8f4d0 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.701+0000 7fafb3400640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 0x55bc051a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.702+0000 7fafabe00640 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7faf9c080300 tx=0x7faf9c0b96c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94590812711dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.702+0000 7fafa2a00640 1 -- 172.21.15.37:0/2844521223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c048040 con 0x55bc04d8f4d0 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.702+0000 7fafa2a00640 1 -- 172.21.15.37:0/2844521223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c083090 con 0x55bc04d8f4d0 2024-09-21T06:35:43.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.702+0000 7fafa2a00640 1 -- 172.21.15.37:0/2844521223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c08ae60 con 0x55bc04d8f4d0 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 -- 172.21.15.37:0/2844521223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 msgr2=0x55bc051b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7faf9c080300 tx=0x7faf9c0b96c0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 -- 172.21.15.37:0/2844521223 shutdown_connections 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d8f4d0 0x55bc051b5e80 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d976b0 0x55bc051a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 --2- 172.21.15.37:0/2844521223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc051b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 -- 172.21.15.37:0/2844521223 >> 172.21.15.37:0/2844521223 conn(0x55bc051a5070 msgr2=0x55bc051a4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 -- 172.21.15.37:0/2844521223 shutdown_connections 2024-09-21T06:35:43.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.703+0000 7fafc088e740 1 -- 172.21.15.37:0/2844521223 wait complete. 2024-09-21T06:35:43.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.704+0000 7fafc088e740 1 -- start start 2024-09-21T06:35:43.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.704+0000 7fafc088e740 1 Processor -- start 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.705+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05177170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.705+0000 7fafc088e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.705+0000 7fafc088e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc052244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafc088e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40ce220 con 0x55bc04d976b0 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4201e10 con 0x55bc04d8f4d0 2024-09-21T06:35:43.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafc088e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x55bc0515aeb0 con 0x55bc04dac4e0 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafc088e740 10 client.0 _ll_register_callbacks cb 0x7fafa4011dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37864/0 (socket says 172.21.15.37:37864) 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 -- 172.21.15.37:0/458958380 learned_addr learned my addr 172.21.15.37:0/458958380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafb3400640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc052244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafb2a00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05177170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 -- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 msgr2=0x55bc052244b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc052244b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 -- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 msgr2=0x55bc05177170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05177170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.706+0000 7fafabe00640 1 -- 172.21.15.37:0/458958380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc051867c0 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafabe00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7faf9c08a240 tx=0x7faf9c0b9a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992a7063fdca118f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c048030 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7faf9c09ee00 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9c083090 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7faf9c08ae60 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 10 client.8871 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55bc051c6780 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c083090 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7faf9c0a52d0 con 0x55bc04d976b0 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 10 client.8871 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 10 client.8871 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-21T06:35:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 20 client.8871 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.707+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d4c3a0 0x55bc052249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.708+0000 7fafc088e740 4 client.8871 mds_command: new command op to 8172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.708+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 --> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc050d0270 con 0x55bc04d4c3a0 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.708+0000 7fafb2a00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d4c3a0 0x55bc052249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.708+0000 7fafb2a00640 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d4c3a0 0x55bc052249f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84085526edfda39a server_cookie=898e305105d5c30e in_seq=0 out_seq=0 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.708+0000 7faf97e00640 10 client.8871 ms_handle_connect on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7faf97e00640 1 -- 172.21.15.37:0/458958380 <== mds.1 v2:172.21.15.37:6832/1950391294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55bc050d0270 con 0x55bc04d4c3a0 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7faf97e00640 10 client.8871 handle_command_reply: tid=0 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 client.8871 shutdown 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 msgr2=0x55bc051776b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7faf9c08a240 tx=0x7faf9c0b9a80 comp rx=0 tx=0).stop 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 shutdown_connections 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:6832/1950391294,v1:172.21.15.37:6833/1950391294] conn(0x55bc04d4c3a0 0x55bc052249f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc04d976b0 0x55bc051776b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bc04dac4e0 0x55bc052244b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 --2- 172.21.15.37:0/458958380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc04d8f4d0 0x55bc05177170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7faf97e00640 0 client.8871 ms_handle_reset on v2:172.21.15.37:6832/1950391294 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 >> 172.21.15.37:0/458958380 conn(0x55bc051a5070 msgr2=0x55bc051c72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 shutdown_connections 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 1 -- 172.21.15.37:0/458958380 wait complete. 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.709+0000 7fafc088e740 20 client.8871 trim_cache size 0 max 16384 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:43.715 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 msgr2=0x7faf9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 0x7faf9007b0a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009b40 tx=0x7fafa4005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 msgr2=0x7fafb4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7faf9c034040 tx=0x7faf9c0035f0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 shutdown_connections 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7faf90078b90 0x7faf9007b0a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4155460 0x7fafb4155c10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafb4162890 0x7fafb4156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 --2- 172.21.15.37:0/3804059690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafb40d2760 0x7fafb415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 >> 172.21.15.37:0/3804059690 conn(0x7fafb40d4360 msgr2=0x7fafb415e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 shutdown_connections 2024-09-21T06:35:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.710+0000 7fafb3e00640 1 -- 172.21.15.37:0/3804059690 wait complete. 2024-09-21T06:35:43.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:43.735 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:43.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 -- 172.21.15.37:0/4214622676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e2806e370 msgr2=0x7f1e280b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4214622676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e2806e370 0x7f1e280b4f90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1e280cd790 tx=0x7f1e1c02a5f0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 -- 172.21.15.37:0/4214622676 shutdown_connections 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4214622676 >> v2:172.21.15.150:3300/0 conn(0x7f1e280ccfc0 0x7f1e280cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4214622676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e2806e370 0x7f1e280b4f90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.961+0000 7f1e2e200640 1 -- 172.21.15.37:0/4214622676 >> 172.21.15.37:0/4214622676 conn(0x7f1e2806f920 msgr2=0x7f1e280b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.962+0000 7f1e2e200640 1 -- 172.21.15.37:0/4214622676 shutdown_connections 2024-09-21T06:35:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.962+0000 7f1e2e200640 1 -- 172.21.15.37:0/4214622676 wait complete. 2024-09-21T06:35:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.964+0000 7f1e2e200640 1 Processor -- start 2024-09-21T06:35:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.964+0000 7f1e2e200640 1 -- start start 2024-09-21T06:35:43.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.965+0000 7f1e2e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2806e370 0x7f1e28159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.965+0000 7f1e2e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.965+0000 7f1e2e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e280af2c0 0x7f1e281f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.965+0000 7f1e2e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2806d560 con 0x7f1e28156c60 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.965+0000 7f1e2e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2806d3e0 con 0x7f1e2806e370 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e2e200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2806d6e0 con 0x7f1e280af2c0 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e2ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2806e370 0x7f1e28159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35576/0 (socket says 172.21.15.37:35576) 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 -- 172.21.15.37:0/4165662974 learned_addr learned my addr 172.21.15.37:0/4165662974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e2d800640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e280af2c0 0x7f1e281f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 -- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e280af2c0 msgr2=0x7f1e281f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e280af2c0 0x7f1e281f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 -- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2806e370 msgr2=0x7f1e28159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2806e370 0x7f1e28159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.966+0000 7f1e27e00640 1 -- 172.21.15.37:0/4165662974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2815a250 con 0x7f1e28156c60 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e27e00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400c340 tx=0x7f1e1400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e24c00640 1 -- 172.21.15.37:0/4165662974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14017070 con 0x7f1e28156c60 2024-09-21T06:35:43.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e24c00640 1 -- 172.21.15.37:0/4165662974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e140053c0 con 0x7f1e28156c60 2024-09-21T06:35:43.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2815f6d0 con 0x7f1e28156c60 2024-09-21T06:35:43.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e24c00640 1 -- 172.21.15.37:0/4165662974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14003850 con 0x7f1e28156c60 2024-09-21T06:35:43.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.967+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281f8fc0 con 0x7f1e28156c60 2024-09-21T06:35:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.971+0000 7f1e24c00640 1 -- 172.21.15.37:0/4165662974 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e1401a070 con 0x7f1e28156c60 2024-09-21T06:35:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.972+0000 7f1e24c00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 0x7f1e0807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.972+0000 7f1e36a15740 1 Processor -- start 2024-09-21T06:35:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.972+0000 7f1e24c00640 1 -- 172.21.15.37:0/4165662974 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f1e1409a1e0 con 0x7f1e28156c60 2024-09-21T06:35:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.972+0000 7f1e36a15740 1 -- start start 2024-09-21T06:35:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.972+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 0x7f1e0807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 0x7f1e0807b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c033000 tx=0x7f1e1c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b336f21c0 con 0x557b333ca6b0 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557b33705870 con 0x557b333c24d0 2024-09-21T06:35:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.973+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557b336f4ca0 con 0x557b333eb390 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2ce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2ce00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:53778/0 (socket says 172.21.15.37:53778) 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2ce00640 1 -- 172.21.15.37:0/2168150271 learned_addr learned my addr 172.21.15.37:0/2168150271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e27e00640 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 -- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 msgr2=0x557b337d9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 -- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 msgr2=0x557b337bc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337bc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.974+0000 7f1e2d800640 1 -- 172.21.15.37:0/2168150271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b334cd7d0 con 0x557b333ca6b0 2024-09-21T06:35:43.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.975+0000 7f1e2d800640 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337bc940 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009ec0 tx=0x7f1e20010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e22149e869ec59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.975+0000 7f1e19600640 1 -- 172.21.15.37:0/2168150271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20016070 con 0x557b333ca6b0 2024-09-21T06:35:43.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.975+0000 7f1e19600640 1 -- 172.21.15.37:0/2168150271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e20004e10 con 0x557b333ca6b0 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.975+0000 7f1e19600640 1 -- 172.21.15.37:0/2168150271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20005260 con 0x557b333ca6b0 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.975+0000 7f1e36a15740 1 -- 172.21.15.37:0/2168150271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 msgr2=0x557b337bc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337bc940 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009ec0 tx=0x7f1e20010740 comp rx=0 tx=0).stop 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 -- 172.21.15.37:0/2168150271 shutdown_connections 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337bc940 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337bc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2168150271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 -- 172.21.15.37:0/2168150271 >> 172.21.15.37:0/2168150271 conn(0x557b333df4e0 msgr2=0x557b33792620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 -- 172.21.15.37:0/2168150271 shutdown_connections 2024-09-21T06:35:43.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.976+0000 7f1e36a15740 1 -- 172.21.15.37:0/2168150271 wait complete. 2024-09-21T06:35:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.977+0000 7f1e36a15740 1 -- start start 2024-09-21T06:35:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.977+0000 7f1e36a15740 1 Processor -- start 2024-09-21T06:35:43.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b3369b410 con 0x557b333ca6b0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557b336f8ab0 con 0x557b333c24d0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.978+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557b33465740 con 0x557b333eb390 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e36a15740 10 client.0 _ll_register_callbacks cb 0x557b33678360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35590/0 (socket says 172.21.15.37:35590) 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 -- 172.21.15.37:0/3285972939 learned_addr learned my addr 172.21.15.37:0/3285972939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e2d800640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 -- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 msgr2=0x557b337d6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 -- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 msgr2=0x557b337a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 -- 172.21.15.37:0/3285972939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b33456970 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e27e00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400cd00 tx=0x7f1e14061c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43940cc5cb560d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.979+0000 7f1e0e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14088020 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f1e1405d710 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e20004e10 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f1e140667f0 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 10 client.8883 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x557b334346a0 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e20004e10 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f1e1405d910 con 0x557b333ca6b0 2024-09-21T06:35:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 10 client.8883 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 10 client.8883 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 20 client.8883 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b333a9f60 0x557b337e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 4 client.8883 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.980+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b337077c0 con 0x557b333a9f60 2024-09-21T06:35:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.981+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b333a9f60 0x557b337e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.981+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b333a9f60 0x557b337e9fc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee2e8d4e2bc788ff server_cookie=7994fe12e316ef73 in_seq=0 out_seq=0 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.981+0000 7f1e0e000640 10 client.8883 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.982+0000 7f1e0e000640 1 -- 172.21.15.37:0/3285972939 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b337077c0 con 0x557b333a9f60 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.982+0000 7f1e0e000640 10 client.8883 handle_command_reply: tid=0 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 client.8883 shutdown 2024-09-21T06:35:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 msgr2=0x557b337d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400cd00 tx=0x7f1e14061c50 comp rx=0 tx=0).stop 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 shutdown_connections 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b333a9f60 0x557b337e9fc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337d17d0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333eb390 0x557b337d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3285972939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b337a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 >> 172.21.15.37:0/3285972939 conn(0x557b333df4e0 msgr2=0x557b337bc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e0e000640 0 client.8883 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 shutdown_connections 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 1 -- 172.21.15.37:0/3285972939 wait complete. 2024-09-21T06:35:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:43.983+0000 7f1e36a15740 20 client.8883 trim_cache size 0 max 16384 2024-09-21T06:35:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.002+0000 7f1e36a15740 1 Processor -- start 2024-09-21T06:35:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.002+0000 7f1e36a15740 1 -- start start 2024-09-21T06:35:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333c24d0 0x557b337facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 0x557b337d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e282027d0 con 0x557b333ca6b0 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28201970 con 0x557b333eb390 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.003+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f1e282020a0 con 0x557b333c24d0 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35598/0 (socket says 172.21.15.37:35598) 2024-09-21T06:35:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 -- 172.21.15.37:0/2280062508 learned_addr learned my addr 172.21.15.37:0/2280062508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 -- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333c24d0 msgr2=0x557b337facd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333c24d0 0x557b337facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 -- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 msgr2=0x557b337d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e2d800640 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 0x557b337d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 0x557b337d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 -- 172.21.15.37:0/2280062508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3378d020 con 0x557b333ca6b0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e2d800640 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 0x557b337d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.004+0000 7f1e27e00640 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f1e14061ac0 tx=0x7f1e1405dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ad9629da0651f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e18c00640 1 -- 172.21.15.37:0/2280062508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14088020 con 0x557b333ca6b0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e18c00640 1 -- 172.21.15.37:0/2280062508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e1406b6a0 con 0x557b333ca6b0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e18c00640 1 -- 172.21.15.37:0/2280062508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14061230 con 0x557b333ca6b0 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 -- 172.21.15.37:0/2280062508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 msgr2=0x557b337a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f1e14061ac0 tx=0x7f1e1405dcf0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 -- 172.21.15.37:0/2280062508 shutdown_connections 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333ca6b0 0x557b337a9ea0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333eb390 0x557b337d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 --2- 172.21.15.37:0/2280062508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333c24d0 0x557b337facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 -- 172.21.15.37:0/2280062508 >> 172.21.15.37:0/2280062508 conn(0x557b333df4e0 msgr2=0x557b337d9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.005+0000 7f1e36a15740 1 -- 172.21.15.37:0/2280062508 shutdown_connections 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.006+0000 7f1e36a15740 1 -- 172.21.15.37:0/2280062508 wait complete. 2024-09-21T06:35:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.006+0000 7f1e36a15740 1 -- start start 2024-09-21T06:35:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.007+0000 7f1e36a15740 1 Processor -- start 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b33856a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333ca6b0 0x557b33793300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b337a9670 con 0x557b333eb390 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28200a10 con 0x557b333c24d0 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x557b336f4eb0 con 0x557b333ca6b0 2024-09-21T06:35:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e36a15740 10 client.0 _ll_register_callbacks cb 0x7f1e14061710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e2d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35604/0 (socket says 172.21.15.37:35604) 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e2d800640 1 -- 172.21.15.37:0/3014382034 learned_addr learned my addr 172.21.15.37:0/3014382034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.008+0000 7f1e27e00640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333ca6b0 0x557b33793300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b33856a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 -- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333ca6b0 msgr2=0x557b33793300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333ca6b0 0x557b33793300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 -- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 msgr2=0x557b33856a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b33856a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 -- 172.21.15.37:0/3014382034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b337acb60 con 0x557b333eb390 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e2d800640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009e90 tx=0x7f1e20009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5e604c7e9060f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e0e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:44.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.009+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20016040 con 0x557b333eb390 2024-09-21T06:35:44.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f1e20003970 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b337d9de0 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f1e2002f050 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 10 client.8889 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x557b337d5230 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557b337d9de0 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7f1e20032020 con 0x557b333eb390 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 10 client.8889 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 10 client.8889 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 20 client.8889 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b337a6b30 0x557b3385ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 4 client.8889 mds_command: new command op to 7079 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.010+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 --> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b3378e580 con 0x557b337a6b30 2024-09-21T06:35:44.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.011+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b337a6b30 0x557b3385ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.011+0000 7f1e2ce00640 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b337a6b30 0x557b3385ac60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8b212eb032b489 server_cookie=373e87fa77a939e0 in_seq=0 out_seq=0 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.011+0000 7f1e0e000640 10 client.8889 ms_handle_connect on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e0e000640 1 -- 172.21.15.37:0/3014382034 <== mds.2 v2:172.21.15.150:6835/1287330873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x557b3378e580 con 0x557b337a6b30 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e0e000640 10 client.8889 handle_command_reply: tid=0 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 client.8889 shutdown 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 msgr2=0x557b33793840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009e90 tx=0x7f1e20009ab0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 shutdown_connections 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:6835/1287330873,v1:172.21.15.150:6837/1287330873] conn(0x557b337a6b30 0x557b3385ac60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b333eb390 0x557b33793840 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557b333ca6b0 0x557b33793300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 --2- 172.21.15.37:0/3014382034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b333c24d0 0x557b33856a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e0e000640 0 client.8889 ms_handle_reset on v2:172.21.15.150:6835/1287330873 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 >> 172.21.15.37:0/3014382034 conn(0x557b333df4e0 msgr2=0x557b337d9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 shutdown_connections 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 1 -- 172.21.15.37:0/3014382034 wait complete. 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.012+0000 7f1e36a15740 20 client.8889 trim_cache size 0 max 16384 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:44.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:44.018 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:44.018 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:44.018 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:44.018 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 msgr2=0x7f1e0807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 0x7f1e0807b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c033000 tx=0x7f1e1c004730 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 msgr2=0x7f1e2815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400c340 tx=0x7f1e1400c800 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 shutdown_connections 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1e08078bc0 0x7f1e0807b0d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28156c60 0x7f1e2815dc20 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e280af2c0 0x7f1e281f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 --2- 172.21.15.37:0/4165662974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2806e370 0x7f1e28159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 >> 172.21.15.37:0/4165662974 conn(0x7f1e2806f920 msgr2=0x7f1e2815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 shutdown_connections 2024-09-21T06:35:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.013+0000 7f1e2e200640 1 -- 172.21.15.37:0/4165662974 wait complete. 2024-09-21T06:35:44.037 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:44.037 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T06:35:44.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 -- 172.21.15.37:0/2428638059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc178162d20 msgr2=0x7fc17815e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 --2- 172.21.15.37:0/2428638059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc178162d20 0x7fc17815e520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009830 tx=0x7fc16c02a4a0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 -- 172.21.15.37:0/2428638059 shutdown_connections 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 --2- 172.21.15.37:0/2428638059 >> v2:172.21.15.150:3300/0 conn(0x7fc178162890 0x7fc1780d3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 --2- 172.21.15.37:0/2428638059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc178162d20 0x7fc17815e520 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 -- 172.21.15.37:0/2428638059 >> 172.21.15.37:0/2428638059 conn(0x7fc1780d0d40 msgr2=0x7fc1780d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 -- 172.21.15.37:0/2428638059 shutdown_connections 2024-09-21T06:35:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.264+0000 7fc17e800640 1 -- 172.21.15.37:0/2428638059 wait complete. 2024-09-21T06:35:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.267+0000 7fc17e800640 1 Processor -- start 2024-09-21T06:35:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.267+0000 7fc17e800640 1 -- start start 2024-09-21T06:35:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e020 0x7fc1780ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc178162d20 0x7fc1780b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780b7af0 con 0x7fc1781f60f0 2024-09-21T06:35:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780b7970 con 0x7fc178162d20 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17e800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780b7c70 con 0x7fc17806e020 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.268+0000 7fc17de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35612/0 (socket says 172.21.15.37:35612) 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 -- 172.21.15.37:0/1651803902 learned_addr learned my addr 172.21.15.37:0/1651803902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17d400640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e020 0x7fc1780ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17ca00640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc178162d20 0x7fc1780b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 -- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e020 msgr2=0x7fc1780ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e020 0x7fc1780ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 -- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc178162d20 msgr2=0x7fc1780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc178162d20 0x7fc1780b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 -- 172.21.15.37:0/1651803902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780aae80 con 0x7fc1781f60f0 2024-09-21T06:35:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.269+0000 7fc17de00640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ba10 tx=0x7fc17000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.270+0000 7fc175600640 1 -- 172.21.15.37:0/1651803902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170019070 con 0x7fc1781f60f0 2024-09-21T06:35:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.270+0000 7fc175600640 1 -- 172.21.15.37:0/1651803902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc17000cc50 con 0x7fc1781f60f0 2024-09-21T06:35:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.270+0000 7fc175600640 1 -- 172.21.15.37:0/1651803902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700146a0 con 0x7fc1781f60f0 2024-09-21T06:35:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.270+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1780ab2f0 con 0x7fc1781f60f0 2024-09-21T06:35:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.270+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1780ab8b0 con 0x7fc1781f60f0 2024-09-21T06:35:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.272+0000 7fc175600640 1 -- 172.21.15.37:0/1651803902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc17001c070 con 0x7fc1781f60f0 2024-09-21T06:35:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.273+0000 7fc175600640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.273+0000 7fc175600640 1 -- 172.21.15.37:0/1651803902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fc17009aca0 con 0x7fc1781f60f0 2024-09-21T06:35:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.274+0000 7fc17d400640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.274+0000 7fc186ffb740 1 Processor -- start 2024-09-21T06:35:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.274+0000 7fc186ffb740 1 -- start start 2024-09-21T06:35:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.274+0000 7fc17d400640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc16c034000 tx=0x7fc16c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x560828379400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828379940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608282af1c0 con 0x560827fa8390 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5608282c2870 con 0x560827f876b0 2024-09-21T06:35:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x5608282b1ca0 con 0x560827f7f4d0 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.275+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35622/0 (socket says 172.21.15.37:35622) 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 -- 172.21.15.37:0/2862993798 learned_addr learned my addr 172.21.15.37:0/2862993798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17de00640 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828379940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17ca00640 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x560828379400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 -- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 msgr2=0x560828379400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x560828379400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 -- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 msgr2=0x560828379940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828379940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 -- 172.21.15.37:0/2862993798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082808a7d0 con 0x560827fa8390 2024-09-21T06:35:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.276+0000 7fc17d400640 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc16c03c990 tx=0x7fc16c03e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d0b826449357f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc169c00640 1 -- 172.21.15.37:0/2862993798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c045480 con 0x560827fa8390 2024-09-21T06:35:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc169c00640 1 -- 172.21.15.37:0/2862993798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc16c045620 con 0x560827fa8390 2024-09-21T06:35:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc169c00640 1 -- 172.21.15.37:0/2862993798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0459d0 con 0x560827fa8390 2024-09-21T06:35:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 -- 172.21.15.37:0/2862993798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 msgr2=0x560828396730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc16c03c990 tx=0x7fc16c03e040 comp rx=0 tx=0).stop 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 -- 172.21.15.37:0/2862993798 shutdown_connections 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828379940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x560828379400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 --2- 172.21.15.37:0/2862993798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x560828396730 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.277+0000 7fc186ffb740 1 -- 172.21.15.37:0/2862993798 >> 172.21.15.37:0/2862993798 conn(0x560827f9c4e0 msgr2=0x56082834f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.278+0000 7fc186ffb740 1 -- 172.21.15.37:0/2862993798 shutdown_connections 2024-09-21T06:35:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.278+0000 7fc186ffb740 1 -- 172.21.15.37:0/2862993798 wait complete. 2024-09-21T06:35:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.279+0000 7fc186ffb740 1 -- start start 2024-09-21T06:35:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.279+0000 7fc186ffb740 1 Processor -- start 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828372ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x5608283676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560828258410 con 0x560827f7f4d0 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5608282b5ab0 con 0x560827f876b0 2024-09-21T06:35:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560828022740 con 0x560827fa8390 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.280+0000 7fc186ffb740 10 client.0 _ll_register_callbacks cb 0x560828235360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35630/0 (socket says 172.21.15.37:35630) 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 -- 172.21.15.37:0/4060751190 learned_addr learned my addr 172.21.15.37:0/4060751190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17de00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x5608283676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17ca00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828372ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 -- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 msgr2=0x5608283676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x5608283676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 -- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 msgr2=0x560828372ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828372ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 -- 172.21.15.37:0/4060751190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560828013970 con 0x560827f7f4d0 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17de00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x5608283676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17ca00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828372ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc17d400640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0095f0 tx=0x7fc16c045520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58daaccf95e1445 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.281+0000 7fc15e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c057070 con 0x560827f7f4d0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16c045620 con 0x560827f7f4d0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fc16c045da0 con 0x560827f7f4d0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fc16c062050 con 0x560827f7f4d0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 10 client.8901 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x560827ff16a0 con 0x560827f7f4d0 2024-09-21T06:35:44.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc16c045620 con 0x560827f7f4d0 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fc16c065020 con 0x560827f7f4d0 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 10 client.8901 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 10 client.8901 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.282+0000 7fc186ffb740 20 client.8901 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x560827f66f60 0x560828371730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc186ffb740 4 client.8901 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608282c47c0 con 0x560827f66f60 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc17ca00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x560827f66f60 0x560828371730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc17ca00640 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x560827f66f60 0x560828371730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad065d14c57b2418 server_cookie=df8915a4c3b7267a in_seq=0 out_seq=0 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.283+0000 7fc15e000640 10 client.8901 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc15e000640 1 -- 172.21.15.37:0/4060751190 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608282c47c0 con 0x560827f66f60 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc15e000640 10 client.8901 handle_command_reply: tid=0 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 client.8901 shutdown 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 msgr2=0x5608283a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0095f0 tx=0x7fc16c045520 comp rx=0 tx=0).stop 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 shutdown_connections 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x560827f66f60 0x560828371730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x5608283676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828372ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 --2- 172.21.15.37:0/4060751190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283a74a0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc15e000640 0 client.8901 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.284+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 >> 172.21.15.37:0/4060751190 conn(0x560827f9c4e0 msgr2=0x56082834f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.285+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 shutdown_connections 2024-09-21T06:35:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.285+0000 7fc186ffb740 1 -- 172.21.15.37:0/4060751190 wait complete. 2024-09-21T06:35:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.285+0000 7fc186ffb740 20 client.8901 trim_cache size 0 max 16384 2024-09-21T06:35:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.304+0000 7fc186ffb740 1 Processor -- start 2024-09-21T06:35:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.304+0000 7fc186ffb740 1 -- start start 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x5608283949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x5608283a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc178202830 con 0x560827fa8390 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1782019d0 con 0x560827f876b0 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.305+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc178202100 con 0x560827f7f4d0 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35634/0 (socket says 172.21.15.37:35634) 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 -- 172.21.15.37:0/693573057 learned_addr learned my addr 172.21.15.37:0/693573057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17ca00640 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x5608283a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17d400640 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x5608283949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 -- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 msgr2=0x5608283949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x5608283949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 -- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 msgr2=0x5608283a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x5608283a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 -- 172.21.15.37:0/693573057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082834c180 con 0x560827fa8390 2024-09-21T06:35:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc17de00640 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc170098ed0 tx=0x7fc17005ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab2e89144bb1231 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc169200640 1 -- 172.21.15.37:0/693573057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170088070 con 0x560827fa8390 2024-09-21T06:35:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc169200640 1 -- 172.21.15.37:0/693573057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc170067220 con 0x560827fa8390 2024-09-21T06:35:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.306+0000 7fc169200640 1 -- 172.21.15.37:0/693573057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17006bed0 con 0x560827fa8390 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 -- 172.21.15.37:0/693573057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 msgr2=0x5608283a73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc170098ed0 tx=0x7fc17005ef40 comp rx=0 tx=0).stop 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 -- 172.21.15.37:0/693573057 shutdown_connections 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827fa8390 0x5608283a73b0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x5608283a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 --2- 172.21.15.37:0/693573057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827f7f4d0 0x5608283949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 -- 172.21.15.37:0/693573057 >> 172.21.15.37:0/693573057 conn(0x560827f9c4e0 msgr2=0x56082838ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 -- 172.21.15.37:0/693573057 shutdown_connections 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.307+0000 7fc186ffb740 1 -- 172.21.15.37:0/693573057 wait complete. 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.308+0000 7fc186ffb740 1 -- start start 2024-09-21T06:35:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.308+0000 7fc186ffb740 1 Processor -- start 2024-09-21T06:35:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828370f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x560828413070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc178200a10 con 0x560827f7f4d0 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560828413f30 con 0x560827f876b0 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x560828349fe0 con 0x560827fa8390 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc186ffb740 10 client.0 _ll_register_callbacks cb 0x7fc1600036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc17d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35642/0 (socket says 172.21.15.37:35642) 2024-09-21T06:35:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.310+0000 7fc17d400640 1 -- 172.21.15.37:0/3627246996 learned_addr learned my addr 172.21.15.37:0/3627246996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17ca00640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828370f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17de00640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x560828413070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 -- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 msgr2=0x560828413070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x560828413070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 -- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 msgr2=0x560828370f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828370f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 -- 172.21.15.37:0/3627246996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608283b86f0 con 0x560827f7f4d0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17de00640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x560828413070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc17d400640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009330 tx=0x7fc16c045830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d7d927ce85ffb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc15e000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c057070 con 0x560827f7f4d0 2024-09-21T06:35:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fc16c05c8f0 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.311+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc170067220 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fc16c06d050 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 10 client.8907 fetch_fsmap learned FSMap version 118 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x56082834a2c0 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc170067220 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 118) ==== 3558+0+0 (secure 0 0 0) 0x7fc16c070020 con 0x560827f7f4d0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 10 client.8907 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 10 client.8907 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 20 client.8907 populate_metadata read hostname 'smithi037' 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5608284135b0 0x5608284172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 4 client.8907 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 --> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608282c0270 con 0x5608284135b0 2024-09-21T06:35:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.312+0000 7fc17ca00640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5608284135b0 0x5608284172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc17ca00640 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5608284135b0 0x5608284172a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab54086786be49d server_cookie=7b2f292f3db6fcd5 in_seq=0 out_seq=0 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc15e000640 10 client.8907 ms_handle_connect on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc15e000640 1 -- 172.21.15.37:0/3627246996 <== mds.0 v2:172.21.15.37:6834/912339030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x5608282c0270 con 0x5608284135b0 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc15e000640 10 client.8907 handle_command_reply: tid=0 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 client.8907 shutdown 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 msgr2=0x5608283709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009330 tx=0x7fc16c045830 comp rx=0 tx=0).stop 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 shutdown_connections 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:6834/912339030,v1:172.21.15.37:6835/912339030] conn(0x5608284135b0 0x5608284172a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560827fa8390 0x560828413070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560827f876b0 0x560828370f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 --2- 172.21.15.37:0/3627246996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560827f7f4d0 0x5608283709f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 >> 172.21.15.37:0/3627246996 conn(0x560827f9c4e0 msgr2=0x560828373520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.313+0000 7fc15e000640 0 client.8907 ms_handle_reset on v2:172.21.15.37:6834/912339030 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 shutdown_connections 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc186ffb740 1 -- 172.21.15.37:0/3627246996 wait complete. 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc186ffb740 20 client.8907 trim_cache size 0 max 16384 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-21T06:35:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7", 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_stats": { 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "078614e0-e7c7-4b57-bec8-f9e3ed9ac2d7": { 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "start_time": "2024-09-21T06:35:31.440721+0000", 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/d1/d2/d3", 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_started": 0, 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_passed": 0, 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_failed": 0, 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "uninline_skipped": 0 2024-09-21T06:35:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-21T06:35:44.323 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:35:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 msgr2=0x7fc15807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc16c034000 tx=0x7fc16c004730 comp rx=0 tx=0).stop 2024-09-21T06:35:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 msgr2=0x7fc1781f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.314+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ba10 tx=0x7fc17000bed0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 shutdown_connections 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc158078b90 0x7fc15807b0a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1781f60f0 0x7fc1781f6500 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc178162d20 0x7fc1780b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 --2- 172.21.15.37:0/1651803902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17806e020 0x7fc1780ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 >> 172.21.15.37:0/1651803902 conn(0x7fc1780d0d40 msgr2=0x7fc1780b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 shutdown_connections 2024-09-21T06:35:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.315+0000 7fc17e800640 1 -- 172.21.15.37:0/1651803902 wait complete. 2024-09-21T06:35:44.338 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:35:44.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 -- 172.21.15.37:0/2741426068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c1627f0 msgr2=0x7f577c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 --2- 172.21.15.37:0/2741426068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c1627f0 0x7f577c06d2b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5764009830 tx=0x7f576402a510 comp rx=0 tx=0).stop 2024-09-21T06:35:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 -- 172.21.15.37:0/2741426068 shutdown_connections 2024-09-21T06:35:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 --2- 172.21.15.37:0/2741426068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c1627f0 0x7f577c06d2b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 --2- 172.21.15.37:0/2741426068 >> v2:172.21.15.150:3300/0 conn(0x7f577c0b9e40 0x7f577c1622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 -- 172.21.15.37:0/2741426068 >> 172.21.15.37:0/2741426068 conn(0x7f577c0d1930 msgr2=0x7f577c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 -- 172.21.15.37:0/2741426068 shutdown_connections 2024-09-21T06:35:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.564+0000 7f577b200640 1 -- 172.21.15.37:0/2741426068 wait complete. 2024-09-21T06:35:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.567+0000 7f577b200640 1 Processor -- start 2024-09-21T06:35:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.567+0000 7f577b200640 1 -- start start 2024-09-21T06:35:44.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c0b9e40 0x7f577c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 0x7f577c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1557c0 con 0x7f577c06d7f0 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f577c06e7f0 con 0x7f577c1627f0 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f577b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f577c155940 con 0x7f577c0b9e40 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f5779e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f5779e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35644/0 (socket says 172.21.15.37:35644) 2024-09-21T06:35:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.568+0000 7f5779e00640 1 -- 172.21.15.37:0/1828305607 learned_addr learned my addr 172.21.15.37:0/1828305607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f577a800640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 0x7f577c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 -- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c0b9e40 msgr2=0x7f577c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c0b9e40 0x7f577c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 -- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 msgr2=0x7f577c0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 0x7f577c0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 -- 172.21.15.37:0/1828305607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c0c8fe0 con 0x7f577c06d7f0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f577a800640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 0x7f577c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5779e00640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f576c00e790 tx=0x7f576c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c00eea0 con 0x7f577c06d7f0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.569+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f576c00cd50 con 0x7f577c06d7f0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.570+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c010690 con 0x7f577c06d7f0 2024-09-21T06:35:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.570+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c0c9790 con 0x7f577c06d7f0 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.570+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c15ab50 con 0x7f577c06d7f0 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.571+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c1609d0 con 0x7f577c06d7f0 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.572+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f576c010830 con 0x7f577c06d7f0 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.573+0000 7f5772000640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 0x7f575807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.573+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f576c09ba10 con 0x7f577c06d7f0 2024-09-21T06:35:44.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.573+0000 7f5773e00640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 0x7f575807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.574+0000 7f5773e00640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 0x7f575807b0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5764034000 tx=0x7f57640046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.577+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f576c063bb0 con 0x7f577c06d7f0 2024-09-21T06:35:44.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.811+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577c155290 con 0x7f577c06d7f0 2024-09-21T06:35:44.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.811+0000 7f5772000640 1 -- 172.21.15.37:0/1828305607 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f576c0634b0 con 0x7f577c06d7f0 2024-09-21T06:35:44.814 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:44.814 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:44.814 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.813+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 msgr2=0x7f575807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.813+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 0x7f575807b0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5764034000 tx=0x7f57640046c0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.813+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 msgr2=0x7f577c0c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.813+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f576c00e790 tx=0x7f576c00ec50 comp rx=0 tx=0).stop 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 shutdown_connections 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5758078b90 0x7f575807b0a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f577c1627f0 0x7f577c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f577c0b9e40 0x7f577c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 --2- 172.21.15.37:0/1828305607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577c06d7f0 0x7f577c0c87d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 >> 172.21.15.37:0/1828305607 conn(0x7f577c0d1930 msgr2=0x7f577c1611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 shutdown_connections 2024-09-21T06:35:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:44.814+0000 7f577b200640 1 -- 172.21.15.37:0/1828305607 wait complete. 2024-09-21T06:35:44.837 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:44.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 -- 172.21.15.37:0/754457483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 msgr2=0x7f11dc06e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 --2- 172.21.15.37:0/754457483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc06e570 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009830 tx=0x7f11c802a480 comp rx=0 tx=0).stop 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 -- 172.21.15.37:0/754457483 shutdown_connections 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 --2- 172.21.15.37:0/754457483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc06e570 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 --2- 172.21.15.37:0/754457483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc15dfe0 0x7f11dc160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 -- 172.21.15.37:0/754457483 >> 172.21.15.37:0/754457483 conn(0x7f11dc06f940 msgr2=0x7f11dc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 -- 172.21.15.37:0/754457483 shutdown_connections 2024-09-21T06:35:45.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.061+0000 7f11e3000640 1 -- 172.21.15.37:0/754457483 wait complete. 2024-09-21T06:35:45.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.064+0000 7f11e3000640 1 Processor -- start 2024-09-21T06:35:45.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.064+0000 7f11e3000640 1 -- start start 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc0cc8d0 0x7f11dc12f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 0x7f11dc12fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0b4c60 con 0x7f11dc06e160 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0b4ae0 con 0x7f11dc15dfe0 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e3000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f11dc0b4de0 con 0x7f11dc0cc8d0 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.065+0000 7f11e1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35658/0 (socket says 172.21.15.37:35658) 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 -- 172.21.15.37:0/2031936777 learned_addr learned my addr 172.21.15.37:0/2031936777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1200640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc0cc8d0 0x7f11dc12f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e2600640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 0x7f11dc12fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 -- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc0cc8d0 msgr2=0x7f11dc12f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc0cc8d0 0x7f11dc12f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 -- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 msgr2=0x7f11dc12fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 0x7f11dc12fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 -- 172.21.15.37:0/2031936777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc12d880 con 0x7f11dc06e160 2024-09-21T06:35:45.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e2600640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 0x7f11dc12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.066+0000 7f11e1c00640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d770 tx=0x7f11d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.067+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0017070 con 0x7f11dc06e160 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.067+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc12db60 con 0x7f11dc06e160 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.067+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11d0009d20 con 0x7f11dc06e160 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.067+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00106c0 con 0x7f11dc06e160 2024-09-21T06:35:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.067+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc12e120 con 0x7f11dc06e160 2024-09-21T06:35:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.068+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc0b5300 con 0x7f11dc06e160 2024-09-21T06:35:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.069+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11d001a070 con 0x7f11dc06e160 2024-09-21T06:35:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.070+0000 7f11d9600640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 0x7f11bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.070+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f11d009b090 con 0x7f11dc06e160 2024-09-21T06:35:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.071+0000 7f11e1200640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 0x7f11bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.071+0000 7f11e1200640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 0x7f11bc07b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f11c8034000 tx=0x7f11c8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:45.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.074+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11d0062bf0 con 0x7f11dc06e160 2024-09-21T06:35:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.347+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f11dc0cc0e0 con 0x7f11dc06e160 2024-09-21T06:35:45.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.348+0000 7f11d9600640 1 -- 172.21.15.37:0/2031936777 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1991 (secure 0 0 0) 0x7f11d0067620 con 0x7f11dc06e160 2024-09-21T06:35:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:45.350 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":267476992,"bytes_avail":772826636288,"bytes_total":773094113280,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-09-21T06:35:39.217213+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 msgr2=0x7f11bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 0x7f11bc07b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f11c8034000 tx=0x7f11c8004730 comp rx=0 tx=0).stop 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 msgr2=0x7f11dc133880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d770 tx=0x7f11d000dc30 comp rx=0 tx=0).stop 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 shutdown_connections 2024-09-21T06:35:45.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.350+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f11bc078b90 0x7f11bc07b0a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11dc15dfe0 0x7f11dc12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11dc0cc8d0 0x7f11dc12f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 --2- 172.21.15.37:0/2031936777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11dc06e160 0x7f11dc133880 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 >> 172.21.15.37:0/2031936777 conn(0x7f11dc06f940 msgr2=0x7f11dc1430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 shutdown_connections 2024-09-21T06:35:45.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:45.351+0000 7f11e3000640 1 -- 172.21.15.37:0/2031936777 wait complete. 2024-09-21T06:35:45.376 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:46.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 -- 172.21.15.37:0/3069072464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea8004d40 msgr2=0x7f3ea80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 --2- 172.21.15.37:0/3069072464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea8004d40 0x7f3ea80051d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009f60 tx=0x7f3ea8005830 comp rx=0 tx=0).stop 2024-09-21T06:35:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ec8c00640 1 -- 172.21.15.37:0/3069072464 reap_dead start 2024-09-21T06:35:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 -- 172.21.15.37:0/3069072464 shutdown_connections 2024-09-21T06:35:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 -- 172.21.15.37:0/3069072464 >> 172.21.15.37:0/3069072464 conn(0x7f3ec40cca40 msgr2=0x7f3ec40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 -- 172.21.15.37:0/3069072464 shutdown_connections 2024-09-21T06:35:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.611+0000 7f3ecac00640 1 -- 172.21.15.37:0/3069072464 wait complete. 2024-09-21T06:35:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.613+0000 7f3ecac00640 1 Processor -- start 2024-09-21T06:35:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.613+0000 7f3ecac00640 1 -- start start 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec40c94d0 0x7f3ec40c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 0x7f3ec41f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec406ef60 con 0x7f3ec409fd90 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec406ede0 con 0x7f3ec41f59f0 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ecac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec406f0e0 con 0x7f3ec40c94d0 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35678/0 (socket says 172.21.15.37:35678) 2024-09-21T06:35:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 -- 172.21.15.37:0/906560237 learned_addr learned my addr 172.21.15.37:0/906560237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ec8c00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 0x7f3ec41f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 -- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec40c94d0 msgr2=0x7f3ec40c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec40c94d0 0x7f3ec40c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 -- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 msgr2=0x7f3ec41f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 0x7f3ec41f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ebbe00640 1 -- 172.21.15.37:0/906560237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec40a06e0 con 0x7f3ec409fd90 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.615+0000 7f3ec8c00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 0x7f3ec41f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ebbe00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00e9a0 tx=0x7f3eac00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00cd80 con 0x7f3ec409fd90 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3eac004020 con 0x7f3ec409fd90 2024-09-21T06:35:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec41f8c90 con 0x7f3ec409fd90 2024-09-21T06:35:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac004430 con 0x7f3ec409fd90 2024-09-21T06:35:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.616+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec41f9250 con 0x7f3ec409fd90 2024-09-21T06:35:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.617+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec406ddf0 con 0x7f3ec409fd90 2024-09-21T06:35:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.619+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3eac0035f0 con 0x7f3ec409fd90 2024-09-21T06:35:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.620+0000 7f3ec0c00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 0x7f3ea007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.620+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f3eac09ba80 con 0x7f3ec409fd90 2024-09-21T06:35:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.620+0000 7f3ec3e00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 0x7f3ea007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.621+0000 7f3ec3e00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 0x7f3ea007b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4003670 tx=0x7f3eb4006430 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.624+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3eac09de20 con 0x7f3ec409fd90 2024-09-21T06:35:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.860+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ec40690c0 con 0x7f3ec409fd90 2024-09-21T06:35:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.861+0000 7f3ec0c00640 1 -- 172.21.15.37:0/906560237 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f3ec40a06e0 con 0x7f3ec409fd90 2024-09-21T06:35:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:46.863 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 msgr2=0x7f3ea007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 0x7f3ea007b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4003670 tx=0x7f3eb4006430 comp rx=0 tx=0).stop 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 msgr2=0x7f3ec40a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00e9a0 tx=0x7f3eac00ee60 comp rx=0 tx=0).stop 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 shutdown_connections 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3ea0078b90 0x7f3ea007b0a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec409fd90 0x7f3ec40a01a0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec41f59f0 0x7f3ec41f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 --2- 172.21.15.37:0/906560237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec40c94d0 0x7f3ec40c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 >> 172.21.15.37:0/906560237 conn(0x7f3ec40cca40 msgr2=0x7f3ec40cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 shutdown_connections 2024-09-21T06:35:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:46.864+0000 7f3ecac00640 1 -- 172.21.15.37:0/906560237 wait complete. 2024-09-21T06:35:46.890 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:46.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:47.043 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:35:47.041+0000 7f5112e00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T06:35:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 -- 172.21.15.37:0/2727490763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c074cb0 msgr2=0x7f937c073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 --2- 172.21.15.37:0/2727490763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c074cb0 0x7f937c073dd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f936c005fd0 tx=0x7f936c02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 -- 172.21.15.37:0/2727490763 shutdown_connections 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 --2- 172.21.15.37:0/2727490763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 0x7f937c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 --2- 172.21.15.37:0/2727490763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c074cb0 0x7f937c073dd0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 -- 172.21.15.37:0/2727490763 >> 172.21.15.37:0/2727490763 conn(0x7f937c0cca40 msgr2=0x7f937c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 -- 172.21.15.37:0/2727490763 shutdown_connections 2024-09-21T06:35:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.113+0000 7f9380e00640 1 -- 172.21.15.37:0/2727490763 wait complete. 2024-09-21T06:35:47.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.116+0000 7f9380e00640 1 Processor -- start 2024-09-21T06:35:47.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.116+0000 7f9380e00640 1 -- start start 2024-09-21T06:35:47.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937c074cb0 0x7f937c1e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 0x7f937c1f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f937c06f360 con 0x7f937c06ddf0 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f937c06f1e0 con 0x7f937c074cb0 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9380e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f937c06f4e0 con 0x7f937c0cbc10 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9379e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9379e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35700/0 (socket says 172.21.15.37:35700) 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.117+0000 7f9379e00640 1 -- 172.21.15.37:0/3732693214 learned_addr learned my addr 172.21.15.37:0/3732693214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9373e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937c074cb0 0x7f937c1e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:47.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f937a800640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 0x7f937c1f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 -- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 msgr2=0x7f937c1f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 0x7f937c1f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 -- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937c074cb0 msgr2=0x7f937c1e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937c074cb0 0x7f937c1e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 -- 172.21.15.37:0/3732693214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c0b0e80 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.118+0000 7f9379e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.119+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c03e030 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.119+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936c003ca0 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.119+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c035040 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.119+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c0b0be0 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.119+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c0b0910 con 0x7f937c06ddf0 2024-09-21T06:35:47.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.120+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c06fa00 con 0x7f937c06ddf0 2024-09-21T06:35:47.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.121+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f936c041030 con 0x7f937c06ddf0 2024-09-21T06:35:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.122+0000 7f9372000640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 0x7f935807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:47.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.122+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f936c0bb510 con 0x7f937c06ddf0 2024-09-21T06:35:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.123+0000 7f9373e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 0x7f935807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.123+0000 7f9373e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 0x7f935807b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f937c1e8120 tx=0x7f936400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.127+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f936c083060 con 0x7f937c06ddf0 2024-09-21T06:35:47.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.397+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f937c1579c0 con 0x7f937c06ddf0 2024-09-21T06:35:47.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.397+0000 7f9372000640 1 -- 172.21.15.37:0/3732693214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1991 (secure 0 0 0) 0x7f936c087a90 con 0x7f937c06ddf0 2024-09-21T06:35:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:47.400 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":495,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":267509760,"bytes_avail":772826603520,"bytes_total":773094113280,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-09-21T06:35:39.217213+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 msgr2=0x7f935807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 0x7f935807b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f937c1e8120 tx=0x7f936400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 msgr2=0x7f937c0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c0038a0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 shutdown_connections 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f9358078b90 0x7f935807b0a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f937c0cbc10 0x7f937c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937c074cb0 0x7f937c1e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 --2- 172.21.15.37:0/3732693214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f937c06ddf0 0x7f937c0b03d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:47.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 >> 172.21.15.37:0/3732693214 conn(0x7f937c0cca40 msgr2=0x7f937c159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:47.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 shutdown_connections 2024-09-21T06:35:47.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:47.400+0000 7f9380e00640 1 -- 172.21.15.37:0/3732693214 wait complete. 2024-09-21T06:35:47.425 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:48.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 -- 172.21.15.37:0/1026905835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96500cc8b0 msgr2=0x7f96500cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 --2- 172.21.15.37:0/1026905835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96500cc8b0 0x7f96500cccc0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9638005fd0 tx=0x7f963802a540 comp rx=0 tx=0).stop 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 -- 172.21.15.37:0/1026905835 shutdown_connections 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 --2- 172.21.15.37:0/1026905835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 0x7f96500ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 --2- 172.21.15.37:0/1026905835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96500cc8b0 0x7f96500cccc0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 -- 172.21.15.37:0/1026905835 >> 172.21.15.37:0/1026905835 conn(0x7f965006f940 msgr2=0x7f96500b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 -- 172.21.15.37:0/1026905835 shutdown_connections 2024-09-21T06:35:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.667+0000 7f964f200640 1 -- 172.21.15.37:0/1026905835 wait complete. 2024-09-21T06:35:48.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.669+0000 7f964f200640 1 Processor -- start 2024-09-21T06:35:48.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.669+0000 7f964f200640 1 -- start start 2024-09-21T06:35:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96500cc8b0 0x7f96500cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 0x7f96500cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96500ce970 con 0x7f965015f760 2024-09-21T06:35:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96500ce7f0 con 0x7f96500cc8b0 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f96500ceaf0 con 0x7f96500cdea0 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35728/0 (socket says 172.21.15.37:35728) 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 -- 172.21.15.37:0/3357653620 learned_addr learned my addr 172.21.15.37:0/3357653620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f9647e00640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 0x7f96500cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964de00640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96500cc8b0 0x7f96500cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 -- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 msgr2=0x7f96500cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 0x7f96500cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 -- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96500cc8b0 msgr2=0x7f96500cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96500cc8b0 0x7f96500cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.671+0000 7f964e800640 1 -- 172.21.15.37:0/3357653620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501486e0 con 0x7f965015f760 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f964e800640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f964800caa0 tx=0x7f964800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:48.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964800adf0 con 0x7f965015f760 2024-09-21T06:35:48.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f964800f540 con 0x7f965015f760 2024-09-21T06:35:48.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964800f990 con 0x7f965015f760 2024-09-21T06:35:48.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9650147b60 con 0x7f965015f760 2024-09-21T06:35:48.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.672+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9650148120 con 0x7f965015f760 2024-09-21T06:35:48.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.673+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96501478b0 con 0x7f965015f760 2024-09-21T06:35:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.674+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f964800fb30 con 0x7f965015f760 2024-09-21T06:35:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.675+0000 7f9646000640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 0x7f962c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.676+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f964809ba20 con 0x7f965015f760 2024-09-21T06:35:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.676+0000 7f964de00640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 0x7f962c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:48.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.677+0000 7f964de00640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 0x7f962c07b0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9640007fd0 tx=0x7f9640009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.680+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9648014030 con 0x7f965015f760 2024-09-21T06:35:48.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.917+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96500cccc0 con 0x7f965015f760 2024-09-21T06:35:48.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.918+0000 7f9646000640 1 -- 172.21.15.37:0/3357653620 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f964800fe20 con 0x7f965015f760 2024-09-21T06:35:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:48.920 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:48.921 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 msgr2=0x7f962c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 0x7f962c07b0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9640007fd0 tx=0x7f9640009040 comp rx=0 tx=0).stop 2024-09-21T06:35:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 msgr2=0x7f96501f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f964800caa0 tx=0x7f964800cf60 comp rx=0 tx=0).stop 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 shutdown_connections 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f962c078b90 0x7f962c07b0a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f965015f760 0x7f96501f5350 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96500cdea0 0x7f96500cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 --2- 172.21.15.37:0/3357653620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96500cc8b0 0x7f96500cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.920+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 >> 172.21.15.37:0/3357653620 conn(0x7f965006f940 msgr2=0x7f965006dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.921+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 shutdown_connections 2024-09-21T06:35:48.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:48.921+0000 7f964f200640 1 -- 172.21.15.37:0/3357653620 wait complete. 2024-09-21T06:35:48.947 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:48.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 -- 172.21.15.37:0/33167969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b1806def0 msgr2=0x7f5b1806e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 --2- 172.21.15.37:0/33167969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b1806def0 0x7f5b1806e300 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005fd0 tx=0x7f5b1002aa90 comp rx=0 tx=0).stop 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 -- 172.21.15.37:0/33167969 shutdown_connections 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 --2- 172.21.15.37:0/33167969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b1806def0 0x7f5b1806e300 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 --2- 172.21.15.37:0/33167969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b18073dd0 0x7f5b1815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.170+0000 7f5b17200640 1 -- 172.21.15.37:0/33167969 >> 172.21.15.37:0/33167969 conn(0x7f5b1806f940 msgr2=0x7f5b180b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:49.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.171+0000 7f5b17200640 1 -- 172.21.15.37:0/33167969 shutdown_connections 2024-09-21T06:35:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.171+0000 7f5b17200640 1 -- 172.21.15.37:0/33167969 wait complete. 2024-09-21T06:35:49.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.173+0000 7f5b17200640 1 Processor -- start 2024-09-21T06:35:49.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.173+0000 7f5b17200640 1 -- start start 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1806def0 0x7f5b18149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b18152530 0x7f5b1815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180cf5f0 con 0x7f5b18073dd0 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180cf470 con 0x7f5b1806def0 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.174+0000 7f5b17200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f5b180cf770 con 0x7f5b18152530 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35742/0 (socket says 172.21.15.37:35742) 2024-09-21T06:35:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 -- 172.21.15.37:0/219799200 learned_addr learned my addr 172.21.15.37:0/219799200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 -- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b18152530 msgr2=0x7f5b1815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b18152530 0x7f5b1815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 -- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1806def0 msgr2=0x7f5b18149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1806def0 0x7f5b18149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 -- 172.21.15.37:0/219799200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b181f9190 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.175+0000 7f5b0fe00640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c3a0 tx=0x7f5b0000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.176+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017070 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.176+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b000053c0 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.176+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00003850 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.176+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181f9470 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.176+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b180b09d0 con 0x7f5b18073dd0 2024-09-21T06:35:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.177+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b180cfc20 con 0x7f5b18073dd0 2024-09-21T06:35:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.178+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b0001a070 con 0x7f5b18073dd0 2024-09-21T06:35:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.179+0000 7f5b0e000640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.179+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f5b0009a2c0 con 0x7f5b18073dd0 2024-09-21T06:35:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.180+0000 7f5b15e00640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:49.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.180+0000 7f5b15e00640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 0x7f5af407b0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5b080058c0 tx=0x7f5b08005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.183+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5b1815c8f0 con 0x7f5b18073dd0 2024-09-21T06:35:49.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.454+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5b181f8f20 con 0x7f5b18073dd0 2024-09-21T06:35:49.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.455+0000 7f5b0e000640 1 -- 172.21.15.37:0/219799200 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1927 (secure 0 0 0) 0x7f5b000624f0 con 0x7f5b18073dd0 2024-09-21T06:35:49.457 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:49.457 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":267509760,"bytes_avail":772826603520,"bytes_total":773094113280},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-09-21T06:35:39.217213+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:49.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 msgr2=0x7f5af407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:49.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 0x7f5af407b0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5b080058c0 tx=0x7f5b08005850 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 msgr2=0x7f5b180b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c3a0 tx=0x7f5b0000c860 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 shutdown_connections 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b18073dd0 0x7f5b180b02c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b18152530 0x7f5b1815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 --2- 172.21.15.37:0/219799200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1806def0 0x7f5b18149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 >> 172.21.15.37:0/219799200 conn(0x7f5b1806f940 msgr2=0x7f5b180b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 shutdown_connections 2024-09-21T06:35:49.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:49.457+0000 7f5b17200640 1 -- 172.21.15.37:0/219799200 wait complete. 2024-09-21T06:35:49.481 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-21T06:35:50.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 -- 172.21.15.37:0/3739296679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c06cee0 msgr2=0x7f4c5c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 --2- 172.21.15.37:0/3739296679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c06cee0 0x7f4c5c06d2f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005fd0 tx=0x7f4c5402a8b0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 -- 172.21.15.37:0/3739296679 shutdown_connections 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 --2- 172.21.15.37:0/3739296679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c06cee0 0x7f4c5c06d2f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 --2- 172.21.15.37:0/3739296679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c071160 0x7f4c5c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 -- 172.21.15.37:0/3739296679 >> 172.21.15.37:0/3739296679 conn(0x7f4c5c0d1020 msgr2=0x7f4c5c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 -- 172.21.15.37:0/3739296679 shutdown_connections 2024-09-21T06:35:50.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.719+0000 7f4c60a00640 1 -- 172.21.15.37:0/3739296679 wait complete. 2024-09-21T06:35:50.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.722+0000 7f4c60a00640 1 Processor -- start 2024-09-21T06:35:50.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.722+0000 7f4c60a00640 1 -- start start 2024-09-21T06:35:50.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c06cee0 0x7f4c5c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c5c160980 0x7f4c5c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c06dbc0 con 0x7f4c5c071160 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c06da40 con 0x7f4c5c160980 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c60a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c06dd40 con 0x7f4c5c06cee0 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.723+0000 7f4c59e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35762/0 (socket says 172.21.15.37:35762) 2024-09-21T06:35:50.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 -- 172.21.15.37:0/1255274854 learned_addr learned my addr 172.21.15.37:0/1255274854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c5a800640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c06cee0 0x7f4c5c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c5b200640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c5c160980 0x7f4c5c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 -- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c06cee0 msgr2=0x7f4c5c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c06cee0 0x7f4c5c14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 -- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c5c160980 msgr2=0x7f4c5c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c5c160980 0x7f4c5c150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 -- 172.21.15.37:0/1255274854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c151a10 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c59e00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e7f0 tx=0x7f4c4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.724+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800eea0 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.725+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c4800cdb0 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.725+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010690 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.725+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c151cf0 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.725+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c143ac0 con 0x7f4c5c071160 2024-09-21T06:35:50.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.726+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c151780 con 0x7f4c5c071160 2024-09-21T06:35:50.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.727+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c48010830 con 0x7f4c5c071160 2024-09-21T06:35:50.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.728+0000 7f4c52a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 0x7f4c3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.728+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f4c4809b9a0 con 0x7f4c5c071160 2024-09-21T06:35:50.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.729+0000 7f4c5a800640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 0x7f4c3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.729+0000 7f4c5a800640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 0x7f4c3c07b0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c007fd0 tx=0x7f4c4c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.733+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c48063b40 con 0x7f4c5c071160 2024-09-21T06:35:50.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:35:50.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:35:50.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.968+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c5c06a580 con 0x7f4c5c071160 2024-09-21T06:35:50.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.969+0000 7f4c52a00640 1 -- 172.21.15.37:0/1255274854 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f4c5c06a580 con 0x7f4c5c071160 2024-09-21T06:35:50.971 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:50.971 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:50.972 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 msgr2=0x7f4c3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 0x7f4c3c07b0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c007fd0 tx=0x7f4c4c009040 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 msgr2=0x7f4c5c14e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e7f0 tx=0x7f4c4800ecb0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 shutdown_connections 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4c3c078b90 0x7f4c3c07b0a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5c071160 0x7f4c5c14e6b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c5c160980 0x7f4c5c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 --2- 172.21.15.37:0/1255274854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c5c06cee0 0x7f4c5c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 >> 172.21.15.37:0/1255274854 conn(0x7f4c5c0d1020 msgr2=0x7f4c5c145040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:50.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 shutdown_connections 2024-09-21T06:35:50.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:50.971+0000 7f4c60a00640 1 -- 172.21.15.37:0/1255274854 wait complete. 2024-09-21T06:35:51.000 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:51.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 -- 172.21.15.37:0/1153858543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 msgr2=0x7f8fb80b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 --2- 172.21.15.37:0/1153858543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb80b4990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8fb81611d0 tx=0x7f8fa402a480 comp rx=0 tx=0).stop 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 -- 172.21.15.37:0/1153858543 shutdown_connections 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 --2- 172.21.15.37:0/1153858543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb80b4990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 --2- 172.21.15.37:0/1153858543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 0x7f8fb80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 -- 172.21.15.37:0/1153858543 >> 172.21.15.37:0/1153858543 conn(0x7f8fb806f940 msgr2=0x7f8fb80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 -- 172.21.15.37:0/1153858543 shutdown_connections 2024-09-21T06:35:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.264+0000 7f8fbd400640 1 -- 172.21.15.37:0/1153858543 wait complete. 2024-09-21T06:35:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.266+0000 7f8fbd400640 1 Processor -- start 2024-09-21T06:35:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.267+0000 7f8fbd400640 1 -- start start 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 0x7f8fb8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb806e560 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb806e3e0 con 0x7f8fb80ccd50 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fbd400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb815d630 con 0x7f8fb80cddc0 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35768/0 (socket says 172.21.15.37:35768) 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.268+0000 7f8fb7e00640 1 -- 172.21.15.37:0/3901820530 learned_addr learned my addr 172.21.15.37:0/3901820530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fbca00640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 0x7f8fb8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 -- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 msgr2=0x7f8fb8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 0x7f8fb8156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 -- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 msgr2=0x7f8fb8158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb8158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 -- 172.21.15.37:0/3901820530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb814def0 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fb7e00640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ea20 tx=0x7f8fa800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800ce40 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fa8004020 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004470 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.269+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb8153a00 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.270+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb8153fc0 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.271+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb814e0c0 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.272+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fa80035f0 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.273+0000 7f8fafe00640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 0x7f8f9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.273+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f8fa809bab0 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.273+0000 7f8fb7400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 0x7f8f9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.274+0000 7f8fb7400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 0x7f8f9807b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4034000 tx=0x7f8fa4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.278+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fa8063c50 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.551+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8fb80b4990 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.551+0000 7f8fafe00640 1 -- 172.21.15.37:0/3901820530 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1914 (secure 0 0 0) 0x7f8fa8063550 con 0x7f8fb80b4ed0 2024-09-21T06:35:51.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":267509760,"bytes_avail":772826603520,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":35,"modified":"2024-09-21T06:35:49.220478+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:51.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.553+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 msgr2=0x7f8f9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.553+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 0x7f8f9807b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4034000 tx=0x7f8fa4004730 comp rx=0 tx=0).stop 2024-09-21T06:35:51.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.553+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 msgr2=0x7f8fb814d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.553+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ea20 tx=0x7f8fa800eee0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 shutdown_connections 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f8f98078bc0 0x7f8f9807b0d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb80cddc0 0x7f8fb8156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb80ccd50 0x7f8fb8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 --2- 172.21.15.37:0/3901820530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fb80b4ed0 0x7f8fb814d9b0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 >> 172.21.15.37:0/3901820530 conn(0x7f8fb806f940 msgr2=0x7f8fb815dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 shutdown_connections 2024-09-21T06:35:51.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:51.554+0000 7f8fbd400640 1 -- 172.21.15.37:0/3901820530 wait complete. 2024-09-21T06:35:51.582 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:35:52.583 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:52.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 -- 172.21.15.37:0/3852756169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 msgr2=0x7f96f40cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 --2- 172.21.15.37:0/3852756169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f40cc020 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f96f4076390 tx=0x7f96e002a560 comp rx=0 tx=0).stop 2024-09-21T06:35:52.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 -- 172.21.15.37:0/3852756169 shutdown_connections 2024-09-21T06:35:52.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 --2- 172.21.15.37:0/3852756169 >> v2:172.21.15.150:3300/0 conn(0x7f96f406ddf0 0x7f96f406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 --2- 172.21.15.37:0/3852756169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f40cc020 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 --2- 172.21.15.37:0/3852756169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f4074cb0 0x7f96f4073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 -- 172.21.15.37:0/3852756169 >> 172.21.15.37:0/3852756169 conn(0x7f96f40cca40 msgr2=0x7f96f40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:52.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 -- 172.21.15.37:0/3852756169 shutdown_connections 2024-09-21T06:35:52.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.817+0000 7f96fc000640 1 -- 172.21.15.37:0/3852756169 wait complete. 2024-09-21T06:35:52.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.819+0000 7f96fc000640 1 Processor -- start 2024-09-21T06:35:52.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.819+0000 7f96fc000640 1 -- start start 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 0x7f96f40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 0x7f96f4153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96f406f170 con 0x7f96f40cbc10 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96f406eff0 con 0x7f96f4074cb0 2024-09-21T06:35:52.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fc000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f96f406f2f0 con 0x7f96f406ddf0 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35782/0 (socket says 172.21.15.37:35782) 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 -- 172.21.15.37:0/4257431116 learned_addr learned my addr 172.21.15.37:0/4257431116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96f9600640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 0x7f96f40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 -- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 msgr2=0x7f96f40c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96f8c00640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 0x7f96f4153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 0x7f96f40c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 -- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 msgr2=0x7f96f4153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 0x7f96f4153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96fa000640 1 -- 172.21.15.37:0/4257431116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f41f64a0 con 0x7f96f40cbc10 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96f9600640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 0x7f96f40c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.821+0000 7f96f8c00640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 0x7f96f4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:52.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96fa000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00cab0 tx=0x7f96ec00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec007df0 con 0x7f96f40cbc10 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96ec00f560 con 0x7f96f40cbc10 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00f9b0 con 0x7f96f40cbc10 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f41f6780 con 0x7f96f40cbc10 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.822+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f40ca550 con 0x7f96f40cbc10 2024-09-21T06:35:52.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.823+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f40c8d10 con 0x7f96f40cbc10 2024-09-21T06:35:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.825+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96ec00fb50 con 0x7f96f40cbc10 2024-09-21T06:35:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.826+0000 7f96f1600640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 0x7f96d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.826+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f96ec09ba60 con 0x7f96f40cbc10 2024-09-21T06:35:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.826+0000 7f96f9600640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 0x7f96d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.827+0000 7f96f9600640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 0x7f96d407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f96f415a730 tx=0x7f96e8005d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:52.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:52.830+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96f40c8d10 con 0x7f96f40cbc10 2024-09-21T06:35:53.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.066+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f415a3f0 con 0x7f96f40cbc10 2024-09-21T06:35:53.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.067+0000 7f96f1600640 1 -- 172.21.15.37:0/4257431116 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f96ec014070 con 0x7f96f40cbc10 2024-09-21T06:35:53.069 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:53.070 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:53.070 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.069+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 msgr2=0x7f96d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.069+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 0x7f96d407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f96f415a730 tx=0x7f96e8005d10 comp rx=0 tx=0).stop 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 msgr2=0x7f96f409f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00cab0 tx=0x7f96ec00cf70 comp rx=0 tx=0).stop 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 shutdown_connections 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f96d4078b90 0x7f96d407b0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40cbc10 0x7f96f409f760 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f4074cb0 0x7f96f4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 --2- 172.21.15.37:0/4257431116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f406ddf0 0x7f96f40c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 >> 172.21.15.37:0/4257431116 conn(0x7f96f40cca40 msgr2=0x7f96f415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 shutdown_connections 2024-09-21T06:35:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.070+0000 7f96fc000640 1 -- 172.21.15.37:0/4257431116 wait complete. 2024-09-21T06:35:53.096 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:53.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 -- 172.21.15.37:0/2020946378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 msgr2=0x7fc21806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 --2- 172.21.15.37:0/2020946378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc21806f710 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc20800a940 tx=0x7fc20802b8f0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 -- 172.21.15.37:0/2020946378 shutdown_connections 2024-09-21T06:35:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 --2- 172.21.15.37:0/2020946378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc21806f710 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 -- 172.21.15.37:0/2020946378 >> 172.21.15.37:0/2020946378 conn(0x7fc2180cca40 msgr2=0x7fc2180cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 -- 172.21.15.37:0/2020946378 shutdown_connections 2024-09-21T06:35:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.320+0000 7fc21e400640 1 -- 172.21.15.37:0/2020946378 wait complete. 2024-09-21T06:35:53.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.322+0000 7fc21e400640 1 Processor -- start 2024-09-21T06:35:53.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.322+0000 7fc21e400640 1 -- start start 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc218070030 0x7fc2180b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc218073ca0 0x7fc2181f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc218071000 con 0x7fc218074d90 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc218070e80 con 0x7fc218073ca0 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc21e400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc2180ce2a0 con 0x7fc218070030 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc216a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc216a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35798/0 (socket says 172.21.15.37:35798) 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc216a00640 1 -- 172.21.15.37:0/3861901783 learned_addr learned my addr 172.21.15.37:0/3861901783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc217400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc218070030 0x7fc2180b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.324+0000 7fc217e00640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc218073ca0 0x7fc2181f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 -- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc218070030 msgr2=0x7fc2180b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc218070030 0x7fc2180b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 -- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc218073ca0 msgr2=0x7fc2181f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc218073ca0 0x7fc2181f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 -- 172.21.15.37:0/3861901783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180b50d0 con 0x7fc218074d90 2024-09-21T06:35:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc216a00640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c430 tx=0x7fc20400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204011790 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc204017070 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2180b2eb0 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204011ce0 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.325+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2180b3150 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.326+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21806f710 con 0x7fc218074d90 2024-09-21T06:35:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.328+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc20401c460 con 0x7fc218074d90 2024-09-21T06:35:53.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.329+0000 7fc20f400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 0x7fc1f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.329+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fc20409ade0 con 0x7fc218074d90 2024-09-21T06:35:53.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.329+0000 7fc217400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 0x7fc1f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.330+0000 7fc217400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 0x7fc1f807b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc208036000 tx=0x7fc208004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.333+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc204063010 con 0x7fc218074d90 2024-09-21T06:35:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.603+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc2180b5f40 con 0x7fc218074d90 2024-09-21T06:35:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.604+0000 7fc20f400640 1 -- 172.21.15.37:0/3861901783 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1914 (secure 0 0 0) 0x7fc204062910 con 0x7fc218074d90 2024-09-21T06:35:53.606 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:53.606 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":267509760,"bytes_avail":772826603520,"bytes_total":773094113280,"write_bytes_sec":693,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":35,"modified":"2024-09-21T06:35:49.220478+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.606+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 msgr2=0x7fc1f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.606+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 0x7fc1f807b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc208036000 tx=0x7fc208004730 comp rx=0 tx=0).stop 2024-09-21T06:35:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 msgr2=0x7fc218073760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c430 tx=0x7fc20400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 shutdown_connections 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc1f8078b90 0x7fc1f807b0a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc218074d90 0x7fc218073760 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc218073ca0 0x7fc2181f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 --2- 172.21.15.37:0/3861901783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc218070030 0x7fc2180b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 >> 172.21.15.37:0/3861901783 conn(0x7fc2180cca40 msgr2=0x7fc21806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 shutdown_connections 2024-09-21T06:35:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:53.607+0000 7fc21e400640 1 -- 172.21.15.37:0/3861901783 wait complete. 2024-09-21T06:35:53.631 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-21T06:35:54.632 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 -- 172.21.15.37:0/407754573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 msgr2=0x7fefe41555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 --2- 172.21.15.37:0/407754573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe41555d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fefe40af940 tx=0x7fefd402a4a0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 -- 172.21.15.37:0/407754573 shutdown_connections 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 --2- 172.21.15.37:0/407754573 >> v2:172.21.15.150:3300/0 conn(0x7fefe40cc8b0 0x7fefe40cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 --2- 172.21.15.37:0/407754573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe41555d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.869+0000 7fefe8e00640 1 -- 172.21.15.37:0/407754573 >> 172.21.15.37:0/407754573 conn(0x7fefe406f940 msgr2=0x7fefe40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.870+0000 7fefe8e00640 1 -- 172.21.15.37:0/407754573 shutdown_connections 2024-09-21T06:35:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.870+0000 7fefe8e00640 1 -- 172.21.15.37:0/407754573 wait complete. 2024-09-21T06:35:54.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.872+0000 7fefe8e00640 1 Processor -- start 2024-09-21T06:35:54.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.872+0000 7fefe8e00640 1 -- start start 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe40cc8b0 0x7fefe41f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 0x7fefe41e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefe406e570 con 0x7fefe415f760 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fefe406e3f0 con 0x7fefe40cc8b0 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.873+0000 7fefe8e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fefe4162df0 con 0x7fefe41622d0 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45512/0 (socket says 172.21.15.37:45512) 2024-09-21T06:35:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 -- 172.21.15.37:0/2140939308 learned_addr learned my addr 172.21.15.37:0/2140939308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefe3400640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe40cc8b0 0x7fefe41f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefe3e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 0x7fefe41e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 -- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 msgr2=0x7fefe41e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 0x7fefe41e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 -- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe40cc8b0 msgr2=0x7fefe41f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe40cc8b0 0x7fefe41f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 -- 172.21.15.37:0/2140939308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe41f5a40 con 0x7fefe415f760 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefe3e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 0x7fefe41e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.874+0000 7fefdaa00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00d6d0 tx=0x7fefcc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.875+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc017070 con 0x7fefe415f760 2024-09-21T06:35:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.875+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefcc004310 con 0x7fefe415f760 2024-09-21T06:35:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.875+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe40c59f0 con 0x7fefe415f760 2024-09-21T06:35:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.875+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc009a80 con 0x7fefe415f760 2024-09-21T06:35:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.875+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41f5fe0 con 0x7fefe415f760 2024-09-21T06:35:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.876+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe40c5760 con 0x7fefe415f760 2024-09-21T06:35:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.877+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefcc0035f0 con 0x7fefe415f760 2024-09-21T06:35:54.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.878+0000 7fefdbe00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 0x7fefc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.878+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fefcc09b240 con 0x7fefe415f760 2024-09-21T06:35:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.879+0000 7fefe3400640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 0x7fefc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.879+0000 7fefe3400640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 0x7fefc007b0a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fefd4034000 tx=0x7fefd4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:54.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:54.883+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefcc063500 con 0x7fefe415f760 2024-09-21T06:35:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.116+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefe40cccc0 con 0x7fefe415f760 2024-09-21T06:35:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.116+0000 7fefdbe00640 1 -- 172.21.15.37:0/2140939308 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7fefcc062e00 con 0x7fefe415f760 2024-09-21T06:35:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:55.119 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 msgr2=0x7fefc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 0x7fefc007b0a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fefd4034000 tx=0x7fefd4004730 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 msgr2=0x7fefe4146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00d6d0 tx=0x7fefcc00db90 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 shutdown_connections 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fefc0078b90 0x7fefc007b0a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe415f760 0x7fefe4146850 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefe41622d0 0x7fefe41e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 --2- 172.21.15.37:0/2140939308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe40cc8b0 0x7fefe41f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 >> 172.21.15.37:0/2140939308 conn(0x7fefe406f940 msgr2=0x7fefe41ec0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 shutdown_connections 2024-09-21T06:35:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.119+0000 7fefe8e00640 1 -- 172.21.15.37:0/2140939308 wait complete. 2024-09-21T06:35:55.143 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-21T06:35:55.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:35:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 -- 172.21.15.37:0/256397966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15ede0 msgr2=0x7fd6ac0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 --2- 172.21.15.37:0/256397966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac0cc820 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd694009860 tx=0x7fd69402a540 comp rx=0 tx=0).stop 2024-09-21T06:35:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 -- 172.21.15.37:0/256397966 shutdown_connections 2024-09-21T06:35:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 --2- 172.21.15.37:0/256397966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac0cc820 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 --2- 172.21.15.37:0/256397966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 0x7fd6ac15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 -- 172.21.15.37:0/256397966 >> 172.21.15.37:0/256397966 conn(0x7fd6ac06f940 msgr2=0x7fd6ac0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 -- 172.21.15.37:0/256397966 shutdown_connections 2024-09-21T06:35:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.367+0000 7fd6ab200640 1 -- 172.21.15.37:0/256397966 wait complete. 2024-09-21T06:35:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.369+0000 7fd6ab200640 1 Processor -- start 2024-09-21T06:35:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.369+0000 7fd6ab200640 1 -- start start 2024-09-21T06:35:55.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 0x7fd6ac13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac13e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac070e70 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac070cf0 con 0x7fd6ac15ede0 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6ab200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac0ce4d0 con 0x7fd6ac15b820 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45522/0 (socket says 172.21.15.37:45522) 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 -- 172.21.15.37:0/329698781 learned_addr learned my addr 172.21.15.37:0/329698781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a3e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 0x7fd6ac13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6aa800640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac13e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 -- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 msgr2=0x7fd6ac13d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 0x7fd6ac13d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 -- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 msgr2=0x7fd6ac13e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac13e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6a9e00640 1 -- 172.21.15.37:0/329698781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac153dd0 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.371+0000 7fd6aa800640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac13e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6a9e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c9a0 tx=0x7fd69c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:55.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c007bb0 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd69c007e40 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00f690 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1540b0 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.372+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac154670 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.373+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac0ce9f0 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.374+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd69c00f830 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.375+0000 7fd6a2000640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 0x7fd68807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:55.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.375+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fd69c09b890 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.376+0000 7fd6a3e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 0x7fd68807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.377+0000 7fd6a3e00640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 0x7fd68807b0a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd694004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.380+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd69c014030 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.651+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd6ac0cdaa0 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.652+0000 7fd6a2000640 1 -- 172.21.15.37:0/329698781 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1738 (secure 0 0 0) 0x7fd69c063a30 con 0x7fd6ac0ccd60 2024-09-21T06:35:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:55.654 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726900067,"num_in_osds":8,"osd_in_since":1726900057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":267509760,"bytes_avail":772826603520,"bytes_total":773094113280,"write_bytes_sec":597,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7079},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8172},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8175}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":36,"modified":"2024-09-21T06:35:53.221797+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:35:55.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 msgr2=0x7fd68807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 0x7fd68807b0a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd694004730 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 msgr2=0x7fd6ac141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c9a0 tx=0x7fd69c00ce60 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 shutdown_connections 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd688078b90 0x7fd68807b0a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6ac15ede0 0x7fd6ac13e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6ac15b820 0x7fd6ac13d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 --2- 172.21.15.37:0/329698781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac0ccd60 0x7fd6ac141160 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 >> 172.21.15.37:0/329698781 conn(0x7fd6ac06f940 msgr2=0x7fd6ac0cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 shutdown_connections 2024-09-21T06:35:55.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:55.654+0000 7fd6ab200640 1 -- 172.21.15.37:0/329698781 wait complete. 2024-09-21T06:35:55.683 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-21T06:35:55.683 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:35:55.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:35:55.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:35:55.873 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:35:55.873 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:35:55.873 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:35:55.873 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:55.909 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:35:55.909 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:55.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff636a00640 1 -- 172.21.15.37:0/1788589763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 msgr2=0x7ff614004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff636a00640 1 --2- 172.21.15.37:0/1788589763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff614004c10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff618009b20 tx=0x7ff614005830 comp rx=0 tx=0).stop 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff634a00640 1 -- 172.21.15.37:0/1788589763 reap_dead start 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff636a00640 1 -- 172.21.15.37:0/1788589763 shutdown_connections 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff636a00640 1 -- 172.21.15.37:0/1788589763 >> 172.21.15.37:0/1788589763 conn(0x7ff6300b1b90 msgr2=0x7ff6300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.252+0000 7ff636a00640 1 -- 172.21.15.37:0/1788589763 shutdown_connections 2024-09-21T06:35:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.253+0000 7ff636a00640 1 -- 172.21.15.37:0/1788589763 wait complete. 2024-09-21T06:35:56.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.255+0000 7ff636a00640 1 Processor -- start 2024-09-21T06:35:56.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.255+0000 7ff636a00640 1 -- start start 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6300c5ed0 0x7ff6300a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 0x7ff6301f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300cf310 con 0x7ff614004800 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff630161060 con 0x7ff6300a0890 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff636a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff6300cf490 con 0x7ff6300c5ed0 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.256+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45532/0 (socket says 172.21.15.37:45532) 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 -- 172.21.15.37:0/1315808731 learned_addr learned my addr 172.21.15.37:0/1315808731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff627400640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6300c5ed0 0x7ff6300a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff634a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 0x7ff6301f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 -- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6300c5ed0 msgr2=0x7ff6300a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6300c5ed0 0x7ff6300a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 -- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 msgr2=0x7ff6301f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 0x7ff6301f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 -- 172.21.15.37:0/1315808731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300a0ca0 con 0x7ff614004800 2024-09-21T06:35:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff634a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 0x7ff6301f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.257+0000 7ff62fe00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e5c0 tx=0x7ff62000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.258+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620015070 con 0x7ff614004800 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.258+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff62000ca00 con 0x7ff614004800 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.258+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff630158230 con 0x7ff614004800 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.258+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62000ce50 con 0x7ff614004800 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.258+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301f5850 con 0x7ff614004800 2024-09-21T06:35:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.259+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff630158400 con 0x7ff614004800 2024-09-21T06:35:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.260+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff620018070 con 0x7ff614004800 2024-09-21T06:35:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.261+0000 7ff62cc00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 0x7ff60c07b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.261+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7ff62009b3f0 con 0x7ff614004800 2024-09-21T06:35:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.262+0000 7ff627400640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 0x7ff60c07b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.262+0000 7ff627400640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 0x7ff60c07b240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff618035000 tx=0x7ff61802d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.265+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff6300a0ca0 con 0x7ff614004800 2024-09-21T06:35:56.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.475+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff63006e200 con 0x7ff614004800 2024-09-21T06:35:56.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.475+0000 7ff62cc00640 1 -- 172.21.15.37:0/1315808731 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7ff63006e740 con 0x7ff614004800 2024-09-21T06:35:56.477 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 msgr2=0x7ff60c07b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 0x7ff60c07b240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff618035000 tx=0x7ff61802d040 comp rx=0 tx=0).stop 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 msgr2=0x7ff6300b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e5c0 tx=0x7ff62000ea80 comp rx=0 tx=0).stop 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 shutdown_connections 2024-09-21T06:35:56.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff60c078d30 0x7ff60c07b240 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6300a0890 0x7ff6301f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6300c5ed0 0x7ff6300a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 --2- 172.21.15.37:0/1315808731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff614004800 0x7ff6300b2ed0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 >> 172.21.15.37:0/1315808731 conn(0x7ff6300b1b90 msgr2=0x7ff6300afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 shutdown_connections 2024-09-21T06:35:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.478+0000 7ff636a00640 1 -- 172.21.15.37:0/1315808731 wait complete. 2024-09-21T06:35:56.502 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:35:56.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:56.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:35:56.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:35:56.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:56.566 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:35:56.602 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:35:56.602 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:35:56.602 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:56.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:35:56.642 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:35:56.642 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:35:56.642 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:56.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:35:56.677 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:35:56.677 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:35:56.677 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:35:56.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:35:56.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 -- 172.21.15.37:0/1443848232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8550071870 msgr2=0x7f855006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 --2- 172.21.15.37:0/1443848232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8550071870 0x7f855006e1f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8550073490 tx=0x7f854802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 -- 172.21.15.37:0/1443848232 shutdown_connections 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 --2- 172.21.15.37:0/1443848232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8550071870 0x7f855006e1f0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 --2- 172.21.15.37:0/1443848232 >> v2:172.21.15.150:3300/0 conn(0x7f85500b3f50 0x7f855015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 -- 172.21.15.37:0/1443848232 >> 172.21.15.37:0/1443848232 conn(0x7f85500d1020 msgr2=0x7f85500d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 -- 172.21.15.37:0/1443848232 shutdown_connections 2024-09-21T06:35:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.945+0000 7f8555200640 1 -- 172.21.15.37:0/1443848232 wait complete. 2024-09-21T06:35:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.947+0000 7f8555200640 1 Processor -- start 2024-09-21T06:35:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.948+0000 7f8555200640 1 -- start start 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f855006dde0 0x7f85501f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85500b3f50 0x7f85501f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f855015f380 con 0x7f855015ace0 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f855015f200 con 0x7f85500b3f50 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f8555200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f855015f500 con 0x7f855006dde0 2024-09-21T06:35:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f854fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f854fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45550/0 (socket says 172.21.15.37:45550) 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.949+0000 7f854fe00640 1 -- 172.21.15.37:0/16014119 learned_addr learned my addr 172.21.15.37:0/16014119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854f400640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f855006dde0 0x7f85501f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f8546a00640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85500b3f50 0x7f85501f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 -- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f855006dde0 msgr2=0x7f85501f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f855006dde0 0x7f85501f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 -- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85500b3f50 msgr2=0x7f85501f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85500b3f50 0x7f85501f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 -- 172.21.15.37:0/16014119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85501f8e20 con 0x7f855015ace0 2024-09-21T06:35:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f854fe00640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ca0 tx=0x7f8548031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548031990 con 0x7f855015ace0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8548031c20 con 0x7f855015ace0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480330d0 con 0x7f855015ace0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.950+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85501f9100 con 0x7f855015ace0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.951+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85501f52f0 con 0x7f855015ace0 2024-09-21T06:35:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.952+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855015f9b0 con 0x7f855015ace0 2024-09-21T06:35:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.953+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8548033270 con 0x7f855015ace0 2024-09-21T06:35:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.954+0000 7f8547e00640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 0x7f852c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.954+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f85480bc4f0 con 0x7f855015ace0 2024-09-21T06:35:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.954+0000 7f854f400640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 0x7f852c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.955+0000 7f854f400640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 0x7f852c07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f854000bb00 tx=0x7f8540005e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:56.958+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f854803f030 con 0x7f855015ace0 2024-09-21T06:35:57.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.197+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8550072430 con 0x7f855015ace0 2024-09-21T06:35:57.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.197+0000 7f8547e00640 1 -- 172.21.15.37:0/16014119 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f85480847b0 con 0x7f855015ace0 2024-09-21T06:35:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:57.200 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:57.200 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:57.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 msgr2=0x7f852c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 0x7f852c07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f854000bb00 tx=0x7f8540005e30 comp rx=0 tx=0).stop 2024-09-21T06:35:57.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 msgr2=0x7f8550142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ca0 tx=0x7f8548031760 comp rx=0 tx=0).stop 2024-09-21T06:35:57.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 shutdown_connections 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f852c078b90 0x7f852c07b0a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f855015ace0 0x7f8550142430 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85500b3f50 0x7f85501f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 --2- 172.21.15.37:0/16014119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f855006dde0 0x7f85501f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 >> 172.21.15.37:0/16014119 conn(0x7f85500d1020 msgr2=0x7f8550156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 shutdown_connections 2024-09-21T06:35:57.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.200+0000 7f8555200640 1 -- 172.21.15.37:0/16014119 wait complete. 2024-09-21T06:35:57.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.451+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3085597456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5406e160 msgr2=0x7f3c5406e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.451+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3085597456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5406e160 0x7f3c5406e570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009830 tx=0x7f3c4002a480 comp rx=0 tx=0).stop 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3085597456 shutdown_connections 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3085597456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5406e160 0x7f3c5406e570 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3085597456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5415dfe0 0x7f3c54160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3085597456 >> 172.21.15.37:0/3085597456 conn(0x7f3c5406f940 msgr2=0x7f3c540b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3085597456 shutdown_connections 2024-09-21T06:35:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.452+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3085597456 wait complete. 2024-09-21T06:35:57.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.454+0000 7f3c5aa00640 1 Processor -- start 2024-09-21T06:35:57.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.454+0000 7f3c5aa00640 1 -- start start 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 0x7f3c54153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5415dfe0 0x7f3c540b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c540b4c50 con 0x7f3c540cc8d0 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c540b4ad0 con 0x7f3c5415dfe0 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5aa00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3c540b4dd0 con 0x7f3c5406e160 2024-09-21T06:35:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c58c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c59600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 0x7f3c54153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c58c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45560/0 (socket says 172.21.15.37:45560) 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c58c00640 1 -- 172.21.15.37:0/3421792550 learned_addr learned my addr 172.21.15.37:0/3421792550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.456+0000 7f3c5a000640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5415dfe0 0x7f3c540b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 -- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 msgr2=0x7f3c54153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 0x7f3c54153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 -- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5415dfe0 msgr2=0x7f3c540b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5415dfe0 0x7f3c540b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 -- 172.21.15.37:0/3421792550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c54131240 con 0x7f3c540cc8d0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c59600640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 0x7f3c54153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c58c00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009950 tx=0x7f3c40033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40031d60 con 0x7f3c540cc8d0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c40033e10 con 0x7f3c540cc8d0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c541543e0 con 0x7f3c540cc8d0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40033310 con 0x7f3c540cc8d0 2024-09-21T06:35:57.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.457+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c54155760 con 0x7f3c540cc8d0 2024-09-21T06:35:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.458+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c540b52f0 con 0x7f3c540cc8d0 2024-09-21T06:35:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.460+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c400334b0 con 0x7f3c540cc8d0 2024-09-21T06:35:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.461+0000 7f3c51600640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.461+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f3c400bc2f0 con 0x7f3c540cc8d0 2024-09-21T06:35:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.462+0000 7f3c59600640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.463+0000 7f3c59600640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 0x7f3c3407b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097a0 tx=0x7f3c48009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.465+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c4003f030 con 0x7f3c540cc8d0 2024-09-21T06:35:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.700+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c540cc0e0 con 0x7f3c540cc8d0 2024-09-21T06:35:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.701+0000 7f3c51600640 1 -- 172.21.15.37:0/3421792550 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f3c40083e40 con 0x7f3c540cc8d0 2024-09-21T06:35:57.704 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:57.704 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 msgr2=0x7f3c3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 0x7f3c3407b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097a0 tx=0x7f3c48009260 comp rx=0 tx=0).stop 2024-09-21T06:35:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 msgr2=0x7f3c5415b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009950 tx=0x7f3c40033c20 comp rx=0 tx=0).stop 2024-09-21T06:35:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 shutdown_connections 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c540cc8d0 0x7f3c5415b670 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c5415dfe0 0x7f3c540b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 --2- 172.21.15.37:0/3421792550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c5406e160 0x7f3c54153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.703+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 >> 172.21.15.37:0/3421792550 conn(0x7f3c5406f940 msgr2=0x7f3c540afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.704+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 shutdown_connections 2024-09-21T06:35:57.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.704+0000 7f3c5aa00640 1 -- 172.21.15.37:0/3421792550 wait complete. 2024-09-21T06:35:57.732 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:35:57.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.958+0000 7f56ea600640 1 -- 172.21.15.37:0/3239462910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e406e160 msgr2=0x7f56e406e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.958+0000 7f56ea600640 1 --2- 172.21.15.37:0/3239462910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e406e160 0x7f56e406e570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009860 tx=0x7f56cc02a510 comp rx=0 tx=0).stop 2024-09-21T06:35:57.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.959+0000 7f56ea600640 1 -- 172.21.15.37:0/3239462910 shutdown_connections 2024-09-21T06:35:57.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.959+0000 7f56ea600640 1 --2- 172.21.15.37:0/3239462910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e406e160 0x7f56e406e570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.959+0000 7f56ea600640 1 -- 172.21.15.37:0/3239462910 >> 172.21.15.37:0/3239462910 conn(0x7f56e406f940 msgr2=0x7f56e40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.959+0000 7f56ea600640 1 -- 172.21.15.37:0/3239462910 shutdown_connections 2024-09-21T06:35:57.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.959+0000 7f56ea600640 1 -- 172.21.15.37:0/3239462910 wait complete. 2024-09-21T06:35:57.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.961+0000 7f56ea600640 1 Processor -- start 2024-09-21T06:35:57.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.961+0000 7f56ea600640 1 -- start start 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.962+0000 7f56ea600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56e406e160 0x7f56e4140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.962+0000 7f56ea600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56ea600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 0x7f56e4161c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56ea600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56e4167b60 con 0x7f56e415dfe0 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56ea600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56e41679e0 con 0x7f56e406e160 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56ea600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f56e4167ce0 con 0x7f56e40cc8d0 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56e9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 0x7f56e4161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56e9200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56e406e160 0x7f56e4140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.37:59916/0 (socket says 172.21.15.37:59916) 2024-09-21T06:35:57.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 -- 172.21.15.37:0/2729313035 learned_addr learned my addr 172.21.15.37:0/2729313035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 -- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56e406e160 msgr2=0x7f56e4140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56e406e160 0x7f56e4140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 -- 172.21.15.37:0/2729313035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 msgr2=0x7f56e4161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 0x7f56e4161c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56dbe00640 1 -- 172.21.15.37:0/2729313035 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e40cddc0 con 0x7f56e40cc8d0 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.963+0000 7f56e9c00640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 0x7f56e4161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.964+0000 7f56dbe00640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009950 tx=0x7f56cc033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.968+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc031b50 con 0x7f56e40cc8d0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.968+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56cc031de0 con 0x7f56e40cc8d0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.968+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc005280 con 0x7f56e40cc8d0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.968+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e40ce2c0 con 0x7f56e40cc8d0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.968+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e40ce880 con 0x7f56e40cc8d0 2024-09-21T06:35:57.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.969+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e40ce010 con 0x7f56e40cc8d0 2024-09-21T06:35:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.971+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56cc005420 con 0x7f56e40cc8d0 2024-09-21T06:35:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.972+0000 7f56e1600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 0x7f56c007b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.972+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f56cc0bc090 con 0x7f56e40cc8d0 2024-09-21T06:35:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.972+0000 7f56e9200640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 0x7f56c007b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.973+0000 7f56e9200640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 0x7f56c007b150 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f56d4000f60 tx=0x7f56d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:57.979+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56cc03f030 con 0x7f56e40cc8d0 2024-09-21T06:35:58.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.177+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56e406e570 con 0x7f56e40cc8d0 2024-09-21T06:35:58.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.179+0000 7f56e1600640 1 -- 172.21.15.37:0/2729313035 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17162 (secure 0 0 0) 0x7f56cc083cf0 con 0x7f56e40cc8d0 2024-09-21T06:35:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:58.181 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":106,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:35:40.880959+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:53.201605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:54.169408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.12","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.34","mappings":[{"from":2,"to":3}]},{"pgid":"16.3c","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 msgr2=0x7f56c007b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 0x7f56c007b150 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f56d4000f60 tx=0x7f56d4009040 comp rx=0 tx=0).stop 2024-09-21T06:35:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 msgr2=0x7f56e41616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009950 tx=0x7f56cc033ba0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 shutdown_connections 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f56c0078c40 0x7f56c007b150 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e415dfe0 0x7f56e4161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56e40cc8d0 0x7f56e41616f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 --2- 172.21.15.37:0/2729313035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56e406e160 0x7f56e4140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 >> 172.21.15.37:0/2729313035 conn(0x7f56e406f940 msgr2=0x7f56e4145960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 shutdown_connections 2024-09-21T06:35:58.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.181+0000 7f56ea600640 1 -- 172.21.15.37:0/2729313035 wait complete. 2024-09-21T06:35:58.209 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:35:58.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 -- 172.21.15.37:0/4021253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb0152530 msgr2=0x7f4fb00b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/4021253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb0152530 0x7f4fb00b3490 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009830 tx=0x7f4f9c02a510 comp rx=0 tx=0).stop 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 -- 172.21.15.37:0/4021253 shutdown_connections 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/4021253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb00b39d0 0x7f4fb006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/4021253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb0152530 0x7f4fb00b3490 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 -- 172.21.15.37:0/4021253 >> 172.21.15.37:0/4021253 conn(0x7f4fb006f940 msgr2=0x7f4fb00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 -- 172.21.15.37:0/4021253 shutdown_connections 2024-09-21T06:35:58.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.434+0000 7f4fb4c00640 1 -- 172.21.15.37:0/4021253 wait complete. 2024-09-21T06:35:58.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.436+0000 7f4fb4c00640 1 Processor -- start 2024-09-21T06:35:58.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.436+0000 7f4fb4c00640 1 -- start start 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb0073dd0 0x7f4fb0155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb0152530 0x7f4fb0144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00cf5b0 con 0x7f4fb00b39d0 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00cf430 con 0x7f4fb0152530 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fb4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb00cf730 con 0x7f4fb0073dd0 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45588/0 (socket says 172.21.15.37:45588) 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 -- 172.21.15.37:0/1363782720 learned_addr learned my addr 172.21.15.37:0/1363782720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faf400640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb0073dd0 0x7f4fb0155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4fafe00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb0152530 0x7f4fb0144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 -- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb0073dd0 msgr2=0x7f4fb0155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb0073dd0 0x7f4fb0155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 -- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb0152530 msgr2=0x7f4fb0144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb0152530 0x7f4fb0144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.438+0000 7f4faea00640 1 -- 172.21.15.37:0/1363782720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01568d0 con 0x7f4fb00b39d0 2024-09-21T06:35:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4faea00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02ab00 tx=0x7f4f9c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c005c90 con 0x7f4fb00b39d0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f9c031e50 con 0x7f4fb00b39d0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c005260 con 0x7f4fb00b39d0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb0156bb0 con 0x7f4fb00b39d0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.439+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb0157170 con 0x7f4fb00b39d0 2024-09-21T06:35:58.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.440+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0156640 con 0x7f4fb00b39d0 2024-09-21T06:35:58.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.442+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f9c005400 con 0x7f4fb00b39d0 2024-09-21T06:35:58.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.443+0000 7f4fa7400640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 0x7f4f9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.443+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f4f9c0bbff0 con 0x7f4fb00b39d0 2024-09-21T06:35:58.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.443+0000 7f4faf400640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 0x7f4f9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.444+0000 7f4faf400640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 0x7f4f9007b0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000f60 tx=0x7f4fa0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.447+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f9c03f030 con 0x7f4fb00b39d0 2024-09-21T06:35:58.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.682+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fb00cfb50 con 0x7f4fb00b39d0 2024-09-21T06:35:58.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.683+0000 7f4fa7400640 1 -- 172.21.15.37:0/1363782720 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7f4f9c083c30 con 0x7f4fb00b39d0 2024-09-21T06:35:58.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:58.686 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:58.686 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:58.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 msgr2=0x7f4f9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 0x7f4f9007b0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000f60 tx=0x7f4fa0009040 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 msgr2=0x7f4fb0157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02ab00 tx=0x7f4f9c033c20 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 shutdown_connections 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4f90078b90 0x7f4f9007b0a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb00b39d0 0x7f4fb0157b80 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb0152530 0x7f4fb0144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 --2- 172.21.15.37:0/1363782720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fb0073dd0 0x7f4fb0155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 >> 172.21.15.37:0/1363782720 conn(0x7f4fb006f940 msgr2=0x7f4fb00cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 shutdown_connections 2024-09-21T06:35:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.686+0000 7f4fb4c00640 1 -- 172.21.15.37:0/1363782720 wait complete. 2024-09-21T06:35:58.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:35:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 -- 172.21.15.37:0/4273044844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb08003ff0 msgr2=0x7fbb08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 --2- 172.21.15.37:0/4273044844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb08003ff0 0x7fbb08004800 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fbb20075910 tx=0x7fbb08005830 comp rx=0 tx=0).stop 2024-09-21T06:35:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb26400640 1 -- 172.21.15.37:0/4273044844 reap_dead start 2024-09-21T06:35:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 -- 172.21.15.37:0/4273044844 shutdown_connections 2024-09-21T06:35:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 -- 172.21.15.37:0/4273044844 >> 172.21.15.37:0/4273044844 conn(0x7fbb200d2350 msgr2=0x7fbb200d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 -- 172.21.15.37:0/4273044844 shutdown_connections 2024-09-21T06:35:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.937+0000 7fbb28400640 1 -- 172.21.15.37:0/4273044844 wait complete. 2024-09-21T06:35:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.939+0000 7fbb28400640 1 Processor -- start 2024-09-21T06:35:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.939+0000 7fbb28400640 1 -- start start 2024-09-21T06:35:58.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb08003ff0 0x7fbb201f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb200cce10 0x7fbb200cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20160c20 con 0x7fbb20160510 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200b7470 con 0x7fbb200cce10 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb28400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbb20160da0 con 0x7fbb08003ff0 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45600/0 (socket says 172.21.15.37:45600) 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 -- 172.21.15.37:0/2323629296 learned_addr learned my addr 172.21.15.37:0/2323629296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb26400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb200cce10 0x7fbb200cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25a00640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb08003ff0 0x7fbb201f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 -- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb08003ff0 msgr2=0x7fbb201f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb08003ff0 0x7fbb201f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 -- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb200cce10 msgr2=0x7fbb200cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb200cce10 0x7fbb200cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.941+0000 7fbb25000640 1 -- 172.21.15.37:0/2323629296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb200cd760 con 0x7fbb20160510 2024-09-21T06:35:58.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb25000640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02aef0 tx=0x7fbb0c033f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c046070 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb0c004030 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb20150450 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004460 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.942+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2006e270 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.943+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2015cf20 con 0x7fbb20160510 2024-09-21T06:35:58.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.945+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb0c049070 con 0x7fbb20160510 2024-09-21T06:35:58.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.946+0000 7fbb1d600640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 0x7fbb0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.946+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7fbb0c0bb200 con 0x7fbb20160510 2024-09-21T06:35:58.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.947+0000 7fbb25a00640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 0x7fbb0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.948+0000 7fbb25a00640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 0x7fbb0007b0d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbb140097f0 tx=0x7fbb14009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:58.951+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb200cd760 con 0x7fbb20160510 2024-09-21T06:35:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.186+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb200b5660 con 0x7fbb20160510 2024-09-21T06:35:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.186+0000 7fbb1d600640 1 -- 172.21.15.37:0/2323629296 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5281 (secure 0 0 0) 0x7fbb0c082d50 con 0x7fbb20160510 2024-09-21T06:35:59.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:59.190 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":118,"btime":"2024-09-21T06:35:39:874944+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:55.216486+0000","modified":"2024-09-21T06:35:39.744919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8175,"mds_1":8172,"mds_2":7079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7079":{"gid":7079,"name":"b","rank":2,"incarnation":112,"state":"up:active","state_seq":14,"addr":"172.21.15.150:6837/1287330873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":1287330873},{"type":"v1","addr":"172.21.15.150:6837","nonce":1287330873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8172":{"gid":8172,"name":"c","rank":1,"incarnation":110,"state":"up:active","state_seq":13,"addr":"172.21.15.37:6833/1950391294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1950391294},{"type":"v1","addr":"172.21.15.37:6833","nonce":1950391294}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":14,"addr":"172.21.15.37:6835/912339030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":912339030},{"type":"v1","addr":"172.21.15.37:6835","nonce":912339030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175,8172,7079]},"id":7}]} 2024-09-21T06:35:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 118 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 msgr2=0x7fbb0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 0x7fbb0007b0d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbb140097f0 tx=0x7fbb14009500 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 msgr2=0x7fbb2014ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02aef0 tx=0x7fbb0c033f10 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 shutdown_connections 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbb00078bc0 0x7fbb0007b0d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb20160510 0x7fbb2014ff10 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb200cce10 0x7fbb200cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 --2- 172.21.15.37:0/2323629296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb08003ff0 0x7fbb201f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 >> 172.21.15.37:0/2323629296 conn(0x7fbb200d2350 msgr2=0x7fbb20152320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 shutdown_connections 2024-09-21T06:35:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.189+0000 7fbb28400640 1 -- 172.21.15.37:0/2323629296 wait complete. 2024-09-21T06:35:59.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:35:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 -- 172.21.15.37:0/3929656434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e4162180 msgr2=0x7f63e4162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 --2- 172.21.15.37:0/3929656434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e4162180 0x7f63e4162590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc02a840 comp rx=0 tx=0).stop 2024-09-21T06:35:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 -- 172.21.15.37:0/3929656434 shutdown_connections 2024-09-21T06:35:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 --2- 172.21.15.37:0/3929656434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e4162180 0x7f63e4162590 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 --2- 172.21.15.37:0/3929656434 >> v2:172.21.15.150:3300/0 conn(0x7f63e40d3ab0 0x7f63e406e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.441+0000 7f63e9600640 1 -- 172.21.15.37:0/3929656434 >> 172.21.15.37:0/3929656434 conn(0x7f63e40d0d40 msgr2=0x7f63e40d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.442+0000 7f63e9600640 1 -- 172.21.15.37:0/3929656434 shutdown_connections 2024-09-21T06:35:59.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.442+0000 7f63e9600640 1 -- 172.21.15.37:0/3929656434 wait complete. 2024-09-21T06:35:59.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.444+0000 7f63e9600640 1 Processor -- start 2024-09-21T06:35:59.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.444+0000 7f63e9600640 1 -- start start 2024-09-21T06:35:59.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40d3ab0 0x7f63e4160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e4162d20 0x7f63e40ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40b7af0 con 0x7f63e4162180 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40b7970 con 0x7f63e4162d20 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63e9600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40b7c70 con 0x7f63e40d3ab0 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.445+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45616/0 (socket says 172.21.15.37:45616) 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 -- 172.21.15.37:0/895811392 learned_addr learned my addr 172.21.15.37:0/895811392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63e3e00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40d3ab0 0x7f63e4160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63e8c00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e4162d20 0x7f63e40ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 -- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40d3ab0 msgr2=0x7f63e4160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40d3ab0 0x7f63e4160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 -- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e4162d20 msgr2=0x7f63e40ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e4162d20 0x7f63e40ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 -- 172.21.15.37:0/895811392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41f54a0 con 0x7f63e4162180 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.446+0000 7f63dbe00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e7f0 tx=0x7f63cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.447+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00eea0 con 0x7f63e4162180 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.447+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc00cdb0 con 0x7f63e4162180 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.447+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc010690 con 0x7f63e4162180 2024-09-21T06:35:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.447+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41f5780 con 0x7f63e4162180 2024-09-21T06:35:59.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.447+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41f5d40 con 0x7f63e4162180 2024-09-21T06:35:59.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.448+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e40b81d0 con 0x7f63e4162180 2024-09-21T06:35:59.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.449+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63cc010830 con 0x7f63e4162180 2024-09-21T06:35:59.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.450+0000 7f63e0c00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 0x7f63c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.450+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f63cc09b8f0 con 0x7f63e4162180 2024-09-21T06:35:59.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.450+0000 7f63e3e00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 0x7f63c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.451+0000 7f63e3e00640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 0x7f63c007b0a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007960 tx=0x7f63d4001290 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.454+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63cc063440 con 0x7f63e4162180 2024-09-21T06:35:59.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.652+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63e40d12f0 con 0x7f63e4162180 2024-09-21T06:35:59.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.653+0000 7f63e0c00640 1 -- 172.21.15.37:0/895811392 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17162 (secure 0 0 0) 0x7f63cc067e70 con 0x7f63e4162180 2024-09-21T06:35:59.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:35:59.656 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":106,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:35:40.880959+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:53.201605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:54.169408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.12","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.34","mappings":[{"from":2,"to":3}]},{"pgid":"16.3c","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.655+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 msgr2=0x7f63c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.655+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 0x7f63c007b0a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007960 tx=0x7f63d4001290 comp rx=0 tx=0).stop 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.655+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 msgr2=0x7f63e40abfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.655+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e7f0 tx=0x7f63cc00ecb0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 shutdown_connections 2024-09-21T06:35:59.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63c0078b90 0x7f63c007b0a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63e4162180 0x7f63e40abfd0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e4162d20 0x7f63e40ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 --2- 172.21.15.37:0/895811392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40d3ab0 0x7f63e4160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 >> 172.21.15.37:0/895811392 conn(0x7f63e40d0d40 msgr2=0x7f63e40cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 shutdown_connections 2024-09-21T06:35:59.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.656+0000 7f63e9600640 1 -- 172.21.15.37:0/895811392 wait complete. 2024-09-21T06:35:59.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 -- 172.21.15.37:0/308453313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 msgr2=0x7f0be00702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 --2- 172.21.15.37:0/308453313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be00702a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009830 tx=0x7f0bd002a520 comp rx=0 tx=0).stop 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 -- 172.21.15.37:0/308453313 shutdown_connections 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 --2- 172.21.15.37:0/308453313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be00702a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 -- 172.21.15.37:0/308453313 >> 172.21.15.37:0/308453313 conn(0x7f0be00d05c0 msgr2=0x7f0be006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 -- 172.21.15.37:0/308453313 shutdown_connections 2024-09-21T06:35:59.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.912+0000 7f0be4c00640 1 -- 172.21.15.37:0/308453313 wait complete. 2024-09-21T06:35:59.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.915+0000 7f0be4c00640 1 Processor -- start 2024-09-21T06:35:59.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.915+0000 7f0be4c00640 1 -- start start 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0be00cf6d0 0x7f0be013ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be01f8810 0x7f0be01416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b3e60 con 0x7f0be00cc0e0 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b3ce0 con 0x7f0be01f8810 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0be4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00b3fe0 con 0x7f0be00cf6d0 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.916+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45644/0 (socket says 172.21.15.37:45644) 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 -- 172.21.15.37:0/2568143730 learned_addr learned my addr 172.21.15.37:0/2568143730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdea00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0be00cf6d0 0x7f0be013ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdfe00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be01f8810 0x7f0be01416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 -- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0be00cf6d0 msgr2=0x7f0be013ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0be00cf6d0 0x7f0be013ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 -- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be01f8810 msgr2=0x7f0be01416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be01f8810 0x7f0be01416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 -- 172.21.15.37:0/2568143730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01f9420 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.917+0000 7f0bdf400640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000c00 tx=0x7f0bd0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.918+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0031d60 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.918+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd00330d0 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.918+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01f9700 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.918+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0033520 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.918+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01f5610 con 0x7f0be00cc0e0 2024-09-21T06:35:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.919+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be00702a0 con 0x7f0be00cc0e0 2024-09-21T06:35:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.920+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bd00035f0 con 0x7f0be00cc0e0 2024-09-21T06:35:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.921+0000 7f0bd7400640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 0x7f0bc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.921+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6217+0+0 (secure 0 0 0) 0x7f0bd00bc580 con 0x7f0be00cc0e0 2024-09-21T06:35:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.922+0000 7f0bdea00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 0x7f0bc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.922+0000 7f0bdea00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 0x7f0bc007b0a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0be0143ca0 tx=0x7f0bcc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:35:59.925+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bd003f030 con 0x7f0be00cc0e0 2024-09-21T06:36:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.162+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0be0144a10 con 0x7f0be00cc0e0 2024-09-21T06:36:00.165 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-21T06:36:00.163+0000 7f510fc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:36:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.165+0000 7f0bd7400640 1 -- 172.21.15.37:0/2568143730 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) ==== 161+0+0 (secure 0 0 0) 0x7f0bd00840d0 con 0x7f0be00cc0e0 2024-09-21T06:36:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:36:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 msgr2=0x7f0bc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 0x7f0bc007b0a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0be0143ca0 tx=0x7f0bcc009040 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 msgr2=0x7f0be01f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000c00 tx=0x7f0bd0033d60 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 shutdown_connections 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f0bc0078b90 0x7f0bc007b0a0 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0be0143ca0 tx=0x7f0bcc009040 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be01f8810 0x7f0be01416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0be00cf6d0 0x7f0be013ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 --2- 172.21.15.37:0/2568143730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0be00cc0e0 0x7f0be01f5e10 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 >> 172.21.15.37:0/2568143730 conn(0x7f0be00d05c0 msgr2=0x7f0be015f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 shutdown_connections 2024-09-21T06:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.168+0000 7f0be4c00640 1 -- 172.21.15.37:0/2568143730 wait complete. 2024-09-21T06:36:00.184 INFO:tasks.ceph.mds.b.smithi150.stderr: -1515> 2024-09-21T06:35:31.442+0000 7f18fae00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1) n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x561b018f5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.646941+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.184 INFO:tasks.ceph.mds.b.smithi150.stderr: -1500> 2024-09-21T06:35:31.442+0000 7f18fae00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/AsLsXsFsx@2},l=8349 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x561b0166c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.185 INFO:tasks.ceph.mds.c.smithi037.stderr: -1623> 2024-09-21T06:35:31.441+0000 7f4147800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1) n(v2 rc2024-09-21T06:35:14.958937+0000 4=0+4) (inest lock) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x556ca3bbf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-21T06:35:14.958937+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.185 INFO:tasks.ceph.mds.c.smithi037.stderr: -1471> 2024-09-21T06:35:31.444+0000 7f414fa00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1) n(v1 rc2024-09-21T06:35:14.958937+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={8349=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x556ca3c18b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.646941+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.645941+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-21T06:35:14.958937+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.189 INFO:tasks.ceph.mds.a.smithi037.stderr: -3528> 2024-09-21T06:35:31.441+0000 7f3a25800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1) n(v3 rc2024-09-21T06:35:14.958937+0000 6=0+6) (inest mix dirty) (ifile excl) caps={8349=pAsLsXsFsx/pAsLsXsFsxcral@4},l=8349 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55f942027080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.643941+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-21T06:35:14.958937+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.189 INFO:tasks.ceph.mds.a.smithi037.stderr: -3393> 2024-09-21T06:35:31.443+0000 7f3a2da00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1) n(v4 rc2024-09-21T06:35:14.958937+0000 5=0+5) (inest sync->mix g=1,2) caps={8349=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55f942027600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:35:14.958937+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-21T06:35:14.644941+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-21T06:35:14.958937+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-21T06:36:00.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:36:00.303 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-09-21T06:36:00.308 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-21T06:36:00.315 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 -- 172.21.15.37:0/2569472441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb101566f0 msgr2=0x7fcb10156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 --2- 172.21.15.37:0/2569472441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb101566f0 0x7fcb10156b00 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcb10070820 tx=0x7fcaf802a560 comp rx=0 tx=0).stop 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 -- 172.21.15.37:0/2569472441 shutdown_connections 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 --2- 172.21.15.37:0/2569472441 >> v2:172.21.15.150:3300/0 conn(0x7fcb1006fca0 0x7fcb100700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 --2- 172.21.15.37:0/2569472441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb101566f0 0x7fcb10156b00 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 --2- 172.21.15.37:0/2569472441 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb100cba70 0x7fcb1015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 -- 172.21.15.37:0/2569472441 >> 172.21.15.37:0/2569472441 conn(0x7fcb100cd570 msgr2=0x7fcb100cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:00.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 -- 172.21.15.37:0/2569472441 shutdown_connections 2024-09-21T06:36:00.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.434+0000 7fcb0f200640 1 -- 172.21.15.37:0/2569472441 wait complete. 2024-09-21T06:36:00.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.437+0000 7fcb0f200640 1 Processor -- start 2024-09-21T06:36:00.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.437+0000 7fcb0f200640 1 -- start start 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006fca0 0x7fcb101f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101566f0 0x7fcb10154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100af9e0 con 0x7fcb100cba70 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100af860 con 0x7fcb101566f0 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb0f200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcb100afb60 con 0x7fcb1006fca0 2024-09-21T06:36:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.438+0000 7fcb05400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45700/0 (socket says 172.21.15.37:45700) 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 -- 172.21.15.37:0/781310935 learned_addr learned my addr 172.21.15.37:0/781310935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb0de00640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006fca0 0x7fcb101f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb0e800640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101566f0 0x7fcb10154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 -- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006fca0 msgr2=0x7fcb101f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006fca0 0x7fcb101f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 -- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101566f0 msgr2=0x7fcb10154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101566f0 0x7fcb10154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 -- 172.21.15.37:0/781310935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb100a9450 con 0x7fcb100cba70 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.439+0000 7fcb05400640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034040 tx=0x7fcaf80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.440+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf803e030 con 0x7fcb100cba70 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.440+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf8003ca0 con 0x7fcb100cba70 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.440+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101f40f0 con 0x7fcb100cba70 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.440+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8035040 con 0x7fcb100cba70 2024-09-21T06:36:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.440+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1015a260 con 0x7fcb100cba70 2024-09-21T06:36:00.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.441+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1015d3b0 con 0x7fcb100cba70 2024-09-21T06:36:00.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.442+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcaf8041030 con 0x7fcb100cba70 2024-09-21T06:36:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.443+0000 7fcb06a00640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 0x7fcaec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.443+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6475+0+0 (secure 0 0 0) 0x7fcaf80bb770 con 0x7fcb100cba70 2024-09-21T06:36:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.443+0000 7fcb0de00640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 0x7fcaec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.444+0000 7fcb0de00640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 0x7fcaec07b0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005fd0 tx=0x7fcb00009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.448+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaf80831c0 con 0x7fcb100cba70 2024-09-21T06:36:00.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:00.686+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcb10159ff0 con 0x7fcb100cba70 2024-09-21T06:36:01.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.167+0000 7fcb06a00640 1 -- 172.21.15.37:0/781310935 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) ==== 104+0+0 (secure 0 0 0) 0x7fcaf8087bf0 con 0x7fcb100cba70 2024-09-21T06:36:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 msgr2=0x7fcaec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 0x7fcaec07b0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005fd0 tx=0x7fcb00009500 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 msgr2=0x7fcb101f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034040 tx=0x7fcaf80038a0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 shutdown_connections 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcaec078b90 0x7fcaec07b0a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb100cba70 0x7fcb101f3bb0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101566f0 0x7fcb10154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 --2- 172.21.15.37:0/781310935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006fca0 0x7fcb101f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 >> 172.21.15.37:0/781310935 conn(0x7fcb100cd570 msgr2=0x7fcb101545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 shutdown_connections 2024-09-21T06:36:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.170+0000 7fcb0f200640 1 -- 172.21.15.37:0/781310935 wait complete. 2024-09-21T06:36:01.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:36:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 -- 172.21.15.37:0/3114789575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68162690 msgr2=0x7f4a6815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 --2- 172.21.15.37:0/3114789575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68162690 0x7f4a6815eb80 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f4a680d5110 tx=0x7f4a6002a980 comp rx=0 tx=0).stop 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 -- 172.21.15.37:0/3114789575 shutdown_connections 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 --2- 172.21.15.37:0/3114789575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68162690 0x7f4a6815eb80 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 --2- 172.21.15.37:0/3114789575 >> v2:172.21.15.150:3300/0 conn(0x7f4a6806d7f0 0x7f4a68162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 --2- 172.21.15.37:0/3114789575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a68155720 0x7f4a6806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 -- 172.21.15.37:0/3114789575 >> 172.21.15.37:0/3114789575 conn(0x7f4a680d1900 msgr2=0x7f4a680d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 -- 172.21.15.37:0/3114789575 shutdown_connections 2024-09-21T06:36:01.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.430+0000 7f4a6e600640 1 -- 172.21.15.37:0/3114789575 wait complete. 2024-09-21T06:36:01.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.432+0000 7f4a6e600640 1 Processor -- start 2024-09-21T06:36:01.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.432+0000 7f4a6e600640 1 -- start start 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6806d7f0 0x7f4a680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68162690 0x7f4a6815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6815f450 con 0x7f4a68155720 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6815f2d0 con 0x7f4a68162690 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a6e600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6815f5d0 con 0x7f4a6806d7f0 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45712/0 (socket says 172.21.15.37:45712) 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.434+0000 7f4a67e00640 1 -- 172.21.15.37:0/211244448 learned_addr learned my addr 172.21.15.37:0/211244448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a6d200640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6806d7f0 0x7f4a680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a6dc00640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68162690 0x7f4a6815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 -- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6806d7f0 msgr2=0x7f4a680ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6806d7f0 0x7f4a680ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 -- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68162690 msgr2=0x7f4a6815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68162690 0x7f4a6815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 -- 172.21.15.37:0/211244448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68147290 con 0x7f4a68155720 2024-09-21T06:36:01.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a67e00640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009fd0 tx=0x7f4a5400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400ccc0 con 0x7f4a68155720 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a5400ee70 con 0x7f4a68155720 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400ce60 con 0x7f4a68155720 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.435+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a68147570 con 0x7f4a68155720 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.436+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a68147b30 con 0x7f4a68155720 2024-09-21T06:36:01.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.437+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a68147000 con 0x7f4a68155720 2024-09-21T06:36:01.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.438+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a54005260 con 0x7f4a68155720 2024-09-21T06:36:01.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.439+0000 7f4a64c00640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 0x7f4a4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:01.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.439+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6475+0+0 (secure 0 0 0) 0x7f4a5409b920 con 0x7f4a68155720 2024-09-21T06:36:01.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.440+0000 7f4a6d200640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 0x7f4a4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:01.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.440+0000 7f4a6d200640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 0x7f4a4807b0d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009a40 tx=0x7f4a58005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.444+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a54063ae0 con 0x7f4a68155720 2024-09-21T06:36:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:01.642+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a680d0090 con 0x7f4a68155720 2024-09-21T06:36:02.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.173+0000 7f4a64c00640 1 -- 172.21.15.37:0/211244448 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f4a680d0090 con 0x7f4a68155720 2024-09-21T06:36:02.175 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:36:02.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 msgr2=0x7f4a4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:02.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 0x7f4a4807b0d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009a40 tx=0x7f4a58005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:02.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 msgr2=0x7f4a6815faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:02.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009fd0 tx=0x7f4a5400ec20 comp rx=0 tx=0).stop 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 shutdown_connections 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f4a48078bc0 0x7f4a4807b0d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68155720 0x7f4a6815faa0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68162690 0x7f4a6815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 --2- 172.21.15.37:0/211244448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6806d7f0 0x7f4a680ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.177+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 >> 172.21.15.37:0/211244448 conn(0x7f4a680d1900 msgr2=0x7f4a680b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.178+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 shutdown_connections 2024-09-21T06:36:02.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.178+0000 7f4a6e600640 1 -- 172.21.15.37:0/211244448 wait complete. 2024-09-21T06:36:02.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:36:02.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 -- 172.21.15.37:0/1230605910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 msgr2=0x7f61f8160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 --2- 172.21.15.37:0/1230605910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f8160100 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f61ec005fd0 tx=0x7f61ec02a5d0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 -- 172.21.15.37:0/1230605910 shutdown_connections 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 --2- 172.21.15.37:0/1230605910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f8160640 0x7f61f8160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 --2- 172.21.15.37:0/1230605910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f8160100 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 -- 172.21.15.37:0/1230605910 >> 172.21.15.37:0/1230605910 conn(0x7f61f80ccac0 msgr2=0x7f61f80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 -- 172.21.15.37:0/1230605910 shutdown_connections 2024-09-21T06:36:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.437+0000 7f61ff400640 1 -- 172.21.15.37:0/1230605910 wait complete. 2024-09-21T06:36:02.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.439+0000 7f61ff400640 1 Processor -- start 2024-09-21T06:36:02.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.439+0000 7f61ff400640 1 -- start start 2024-09-21T06:36:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f806ef30 0x7f61f81f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f8160640 0x7f61f81e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61f806fa50 con 0x7f61f80ca690 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61f806e500 con 0x7f61f8160640 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61ff400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f61f806fbd0 con 0x7f61f806ef30 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45728/0 (socket says 172.21.15.37:45728) 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61f7e00640 1 -- 172.21.15.37:0/3620906916 learned_addr learned my addr 172.21.15.37:0/3620906916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61fca00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f806ef30 0x7f61f81f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.441+0000 7f61fd400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f8160640 0x7f61f81e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 -- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f806ef30 msgr2=0x7f61f81f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f806ef30 0x7f61f81f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 -- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f8160640 msgr2=0x7f61f81e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f8160640 0x7f61f81e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 -- 172.21.15.37:0/3620906916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80c69c0 con 0x7f61f80ca690 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f7e00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f61e400d6d0 tx=0x7f61e400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4017070 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e4005330 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f80b5af0 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4009820 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.442+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f80c6650 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.444+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f8160100 con 0x7f61f80ca690 2024-09-21T06:36:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.445+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61e4009bd0 con 0x7f61f80ca690 2024-09-21T06:36:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.446+0000 7f61f4c00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 0x7f61d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.446+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6037+0+0 (secure 0 0 0) 0x7f61e409b0f0 con 0x7f61f80ca690 2024-09-21T06:36:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.447+0000 7f61fca00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 0x7f61d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.447+0000 7f61fca00640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 0x7f61d807b0d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61ec033000 tx=0x7f61ec004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:02.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.451+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61e4062d80 con 0x7f61f80ca690 2024-09-21T06:36:02.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:02.648+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f61f80b5f20 con 0x7f61f80ca690 2024-09-21T06:36:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.175+0000 7f61f4c00640 1 -- 172.21.15.37:0/3620906916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f61e40677b0 con 0x7f61f80ca690 2024-09-21T06:36:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.177+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 msgr2=0x7f61d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.177+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 0x7f61d807b0d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61ec033000 tx=0x7f61ec004730 comp rx=0 tx=0).stop 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.177+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 msgr2=0x7f61f81e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.177+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f61e400d6d0 tx=0x7f61e400db90 comp rx=0 tx=0).stop 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 shutdown_connections 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f61d8078bc0 0x7f61d807b0d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61f80ca690 0x7f61f81e7e90 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f8160640 0x7f61f81e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 --2- 172.21.15.37:0/3620906916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f806ef30 0x7f61f81f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 >> 172.21.15.37:0/3620906916 conn(0x7f61f80ccac0 msgr2=0x7f61f80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 shutdown_connections 2024-09-21T06:36:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.178+0000 7f61ff400640 1 -- 172.21.15.37:0/3620906916 wait complete. 2024-09-21T06:36:03.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 -- 172.21.15.37:0/2148743560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 msgr2=0x7f21c40ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 --2- 172.21.15.37:0/2148743560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c40ade20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f21c40af710 tx=0x7f21b402a540 comp rx=0 tx=0).stop 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 -- 172.21.15.37:0/2148743560 shutdown_connections 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 --2- 172.21.15.37:0/2148743560 >> v2:172.21.15.150:3300/0 conn(0x7f21c4162880 0x7f21c406fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 --2- 172.21.15.37:0/2148743560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4073a00 0x7f21c4162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 --2- 172.21.15.37:0/2148743560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c40ade20 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.432+0000 7f21c9000640 1 -- 172.21.15.37:0/2148743560 >> 172.21.15.37:0/2148743560 conn(0x7f21c40d05c0 msgr2=0x7f21c406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.433+0000 7f21c9000640 1 -- 172.21.15.37:0/2148743560 shutdown_connections 2024-09-21T06:36:03.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.433+0000 7f21c9000640 1 -- 172.21.15.37:0/2148743560 wait complete. 2024-09-21T06:36:03.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.435+0000 7f21c9000640 1 Processor -- start 2024-09-21T06:36:03.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.436+0000 7f21c9000640 1 -- start start 2024-09-21T06:36:03.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c4073a00 0x7f21c41f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4162880 0x7f21c413c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4070750 con 0x7f21c40aef80 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21c40705d0 con 0x7f21c4073a00 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c9000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f21c41550c0 con 0x7f21c4162880 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c4073a00 0x7f21c41f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45754/0 (socket says 172.21.15.37:45754) 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 -- 172.21.15.37:0/3032782698 learned_addr learned my addr 172.21.15.37:0/3032782698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:03.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c3e00640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4162880 0x7f21c413c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 -- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4162880 msgr2=0x7f21c413c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4162880 0x7f21c413c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 -- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c4073a00 msgr2=0x7f21c41f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c4073a00 0x7f21c41f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 -- 172.21.15.37:0/3032782698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c41e9f40 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.438+0000 7f21c2a00640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c940 tx=0x7f21b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.439+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0007bb0 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.439+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b0007e40 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.439+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c41e80d0 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.439+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b000f670 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.439+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c4155a10 con 0x7f21c40aef80 2024-09-21T06:36:03.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.440+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c41603b0 con 0x7f21c40aef80 2024-09-21T06:36:03.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.442+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21b000f810 con 0x7f21c40aef80 2024-09-21T06:36:03.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.442+0000 7f21bb400640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.442+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f21b009b2b0 con 0x7f21c40aef80 2024-09-21T06:36:03.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.443+0000 7f21c3400640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.443+0000 7f21c3400640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 0x7f21a407b0a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f21b4033000 tx=0x7f21b4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.446+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21c41603b0 con 0x7f21c40aef80 2024-09-21T06:36:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.647+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f21c40ade20 con 0x7f21c40aef80 2024-09-21T06:36:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.652+0000 7f21bb400640 1 -- 172.21.15.37:0/3032782698 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f21b0014030 con 0x7f21c40aef80 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 msgr2=0x7f21a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 0x7f21a407b0a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f21b4033000 tx=0x7f21b4004730 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 msgr2=0x7f21c4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c940 tx=0x7f21b000ce00 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 shutdown_connections 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c40aef80 0x7f21c4158750 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21c4162880 0x7f21c413c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 --2- 172.21.15.37:0/3032782698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c4073a00 0x7f21c41f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 >> 172.21.15.37:0/3032782698 conn(0x7f21c40d05c0 msgr2=0x7f21c415fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 shutdown_connections 2024-09-21T06:36:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.654+0000 7f21c9000640 1 -- 172.21.15.37:0/3032782698 wait complete. 2024-09-21T06:36:03.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 -- 172.21.15.37:0/2428356559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c015b900 msgr2=0x7ff5c006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/2428356559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c015b900 0x7ff5c006e720 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0073490 tx=0x7ff5ac02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 -- 172.21.15.37:0/2428356559 shutdown_connections 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/2428356559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c015b900 0x7ff5c006e720 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/2428356559 >> v2:172.21.15.150:3300/0 conn(0x7ff5c0072320 0x7ff5c0152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.906+0000 7ff5c6a00640 1 -- 172.21.15.37:0/2428356559 >> 172.21.15.37:0/2428356559 conn(0x7ff5c00d1020 msgr2=0x7ff5c00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.907+0000 7ff5c6a00640 1 -- 172.21.15.37:0/2428356559 shutdown_connections 2024-09-21T06:36:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.907+0000 7ff5c6a00640 1 -- 172.21.15.37:0/2428356559 wait complete. 2024-09-21T06:36:03.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.910+0000 7ff5c6a00640 1 Processor -- start 2024-09-21T06:36:03.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.910+0000 7ff5c6a00640 1 -- start start 2024-09-21T06:36:03.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5c0071160 0x7ff5c0145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c015b900 0x7ff5c015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00b40d0 con 0x7ff5c0072320 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00b3f50 con 0x7ff5c015b900 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00b4250 con 0x7ff5c0071160 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50624/0 (socket says 172.21.15.37:50624) 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 -- 172.21.15.37:0/3937491550 learned_addr learned my addr 172.21.15.37:0/3937491550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c5600640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5c0071160 0x7ff5c0145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c6000640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c015b900 0x7ff5c015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 -- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5c0071160 msgr2=0x7ff5c0145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5c0071160 0x7ff5c0145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 -- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c015b900 msgr2=0x7ff5c015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c015b900 0x7ff5c015aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.911+0000 7ff5c4c00640 1 -- 172.21.15.37:0/3937491550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c01e3b40 con 0x7ff5c0072320 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5c4c00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0098f0 tx=0x7ff5ac033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac031d60 con 0x7ff5c0072320 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5ac033e10 con 0x7ff5c0072320 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac005260 con 0x7ff5c0072320 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c01e3e20 con 0x7ff5c0072320 2024-09-21T06:36:03.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.912+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01e43e0 con 0x7ff5c0072320 2024-09-21T06:36:03.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.913+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c01e38b0 con 0x7ff5c0072320 2024-09-21T06:36:03.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.914+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5ac005400 con 0x7ff5c0072320 2024-09-21T06:36:03.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.915+0000 7ff5bd600640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.915+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7ff5ac0bbe70 con 0x7ff5c0072320 2024-09-21T06:36:03.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.916+0000 7ff5c5600640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.918+0000 7ff5c5600640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 0x7ff5a007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400bb50 tx=0x7ff5b400b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:03.920+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5ac03f030 con 0x7ff5c0072320 2024-09-21T06:36:04.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:04.188+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7ff5c00b4770 con 0x7ff5c0072320 2024-09-21T06:36:05.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.194+0000 7ff5bd600640 1 -- 172.21.15.37:0/3937491550 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v434) ==== 165+0+0 (secure 0 0 0) 0x7ff5ac005d00 con 0x7ff5c0072320 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 msgr2=0x7ff5a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 0x7ff5a007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400bb50 tx=0x7ff5b400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 msgr2=0x7ff5c015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0098f0 tx=0x7ff5ac033c20 comp rx=0 tx=0).stop 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 shutdown_connections 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5c0072320 0x7ff5c015a570 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c015b900 0x7ff5c015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 --2- 172.21.15.37:0/3937491550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5c0071160 0x7ff5c0145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 >> 172.21.15.37:0/3937491550 conn(0x7ff5c00d1020 msgr2=0x7ff5c00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:05.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 shutdown_connections 2024-09-21T06:36:05.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:05.197+0000 7ff5c6a00640 1 -- 172.21.15.37:0/3937491550 wait complete. 2024-09-21T06:36:05.223 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-21T06:36:05.224 INFO:tasks.cephfs_test_runner: 2024-09-21T06:36:05.224 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T06:36:05.224 INFO:tasks.cephfs_test_runner:Ran 6 tests in 440.217s 2024-09-21T06:36:05.224 INFO:tasks.cephfs_test_runner: 2024-09-21T06:36:05.224 INFO:tasks.cephfs_test_runner:OK 2024-09-21T06:36:05.224 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T06:36:05.235 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T06:36:05.248 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T06:36:05.248 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:36:05.248 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:36:05.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:36:05.287 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-21T06:36:05.287 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:36:05.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:36:05.363 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:36:05.364 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:36:05.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:36:05.404 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T06:36:05.404 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:36:05.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:36:05.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:05.483 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:36:05.519 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:36:05.520 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:36:05.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:05.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:05.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:05.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:36:05.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:05.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-21T06:36:06.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.164 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-21T06:36:06.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-21T06:36:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-09-21T06:36:06.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:06.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.304 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:36:06.344 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T06:36:06.344 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T06:36:06.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.344 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T06:36:06.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T06:36:06.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:06.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-09-21T06:36:07.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:07.011 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-09-21T06:36:07.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:07.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-09-21T06:36:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-09-21T06:36:07.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:07.150 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T06:36:07.160 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:36:07.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:36:07.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 -- 172.21.15.37:0/1354931254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 msgr2=0x7f3e2406def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 --2- 172.21.15.37:0/1354931254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2406def0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3e2406ea80 tx=0x7f3e2002aa30 comp rx=0 tx=0).stop 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 -- 172.21.15.37:0/1354931254 shutdown_connections 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 --2- 172.21.15.37:0/1354931254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2406def0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 --2- 172.21.15.37:0/1354931254 >> v2:172.21.15.150:3300/0 conn(0x7f3e24152530 0x7f3e240b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 -- 172.21.15.37:0/1354931254 >> 172.21.15.37:0/1354931254 conn(0x7f3e2406f940 msgr2=0x7f3e240b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 -- 172.21.15.37:0/1354931254 shutdown_connections 2024-09-21T06:36:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.402+0000 7f3e2b200640 1 -- 172.21.15.37:0/1354931254 wait complete. 2024-09-21T06:36:07.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.404+0000 7f3e2b200640 1 Processor -- start 2024-09-21T06:36:07.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.404+0000 7f3e2b200640 1 -- start start 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e24073dd0 0x7f3e2414df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 0x7f3e241f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e240cf5f0 con 0x7f3e240b39d0 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e240cf470 con 0x7f3e24152530 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3e240cf770 con 0x7f3e24073dd0 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e29400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e29400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50632/0 (socket says 172.21.15.37:50632) 2024-09-21T06:36:07.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e29400640 1 -- 172.21.15.37:0/4097229624 learned_addr learned my addr 172.21.15.37:0/4097229624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e2a800640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 0x7f3e241f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.406+0000 7f3e29e00640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e24073dd0 0x7f3e2414df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 -- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e24073dd0 msgr2=0x7f3e2414df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e24073dd0 0x7f3e2414df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 -- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 msgr2=0x7f3e241f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 0x7f3e241f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 -- 172.21.15.37:0/4097229624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2414ee30 con 0x7f3e240b39d0 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e2a800640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 0x7f3e241f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e29400640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d780 tx=0x7f3e1400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14017070 con 0x7f3e240b39d0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e14009b10 con 0x7f3e240b39d0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14010460 con 0x7f3e240b39d0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2414f110 con 0x7f3e240b39d0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.407+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241f43e0 con 0x7f3e240b39d0 2024-09-21T06:36:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.408+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2415b590 con 0x7f3e240b39d0 2024-09-21T06:36:07.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.410+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e1401a070 con 0x7f3e240b39d0 2024-09-21T06:36:07.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.411+0000 7f3e1e000640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 0x7f3e0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.411+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f3e1409ae90 con 0x7f3e240b39d0 2024-09-21T06:36:07.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.411+0000 7f3e29e00640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 0x7f3e0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.412+0000 7f3e29e00640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 0x7f3e0807b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:07.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.420+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e2415b590 con 0x7f3e240b39d0 2024-09-21T06:36:07.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.617+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e2414a5c0 con 0x7f3e08078b90 2024-09-21T06:36:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.618+0000 7f3e1e000640 1 -- 172.21.15.37:0/4097229624 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7f3e2414a5c0 con 0x7f3e08078b90 2024-09-21T06:36:07.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:36:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 msgr2=0x7f3e0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 0x7f3e0807b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18008040 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 msgr2=0x7f3e2413fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d780 tx=0x7f3e1400dc40 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 shutdown_connections 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3e08078b90 0x7f3e0807b0a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e240b39d0 0x7f3e2413fc70 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24152530 0x7f3e241f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 --2- 172.21.15.37:0/4097229624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e24073dd0 0x7f3e2414df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 >> 172.21.15.37:0/4097229624 conn(0x7f3e2406f940 msgr2=0x7f3e24154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 shutdown_connections 2024-09-21T06:36:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.621+0000 7f3e2b200640 1 -- 172.21.15.37:0/4097229624 wait complete. 2024-09-21T06:36:07.645 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-21T06:36:07.225410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261368,"kb_used_data":11856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713352,"statfs":{"total":773094113280,"available":772826472448,"internally_reserved":0,"allocated":12140544,"data_stored":6362405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-125851,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-258,"num_write_kb":-210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.055319"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":272,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T06:36:03.003902+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:36:03.003902+0000","last_peered":"2024-09-21T06:36:03.003902+0000","last_clean":"2024-09-21T06:36:03.003902+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:36:03.003902+0000","last_undegraded":"2024-09-21T06:36:03.003902+0000","last_fullsized":"2024-09-21T06:36:03.003902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1327104,"data_stored":616449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":701402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":622480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":724308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":632361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1229869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":616491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1945600,"data_stored":1219045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:07.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 -- 172.21.15.37:0/2432691065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d806d9c0 msgr2=0x7f63d8154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 --2- 172.21.15.37:0/2432691065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d806d9c0 0x7f63d8154ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005fd0 tx=0x7f63d002aa90 comp rx=0 tx=0).stop 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 -- 172.21.15.37:0/2432691065 shutdown_connections 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 --2- 172.21.15.37:0/2432691065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d806d9c0 0x7f63d8154ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 --2- 172.21.15.37:0/2432691065 >> v2:172.21.15.150:3300/0 conn(0x7f63d80b9e40 0x7f63d81622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 -- 172.21.15.37:0/2432691065 >> 172.21.15.37:0/2432691065 conn(0x7f63d80d1930 msgr2=0x7f63d80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 -- 172.21.15.37:0/2432691065 shutdown_connections 2024-09-21T06:36:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.876+0000 7f63de400640 1 -- 172.21.15.37:0/2432691065 wait complete. 2024-09-21T06:36:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.879+0000 7f63de400640 1 Processor -- start 2024-09-21T06:36:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.879+0000 7f63de400640 1 -- start start 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d80b9e40 0x7f63d8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63d81627f0 0x7f63d81f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8155770 con 0x7f63d806d9c0 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81555f0 con 0x7f63d81627f0 2024-09-21T06:36:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63de400640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f63d81558f0 con 0x7f63d80b9e40 2024-09-21T06:36:07.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.880+0000 7f63dd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50636/0 (socket says 172.21.15.37:50636) 2024-09-21T06:36:07.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 -- 172.21.15.37:0/3561912971 learned_addr learned my addr 172.21.15.37:0/3561912971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:07.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dda00640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63d81627f0 0x7f63d81f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63d7e00640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d80b9e40 0x7f63d8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 -- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d80b9e40 msgr2=0x7f63d8153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d80b9e40 0x7f63d8153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 -- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63d81627f0 msgr2=0x7f63d81f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63d81627f0 0x7f63d81f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 -- 172.21.15.37:0/3561912971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d81f53a0 con 0x7f63d806d9c0 2024-09-21T06:36:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.881+0000 7f63dd000640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e9f0 tx=0x7f63cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.882+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00cd80 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.882+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc004020 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.882+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d81f5660 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.882+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc004470 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.882+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d81f5c20 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.883+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d8154ea0 con 0x7f63d806d9c0 2024-09-21T06:36:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.884+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63cc0035f0 con 0x7f63d806d9c0 2024-09-21T06:36:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.885+0000 7f63d4c00640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 0x7f63b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.886+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f63cc09b4f0 con 0x7f63d806d9c0 2024-09-21T06:36:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.886+0000 7f63d7e00640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 0x7f63b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.887+0000 7f63d7e00640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 0x7f63b807b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63d81592a0 tx=0x7f63c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:07.890+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63cc063300 con 0x7f63d806d9c0 2024-09-21T06:36:08.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.085+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63d8069260 con 0x7f63b8078b90 2024-09-21T06:36:08.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.086+0000 7f63d4c00640 1 -- 172.21.15.37:0/3561912971 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7f63d8069260 con 0x7f63b8078b90 2024-09-21T06:36:08.087 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:08.087 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.088+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 msgr2=0x7f63b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.088+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 0x7f63b807b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63d81592a0 tx=0x7f63c4009040 comp rx=0 tx=0).stop 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.088+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 msgr2=0x7f63d80c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.088+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e9f0 tx=0x7f63cc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 shutdown_connections 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f63b8078b90 0x7f63b807b0a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63d81627f0 0x7f63d81f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63d80b9e40 0x7f63d8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 --2- 172.21.15.37:0/3561912971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63d806d9c0 0x7f63d80c9ff0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 >> 172.21.15.37:0/3561912971 conn(0x7f63d80d1930 msgr2=0x7f63d81616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 shutdown_connections 2024-09-21T06:36:08.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.089+0000 7f63de400640 1 -- 172.21.15.37:0/3561912971 wait complete. 2024-09-21T06:36:08.116 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-21T06:36:07.225410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261368,"kb_used_data":11856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713352,"statfs":{"total":773094113280,"available":772826472448,"internally_reserved":0,"allocated":12140544,"data_stored":6362405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-125851,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-258,"num_write_kb":-210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.055319"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":272,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T06:36:03.003902+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:36:03.003902+0000","last_peered":"2024-09-21T06:36:03.003902+0000","last_clean":"2024-09-21T06:36:03.003902+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:36:03.003902+0000","last_undegraded":"2024-09-21T06:36:03.003902+0000","last_fullsized":"2024-09-21T06:36:03.003902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1327104,"data_stored":616449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":701402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":622480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":724308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":632361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1229869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":616491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1945600,"data_stored":1219045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:08.117 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:36:08.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 -- 172.21.15.37:0/121494782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f0003a30 msgr2=0x7fb3f0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 --2- 172.21.15.37:0/121494782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f0003a30 0x7fb3f0003ea0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c2e0 tx=0x7fb3f0005830 comp rx=0 tx=0).stop 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb40be00640 1 -- 172.21.15.37:0/121494782 reap_dead start 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 -- 172.21.15.37:0/121494782 shutdown_connections 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 -- 172.21.15.37:0/121494782 >> 172.21.15.37:0/121494782 conn(0x7fb40c0cca40 msgr2=0x7fb40c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 -- 172.21.15.37:0/121494782 shutdown_connections 2024-09-21T06:36:08.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.345+0000 7fb412800640 1 -- 172.21.15.37:0/121494782 wait complete. 2024-09-21T06:36:08.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.348+0000 7fb412800640 1 Processor -- start 2024-09-21T06:36:08.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.348+0000 7fb412800640 1 -- start start 2024-09-21T06:36:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3f0003a30 0x7fb40c157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c156150 0x7fb40c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c06df70 con 0x7fb40c156aa0 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c06ddf0 con 0x7fb40c156150 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.349+0000 7fb412800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c06e0f0 con 0x7fb3f0003a30 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50640/0 (socket says 172.21.15.37:50640) 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 -- 172.21.15.37:0/2833928550 learned_addr learned my addr 172.21.15.37:0/2833928550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40b400640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3f0003a30 0x7fb40c157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40aa00640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c156150 0x7fb40c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 -- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3f0003a30 msgr2=0x7fb40c157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3f0003a30 0x7fb40c157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 -- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c156150 msgr2=0x7fb40c156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c156150 0x7fb40c156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.350+0000 7fb40be00640 1 -- 172.21.15.37:0/2833928550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c158470 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb40be00640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb40c072b90 tx=0x7fb404001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404017070 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4040108c0 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c0a6a00 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404010d10 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.351+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c0a6ca0 con 0x7fb40c156aa0 2024-09-21T06:36:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.352+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c06f820 con 0x7fb40c156aa0 2024-09-21T06:36:08.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.354+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb40401a070 con 0x7fb40c156aa0 2024-09-21T06:36:08.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.355+0000 7fb403400640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 0x7fb3ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.355+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fb40409a970 con 0x7fb40c156aa0 2024-09-21T06:36:08.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.355+0000 7fb40b400640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 0x7fb3ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.356+0000 7fb40b400640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 0x7fb3ec07b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc033000 tx=0x7fb3fc038040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.363+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb40c06e740 con 0x7fb40c156aa0 2024-09-21T06:36:08.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.555+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb40c1590c0 con 0x7fb3ec078b90 2024-09-21T06:36:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.556+0000 7fb403400640 1 -- 172.21.15.37:0/2833928550 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7fb40c1590c0 con 0x7fb3ec078b90 2024-09-21T06:36:08.557 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 msgr2=0x7fb3ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 0x7fb3ec07b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc033000 tx=0x7fb3fc038040 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 msgr2=0x7fb40c0a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb40c072b90 tx=0x7fb404001950 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 shutdown_connections 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb3ec078b90 0x7fb3ec07b0a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40c156aa0 0x7fb40c0a74a0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c156150 0x7fb40c156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 --2- 172.21.15.37:0/2833928550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3f0003a30 0x7fb40c157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 >> 172.21.15.37:0/2833928550 conn(0x7fb40c0cca40 msgr2=0x7fb40c0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 shutdown_connections 2024-09-21T06:36:08.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.558+0000 7fb412800640 1 -- 172.21.15.37:0/2833928550 wait complete. 2024-09-21T06:36:08.584 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-21T06:36:07.225410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261368,"kb_used_data":11856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713352,"statfs":{"total":773094113280,"available":772826472448,"internally_reserved":0,"allocated":12140544,"data_stored":6362405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-125851,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-258,"num_write_kb":-210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.055319"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":272,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T06:36:03.003902+0000","last_change":"2024-09-21T06:27:50.899879+0000","last_active":"2024-09-21T06:36:03.003902+0000","last_peered":"2024-09-21T06:36:03.003902+0000","last_clean":"2024-09-21T06:36:03.003902+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:36:03.003902+0000","last_undegraded":"2024-09-21T06:36:03.003902+0000","last_fullsized":"2024-09-21T06:36:03.003902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:27:49.879183+0000","last_clean_scrub_stamp":"2024-09-21T06:27:49.879183+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:29.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1327104,"data_stored":616449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":701402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":622480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":724308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":632361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1229869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":616491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1945600,"data_stored":1219045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:08.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:36:08.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 -- 172.21.15.37:0/2045376470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5240ceff0 msgr2=0x7fa52415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 --2- 172.21.15.37:0/2045376470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5240ceff0 0x7fa52415e5e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa518005fd0 tx=0x7fa51802a4b0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 -- 172.21.15.37:0/2045376470 shutdown_connections 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 --2- 172.21.15.37:0/2045376470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240cfbc0 0x7fa5240b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 --2- 172.21.15.37:0/2045376470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5240ceff0 0x7fa52415e5e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 -- 172.21.15.37:0/2045376470 >> 172.21.15.37:0/2045376470 conn(0x7fa5240d4360 msgr2=0x7fa5240d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 -- 172.21.15.37:0/2045376470 shutdown_connections 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.811+0000 7fa52b200640 1 -- 172.21.15.37:0/2045376470 wait complete. 2024-09-21T06:36:08.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.813+0000 7fa52b200640 1 Processor -- start 2024-09-21T06:36:08.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.813+0000 7fa52b200640 1 -- start start 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 0x7fa52415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 0x7fa524152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5240b8db0 con 0x7fa524152e70 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5240b8c30 con 0x7fa5240ceff0 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52b200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa5240b8f30 con 0x7fa5240d2520 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50656/0 (socket says 172.21.15.37:50656) 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa52a800640 1 -- 172.21.15.37:0/2342599247 learned_addr learned my addr 172.21.15.37:0/2342599247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa529400640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 0x7fa524152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.815+0000 7fa529e00640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 0x7fa52415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 -- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 msgr2=0x7fa524152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 0x7fa524152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 -- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 msgr2=0x7fa52415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 0x7fa52415ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 -- 172.21.15.37:0/2342599247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5241f6570 con 0x7fa524152e70 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa529400640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 0x7fa524152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa529e00640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 0x7fa52415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:08.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52a800640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa52000ea80 tx=0x7fa52000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520004b20 con 0x7fa524152e70 2024-09-21T06:36:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa520004ce0 con 0x7fa524152e70 2024-09-21T06:36:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520005280 con 0x7fa524152e70 2024-09-21T06:36:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.816+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5241616c0 con 0x7fa524152e70 2024-09-21T06:36:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.817+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa524152a60 con 0x7fa524152e70 2024-09-21T06:36:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.818+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5240cedd0 con 0x7fa524152e70 2024-09-21T06:36:08.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.819+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa520005420 con 0x7fa524152e70 2024-09-21T06:36:08.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.820+0000 7fa51e000640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 0x7fa50807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.820+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fa52009b8f0 con 0x7fa524152e70 2024-09-21T06:36:08.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.820+0000 7fa529e00640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 0x7fa50807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.821+0000 7fa529e00640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 0x7fa50807b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa518033000 tx=0x7fa518004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:08.824+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa520063df0 con 0x7fa524152e70 2024-09-21T06:36:09.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.020+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5241f62e0 con 0x7fa524152e70 2024-09-21T06:36:09.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.021+0000 7fa51e000640 1 -- 172.21.15.37:0/2342599247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13587 (secure 0 0 0) 0x7fa5200636f0 con 0x7fa524152e70 2024-09-21T06:36:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:09.023 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":109,"fsid":"13e9e1b2-9d52-4cf2-a477-95a0639530e0","created":"2024-09-21T06:27:31.994916+0000","modified":"2024-09-21T06:36:03.174414+0000","last_up_change":"2024-09-21T06:27:47.874438+0000","last_in_change":"2024-09-21T06:27:37.795248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:27:49.118619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4f8bc6ac-56bf-4ff9-9e0a-a21fefcfa234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6809","nonce":787824175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6811","nonce":787824175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6816","nonce":787824175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":787824175},{"type":"v1","addr":"172.21.15.37:6813","nonce":787824175}]},"public_addr":"172.21.15.37:6809/787824175","cluster_addr":"172.21.15.37:6811/787824175","heartbeat_back_addr":"172.21.15.37:6816/787824175","heartbeat_front_addr":"172.21.15.37:6813/787824175","state":["exists","up"]},{"osd":1,"uuid":"737fcbb4-0a68-4b75-9ba3-1372051e3295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6825","nonce":1275190192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6827","nonce":1275190192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6831","nonce":1275190192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1275190192},{"type":"v1","addr":"172.21.15.37:6829","nonce":1275190192}]},"public_addr":"172.21.15.37:6825/1275190192","cluster_addr":"172.21.15.37:6827/1275190192","heartbeat_back_addr":"172.21.15.37:6831/1275190192","heartbeat_front_addr":"172.21.15.37:6829/1275190192","state":["exists","up"]},{"osd":2,"uuid":"814b0688-5744-4302-a267-b7ed03896600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6801","nonce":2037276396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6803","nonce":2037276396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6807","nonce":2037276396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2037276396},{"type":"v1","addr":"172.21.15.37:6805","nonce":2037276396}]},"public_addr":"172.21.15.37:6801/2037276396","cluster_addr":"172.21.15.37:6803/2037276396","heartbeat_back_addr":"172.21.15.37:6807/2037276396","heartbeat_front_addr":"172.21.15.37:6805/2037276396","state":["exists","up"]},{"osd":3,"uuid":"cc4bfd5a-b66e-40d7-bb06-7d2f6c9c78a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6815","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6817","nonce":1242254511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6819","nonce":1242254511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6823","nonce":1242254511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1242254511},{"type":"v1","addr":"172.21.15.37:6821","nonce":1242254511}]},"public_addr":"172.21.15.37:6817/1242254511","cluster_addr":"172.21.15.37:6819/1242254511","heartbeat_back_addr":"172.21.15.37:6823/1242254511","heartbeat_front_addr":"172.21.15.37:6821/1242254511","state":["exists","up"]},{"osd":4,"uuid":"9d41f63c-f130-4048-8aea-3bb2e496b081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6819","nonce":2746357825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6821","nonce":2746357825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6825","nonce":2746357825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2746357825},{"type":"v1","addr":"172.21.15.150:6823","nonce":2746357825}]},"public_addr":"172.21.15.150:6819/2746357825","cluster_addr":"172.21.15.150:6821/2746357825","heartbeat_back_addr":"172.21.15.150:6825/2746357825","heartbeat_front_addr":"172.21.15.150:6823/2746357825","state":["exists","up"]},{"osd":5,"uuid":"fe3eab60-21a1-4841-baa9-a8c004fe90c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6803","nonce":2870078774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6805","nonce":2870078774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6809","nonce":2870078774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2870078774},{"type":"v1","addr":"172.21.15.150:6807","nonce":2870078774}]},"public_addr":"172.21.15.150:6803/2870078774","cluster_addr":"172.21.15.150:6805/2870078774","heartbeat_back_addr":"172.21.15.150:6809/2870078774","heartbeat_front_addr":"172.21.15.150:6807/2870078774","state":["exists","up"]},{"osd":6,"uuid":"af143beb-c0f0-4231-92ac-b08f9865c704","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6811","nonce":3705202380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6813","nonce":3705202380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6817","nonce":3705202380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3705202380},{"type":"v1","addr":"172.21.15.150:6815","nonce":3705202380}]},"public_addr":"172.21.15.150:6811/3705202380","cluster_addr":"172.21.15.150:6813/3705202380","heartbeat_back_addr":"172.21.15.150:6817/3705202380","heartbeat_front_addr":"172.21.15.150:6815/3705202380","state":["exists","up"]},{"osd":7,"uuid":"00fa710d-4aed-4575-b6e8-91b3a4cb057a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6827","nonce":1641077969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6829","nonce":1641077969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6833","nonce":1641077969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1641077969},{"type":"v1","addr":"172.21.15.150:6831","nonce":1641077969}]},"public_addr":"172.21.15.150:6827/1641077969","cluster_addr":"172.21.15.150:6829/1641077969","heartbeat_back_addr":"172.21.15.150:6833/1641077969","heartbeat_front_addr":"172.21.15.150:6831/1641077969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.733161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.701068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.160493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.732187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.617727+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:43.862324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.423083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:27:44.499431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6832/1950391294":"2024-09-22T06:36:00.164240+0000","172.21.15.37:6833/1950391294":"2024-09-22T06:36:00.164240+0000","172.21.15.37:6835/912339030":"2024-09-22T06:36:00.164245+0000","172.21.15.37:6834/912339030":"2024-09-22T06:36:00.164245+0000","172.21.15.150:6837/1287330873":"2024-09-22T06:36:00.164221+0000","172.21.15.150:6835/1287330873":"2024-09-22T06:36:00.164221+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.023+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 msgr2=0x7fa50807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.023+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 0x7fa50807b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa518033000 tx=0x7fa518004730 comp rx=0 tx=0).stop 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.023+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 msgr2=0x7fa524155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.023+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa52000ea80 tx=0x7fa52000ef40 comp rx=0 tx=0).stop 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 shutdown_connections 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa508078b90 0x7fa50807b0a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa524152e70 0x7fa524155a40 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:09.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5240d2520 0x7fa524152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 --2- 172.21.15.37:0/2342599247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ceff0 0x7fa52415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 >> 172.21.15.37:0/2342599247 conn(0x7fa5240d4360 msgr2=0x7fa5240d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 shutdown_connections 2024-09-21T06:36:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:09.024+0000 7fa52b200640 1 -- 172.21.15.37:0/2342599247 wait complete. 2024-09-21T06:36:10.048 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T06:36:10.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:10.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 -- 172.21.15.37:0/1645556997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c06e2f0 msgr2=0x7f399c160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 --2- 172.21.15.37:0/1645556997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c06e2f0 0x7f399c160a50 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f399c075dd0 tx=0x7f399402aac0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 -- 172.21.15.37:0/1645556997 shutdown_connections 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 --2- 172.21.15.37:0/1645556997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c06e2f0 0x7f399c160a50 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 --2- 172.21.15.37:0/1645556997 >> v2:172.21.15.150:3300/0 conn(0x7f399c0ca690 0x7f399c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 -- 172.21.15.37:0/1645556997 >> 172.21.15.37:0/1645556997 conn(0x7f399c0ccac0 msgr2=0x7f399c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 -- 172.21.15.37:0/1645556997 shutdown_connections 2024-09-21T06:36:10.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.290+0000 7f39a1200640 1 -- 172.21.15.37:0/1645556997 wait complete. 2024-09-21T06:36:10.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.292+0000 7f39a1200640 1 Processor -- start 2024-09-21T06:36:10.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.292+0000 7f39a1200640 1 -- start start 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f399c06e2f0 0x7f399c1f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f399c0ca690 0x7f399c1e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 0x7f399c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f399c06f7c0 con 0x7f399c160640 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f399c06f640 con 0x7f399c0ca690 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f39a1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f399c06f940 con 0x7f399c06e2f0 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f399a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 0x7f399c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f399a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 0x7f399c157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50682/0 (socket says 172.21.15.37:50682) 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.294+0000 7f399a800640 1 -- 172.21.15.37:0/853121556 learned_addr learned my addr 172.21.15.37:0/853121556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3999e00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f399c06e2f0 0x7f399c1f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3991400640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f399c0ca690 0x7f399c1e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 -- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f399c06e2f0 msgr2=0x7f399c1f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f399c06e2f0 0x7f399c1f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 -- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f399c0ca690 msgr2=0x7f399c1e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f399c0ca690 0x7f399c1e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c1581b0 con 0x7f399c160640 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3999e00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f399c06e2f0 0x7f399c1f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f399a800640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 0x7f399c157c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f399402b0b0 tx=0x7f3994003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994003e00 con 0x7f399c160640 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3994004250 con 0x7f399c160640 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.295+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39940046a0 con 0x7f399c160640 2024-09-21T06:36:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.296+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c074e20 con 0x7f399c160640 2024-09-21T06:36:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.296+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c1e7450 con 0x7f399c160640 2024-09-21T06:36:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.296+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f399c160a50 con 0x7f399c160640 2024-09-21T06:36:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.298+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3994049430 con 0x7f399c160640 2024-09-21T06:36:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3992a00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3978078b90 0x7f397807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f39940bba30 con 0x7f399c160640 2024-09-21T06:36:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3992a00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f397807f1c0 0x7f3978081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3994031e50 con 0x7f397807f1c0 2024-09-21T06:36:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f39940c02a0 con 0x7f399c160640 2024-09-21T06:36:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3991400640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f397807f1c0 0x7f3978081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.299+0000 7f3999e00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3978078b90 0x7f397807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.300+0000 7f3991400640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f397807f1c0 0x7f3978081650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.300+0000 7f3999e00640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3978078b90 0x7f397807b0a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f398c009b80 tx=0x7f398c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.301+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== osd.0 v2:172.21.15.37:6808/787824175 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3994031e50 con 0x7f397807f1c0 2024-09-21T06:36:10.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.314+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 --> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f399c074bd0 con 0x7f397807f1c0 2024-09-21T06:36:10.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.314+0000 7f3992a00640 1 -- 172.21.15.37:0/853121556 <== osd.0 v2:172.21.15.37:6808/787824175 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f399c074bd0 con 0x7f397807f1c0 2024-09-21T06:36:10.316 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:10.316 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:10.316 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f397807f1c0 msgr2=0x7f3978081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:6808/787824175,v1:172.21.15.37:6809/787824175] conn(0x7f397807f1c0 0x7f3978081650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3978078b90 msgr2=0x7f397807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3978078b90 0x7f397807b0a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f398c009b80 tx=0x7f398c005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 msgr2=0x7f399c157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 --2- 172.21.15.37:0/853121556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f399c160640 0x7f399c157c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f399402b0b0 tx=0x7f3994003c10 comp rx=0 tx=0).stop 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f399a800640 1 -- 172.21.15.37:0/853121556 reap_dead start 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 shutdown_connections 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 >> 172.21.15.37:0/853121556 conn(0x7f399c0ccac0 msgr2=0x7f399c0cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 shutdown_connections 2024-09-21T06:36:10.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.315+0000 7f39a1200640 1 -- 172.21.15.37:0/853121556 wait complete. 2024-09-21T06:36:10.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T06:36:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 -- 172.21.15.37:0/3841507160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c0cf6d0 msgr2=0x7fdd2c15fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3841507160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c0cf6d0 0x7fdd2c15fee0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c156ed0 tx=0x7fdd1402a560 comp rx=0 tx=0).stop 2024-09-21T06:36:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 -- 172.21.15.37:0/3841507160 shutdown_connections 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3841507160 >> v2:172.21.15.150:3300/0 conn(0x7fdd2c160420 0x7fdd2c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3841507160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c0cf6d0 0x7fdd2c15fee0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3841507160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cc0e0 0x7fdd2c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 -- 172.21.15.37:0/3841507160 >> 172.21.15.37:0/3841507160 conn(0x7fdd2c0d05c0 msgr2=0x7fdd2c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 -- 172.21.15.37:0/3841507160 shutdown_connections 2024-09-21T06:36:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.567+0000 7fdd2be00640 1 -- 172.21.15.37:0/3841507160 wait complete. 2024-09-21T06:36:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.569+0000 7fdd2be00640 1 Processor -- start 2024-09-21T06:36:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.569+0000 7fdd2be00640 1 -- start start 2024-09-21T06:36:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd2c0cc0e0 0x7fdd2c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cf6d0 0x7fdd2c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c15d1c0 con 0x7fdd2c160420 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c15d040 con 0x7fdd2c0cc0e0 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2be00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c15d340 con 0x7fdd2c0cf6d0 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd2c0cc0e0 0x7fdd2c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd23e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cf6d0 0x7fdd2c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50700/0 (socket says 172.21.15.37:50700) 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 -- 172.21.15.37:0/3593871040 learned_addr learned my addr 172.21.15.37:0/3593871040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 -- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cf6d0 msgr2=0x7fdd2c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cf6d0 0x7fdd2c0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.571+0000 7fdd2b400640 1 -- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd2c0cc0e0 msgr2=0x7fdd2c155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd2b400640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd2c0cc0e0 0x7fdd2c155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd2b400640 1 -- 172.21.15.37:0/3593871040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c15aef0 con 0x7fdd2c160420 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd2b400640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e990 tx=0x7fdd2400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004d20 con 0x7fdd2c160420 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd2400cb00 con 0x7fdd2c160420 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c15b470 con 0x7fdd2c160420 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24005020 con 0x7fdd2c160420 2024-09-21T06:36:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.572+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c15a950 con 0x7fdd2c160420 2024-09-21T06:36:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.573+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c0cde70 con 0x7fdd2c160420 2024-09-21T06:36:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.575+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd240051c0 con 0x7fdd2c160420 2024-09-21T06:36:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.576+0000 7fdd23400640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 0x7fdd0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.576+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fdd2409b500 con 0x7fdd2c160420 2024-09-21T06:36:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.576+0000 7fdd2aa00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 0x7fdd0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.577+0000 7fdd2aa00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 0x7fdd0807b0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c005fd0 tx=0x7fdd1c00b260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.580+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdd2c0cde70 con 0x7fdd2c160420 2024-09-21T06:36:10.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.775+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fdd2c0c9250 con 0x7fdd08078b90 2024-09-21T06:36:10.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.776+0000 7fdd23400640 1 -- 172.21.15.37:0/3593871040 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdd2c0c9250 con 0x7fdd08078b90 2024-09-21T06:36:10.777 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 msgr2=0x7fdd0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 0x7fdd0807b0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c005fd0 tx=0x7fdd1c00b260 comp rx=0 tx=0).stop 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 msgr2=0x7fdd2c0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e990 tx=0x7fdd2400ee50 comp rx=0 tx=0).stop 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 shutdown_connections 2024-09-21T06:36:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fdd08078b90 0x7fdd0807b0a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2c160420 0x7fdd2c0ca1a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd2c0cf6d0 0x7fdd2c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 --2- 172.21.15.37:0/3593871040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd2c0cc0e0 0x7fdd2c155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.778+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 >> 172.21.15.37:0/3593871040 conn(0x7fdd2c0d05c0 msgr2=0x7fdd2c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.779+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 shutdown_connections 2024-09-21T06:36:10.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:10.779+0000 7fdd2be00640 1 -- 172.21.15.37:0/3593871040 wait complete. 2024-09-21T06:36:10.802 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T06:36:10.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.031+0000 7f187e200640 1 -- 172.21.15.37:0/2451881013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f187806e2f0 msgr2=0x7f1878160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.031+0000 7f187e200640 1 --2- 172.21.15.37:0/2451881013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f187806e2f0 0x7f1878160a50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f1870005fd0 tx=0x7f187002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 -- 172.21.15.37:0/2451881013 shutdown_connections 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 --2- 172.21.15.37:0/2451881013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f187806e2f0 0x7f1878160a50 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 --2- 172.21.15.37:0/2451881013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18780ca690 0x7f1878160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 -- 172.21.15.37:0/2451881013 >> 172.21.15.37:0/2451881013 conn(0x7f18780ccac0 msgr2=0x7f18780cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 -- 172.21.15.37:0/2451881013 shutdown_connections 2024-09-21T06:36:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.032+0000 7f187e200640 1 -- 172.21.15.37:0/2451881013 wait complete. 2024-09-21T06:36:11.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.034+0000 7f187e200640 1 Processor -- start 2024-09-21T06:36:11.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.034+0000 7f187e200640 1 -- start start 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f187806e2f0 0x7f18781f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 0x7f1878158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1878160640 0x7f18780cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f187806f9d0 con 0x7f18780ca690 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f187806f850 con 0x7f1878160640 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.035+0000 7f187e200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f187806fb50 con 0x7f187806e2f0 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 0x7f1878158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f1877400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f187806e2f0 0x7f18781f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 0x7f1878158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50710/0 (socket says 172.21.15.37:50710) 2024-09-21T06:36:11.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 -- 172.21.15.37:0/3074527847 learned_addr learned my addr 172.21.15.37:0/3074527847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:11.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f1877e00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1878160640 0x7f18780cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 -- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f187806e2f0 msgr2=0x7f18781f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f187806e2f0 0x7f18781f9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 -- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1878160640 msgr2=0x7f18780cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1878160640 0x7f18780cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.036+0000 7f186ea00640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18781f5e50 con 0x7f18780ca690 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f186ea00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 0x7f1878158080 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f186000c3c0 tx=0x7f186000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860017070 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18600035f0 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781f6130 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860003a80 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.037+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781f66f0 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.038+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1878160a50 con 0x7f18780ca690 2024-09-21T06:36:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.039+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f186001a070 con 0x7f18780ca690 2024-09-21T06:36:11.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.040+0000 7f186fe00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1854078bc0 0x7f185407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.040+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f186009a660 con 0x7f18780ca690 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.040+0000 7f186fe00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f185407f1f0 0x7f1854081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.040+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1854081d50 con 0x7f185407f1f0 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.040+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f186009aa10 con 0x7f18780ca690 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.041+0000 7f1877400640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1854078bc0 0x7f185407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.041+0000 7f1877e00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f185407f1f0 0x7f1854081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.041+0000 7f1877400640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1854078bc0 0x7f185407b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1868009b30 tx=0x7f1868005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.041+0000 7f1877e00640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f185407f1f0 0x7f1854081680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.042+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== osd.1 v2:172.21.15.37:6824/1275190192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1854081d50 con 0x7f185407f1f0 2024-09-21T06:36:11.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.055+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 --> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f18781f5bd0 con 0x7f185407f1f0 2024-09-21T06:36:11.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f186fe00640 1 -- 172.21.15.37:0/3074527847 <== osd.1 v2:172.21.15.37:6824/1275190192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f18781f5bd0 con 0x7f185407f1f0 2024-09-21T06:36:11.057 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:11.057 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:11.057 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f185407f1f0 msgr2=0x7f1854081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:6824/1275190192,v1:172.21.15.37:6825/1275190192] conn(0x7f185407f1f0 0x7f1854081680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1854078bc0 msgr2=0x7f185407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f1854078bc0 0x7f185407b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1868009b30 tx=0x7f1868005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 msgr2=0x7f1878158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 --2- 172.21.15.37:0/3074527847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18780ca690 0x7f1878158080 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f186000c3c0 tx=0x7f186000c880 comp rx=0 tx=0).stop 2024-09-21T06:36:11.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f1877e00640 1 -- 172.21.15.37:0/3074527847 reap_dead start 2024-09-21T06:36:11.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 shutdown_connections 2024-09-21T06:36:11.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 >> 172.21.15.37:0/3074527847 conn(0x7f18780ccac0 msgr2=0x7f187806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.056+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 shutdown_connections 2024-09-21T06:36:11.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.057+0000 7f187e200640 1 -- 172.21.15.37:0/3074527847 wait complete. 2024-09-21T06:36:11.079 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T06:36:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 -- 172.21.15.37:0/2868208917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3dd80053a0 msgr2=0x7f3dd800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 --2- 172.21.15.37:0/2868208917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3dd80053a0 0x7f3dd800ba20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00c610 tx=0x7f3dd8005830 comp rx=0 tx=0).stop 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 -- 172.21.15.37:0/2868208917 shutdown_connections 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 --2- 172.21.15.37:0/2868208917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3dd80053a0 0x7f3dd800ba20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 -- 172.21.15.37:0/2868208917 >> 172.21.15.37:0/2868208917 conn(0x7f3df40d05c0 msgr2=0x7f3df406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 -- 172.21.15.37:0/2868208917 shutdown_connections 2024-09-21T06:36:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.306+0000 7f3df9800640 1 -- 172.21.15.37:0/2868208917 wait complete. 2024-09-21T06:36:11.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.308+0000 7f3df9800640 1 Processor -- start 2024-09-21T06:36:11.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.309+0000 7f3df9800640 1 -- start start 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd80053a0 0x7f3df41562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df41442d0 0x7f3df4143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3df415fee0 con 0x7f3df413d1d0 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4070780 con 0x7f3df41442d0 2024-09-21T06:36:11.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df9800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3df4160060 con 0x7f3dd80053a0 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50724/0 (socket says 172.21.15.37:50724) 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.310+0000 7f3df8e00640 1 -- 172.21.15.37:0/3939422041 learned_addr learned my addr 172.21.15.37:0/3939422041 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df3e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd80053a0 0x7f3df41562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3debe00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df41442d0 0x7f3df4143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 -- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd80053a0 msgr2=0x7f3df41562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd80053a0 0x7f3df41562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 -- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df41442d0 msgr2=0x7f3df4143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df41442d0 0x7f3df4143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 -- 172.21.15.37:0/3939422041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df413da20 con 0x7f3df413d1d0 2024-09-21T06:36:11.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df8e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02b4d0 tx=0x7f3dec002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.311+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec039070 con 0x7f3df413d1d0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.312+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dec02bce0 con 0x7f3df413d1d0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.312+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec006290 con 0x7f3df413d1d0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.312+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41567f0 con 0x7f3df413d1d0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.312+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df4142820 con 0x7f3df413d1d0 2024-09-21T06:36:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.313+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df4144770 con 0x7f3df413d1d0 2024-09-21T06:36:11.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.314+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dec006430 con 0x7f3df413d1d0 2024-09-21T06:36:11.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.315+0000 7f3df0c00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 0x7f3dd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.315+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f3dec0baa60 con 0x7f3df413d1d0 2024-09-21T06:36:11.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.315+0000 7f3df3e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 0x7f3dd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.316+0000 7f3df3e00640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 0x7f3dd007b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006680 tx=0x7f3de4006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.319+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dec045030 con 0x7f3df413d1d0 2024-09-21T06:36:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.517+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f3df41605c0 con 0x7f3dd0078b90 2024-09-21T06:36:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.518+0000 7f3df0c00640 1 -- 172.21.15.37:0/3939422041 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3df41605c0 con 0x7f3dd0078b90 2024-09-21T06:36:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T06:36:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 msgr2=0x7f3dd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 0x7f3dd007b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006680 tx=0x7f3de4006b70 comp rx=0 tx=0).stop 2024-09-21T06:36:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 msgr2=0x7f3df4143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02b4d0 tx=0x7f3dec002170 comp rx=0 tx=0).stop 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 shutdown_connections 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3dd0078b90 0x7f3dd007b0a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df413d1d0 0x7f3df4143840 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df41442d0 0x7f3df4143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 --2- 172.21.15.37:0/3939422041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd80053a0 0x7f3df41562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 >> 172.21.15.37:0/3939422041 conn(0x7f3df40d05c0 msgr2=0x7f3df40743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 shutdown_connections 2024-09-21T06:36:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.520+0000 7f3df9800640 1 -- 172.21.15.37:0/3939422041 wait complete. 2024-09-21T06:36:11.547 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T06:36:11.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 -- 172.21.15.37:0/3733613808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e0160810 msgr2=0x7f09e006d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 --2- 172.21.15.37:0/3733613808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e0160810 0x7f09e006d2e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f09e00ae500 tx=0x7f09cc02a560 comp rx=0 tx=0).stop 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 -- 172.21.15.37:0/3733613808 shutdown_connections 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 --2- 172.21.15.37:0/3733613808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e0160810 0x7f09e006d2e0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 --2- 172.21.15.37:0/3733613808 >> v2:172.21.15.150:3300/0 conn(0x7f09e0160c20 0x7f09e006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 -- 172.21.15.37:0/3733613808 >> 172.21.15.37:0/3733613808 conn(0x7f09e00ccac0 msgr2=0x7f09e00cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 -- 172.21.15.37:0/3733613808 shutdown_connections 2024-09-21T06:36:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.776+0000 7f09e8600640 1 -- 172.21.15.37:0/3733613808 wait complete. 2024-09-21T06:36:11.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.779+0000 7f09e8600640 1 Processor -- start 2024-09-21T06:36:11.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.779+0000 7f09e8600640 1 -- start start 2024-09-21T06:36:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 0x7f09e00c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e0160810 0x7f09e01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0160c20 0x7f09e00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09e006e0f0 con 0x7f09e00ca690 2024-09-21T06:36:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09e006df70 con 0x7f09e0160c20 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.780+0000 7f09e8600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f09e006e270 con 0x7f09e0160810 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 0x7f09e00c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 0x7f09e00c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50754/0 (socket says 172.21.15.37:50754) 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 -- 172.21.15.37:0/549712378 learned_addr learned my addr 172.21.15.37:0/549712378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5200640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e0160810 0x7f09e01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e6600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0160c20 0x7f09e00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 -- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e0160810 msgr2=0x7f09e01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e0160810 0x7f09e01f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 -- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0160c20 msgr2=0x7f09e00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0160c20 0x7f09e00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e00c6760 con 0x7f09e00ca690 2024-09-21T06:36:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e6600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0160c20 0x7f09e00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.781+0000 7f09e5c00640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 0x7f09e00c3b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b410 tx=0x7f09d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.782+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4015020 con 0x7f09e00ca690 2024-09-21T06:36:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.782+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d4011920 con 0x7f09e00ca690 2024-09-21T06:36:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.782+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4011d70 con 0x7f09e00ca690 2024-09-21T06:36:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.782+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01f77b0 con 0x7f09e00ca690 2024-09-21T06:36:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.782+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01f7d70 con 0x7f09e00ca690 2024-09-21T06:36:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.783+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09e0160100 con 0x7f09e00ca690 2024-09-21T06:36:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.784+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09d4018020 con 0x7f09e00ca690 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.785+0000 7f09dd600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f09c0078bc0 0x7f09c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.785+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f09d409b120 con 0x7f09e00ca690 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.785+0000 7f09dd600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f09c007f1f0 0x7f09c0081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.785+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09c0081d50 con 0x7f09c007f1f0 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.785+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f09d409b5b0 con 0x7f09e00ca690 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.786+0000 7f09e5200640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f09c0078bc0 0x7f09c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.786+0000 7f09e6600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f09c007f1f0 0x7f09c0081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.786+0000 7f09e5200640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f09c0078bc0 0x7f09c007b0d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f09cc033000 tx=0x7f09cc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.786+0000 7f09e6600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f09c007f1f0 0x7f09c0081680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.787+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== osd.2 v2:172.21.15.37:6800/2037276396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09c0081d50 con 0x7f09c007f1f0 2024-09-21T06:36:11.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.800+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 --> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09e00b5f20 con 0x7f09c007f1f0 2024-09-21T06:36:11.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09dd600640 1 -- 172.21.15.37:0/549712378 <== osd.2 v2:172.21.15.37:6800/2037276396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f09e00b5f20 con 0x7f09c007f1f0 2024-09-21T06:36:11.802 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:11.802 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:11.802 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f09c007f1f0 msgr2=0x7f09c0081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:6800/2037276396,v1:172.21.15.37:6801/2037276396] conn(0x7f09c007f1f0 0x7f09c0081680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f09c0078bc0 msgr2=0x7f09c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f09c0078bc0 0x7f09c007b0d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f09cc033000 tx=0x7f09cc004730 comp rx=0 tx=0).stop 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 msgr2=0x7f09e00c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 --2- 172.21.15.37:0/549712378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e00ca690 0x7f09e00c3b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b410 tx=0x7f09d400b8d0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e6600640 1 -- 172.21.15.37:0/549712378 reap_dead start 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 shutdown_connections 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 >> 172.21.15.37:0/549712378 conn(0x7f09e00ccac0 msgr2=0x7f09e00aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 shutdown_connections 2024-09-21T06:36:11.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:11.801+0000 7f09e8600640 1 -- 172.21.15.37:0/549712378 wait complete. 2024-09-21T06:36:11.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T06:36:12.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 -- 172.21.15.37:0/210208363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0072d90 msgr2=0x7f3cd00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/210208363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0072d90 0x7f3cd00731a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3cd015cda0 tx=0x7f3cc802a920 comp rx=0 tx=0).stop 2024-09-21T06:36:12.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 -- 172.21.15.37:0/210208363 shutdown_connections 2024-09-21T06:36:12.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/210208363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0072d90 0x7f3cd00731a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 -- 172.21.15.37:0/210208363 >> 172.21.15.37:0/210208363 conn(0x7f3cd00b1b90 msgr2=0x7f3cd00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 -- 172.21.15.37:0/210208363 shutdown_connections 2024-09-21T06:36:12.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.053+0000 7f3cd4c00640 1 -- 172.21.15.37:0/210208363 wait complete. 2024-09-21T06:36:12.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.055+0000 7f3cd4c00640 1 Processor -- start 2024-09-21T06:36:12.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.055+0000 7f3cd4c00640 1 -- start start 2024-09-21T06:36:12.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 0x7f3cd00c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 0x7f3cd00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0073860 con 0x7f3cd006ddf0 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd00736e0 con 0x7f3cd0156a50 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cd4c00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd00739e0 con 0x7f3cd0072d90 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50766/0 (socket says 172.21.15.37:50766) 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 -- 172.21.15.37:0/2832688927 learned_addr learned my addr 172.21.15.37:0/2832688927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cc5400640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 0x7f3cd00c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cce800640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 0x7f3cd00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 -- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 msgr2=0x7f3cd00c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 0x7f3cd00c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 -- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 msgr2=0x7f3cd00b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 0x7f3cd00b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3ccde00640 1 -- 172.21.15.37:0/2832688927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd00b3540 con 0x7f3cd006ddf0 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cc5400640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 0x7f3cd00c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:12.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.057+0000 7f3cce800640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 0x7f3cd00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3ccde00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001220 tx=0x7f3cc000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011790 con 0x7f3cd006ddf0 2024-09-21T06:36:12.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd00b3820 con 0x7f3cd006ddf0 2024-09-21T06:36:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cc0017070 con 0x7f3cd006ddf0 2024-09-21T06:36:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011cc0 con 0x7f3cd006ddf0 2024-09-21T06:36:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.058+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd00b3de0 con 0x7f3cd006ddf0 2024-09-21T06:36:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.059+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd00b3290 con 0x7f3cd006ddf0 2024-09-21T06:36:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.061+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3cc001c460 con 0x7f3cd006ddf0 2024-09-21T06:36:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.062+0000 7f3cc6a00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 0x7f3cac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.062+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f3cc009aaf0 con 0x7f3cd006ddf0 2024-09-21T06:36:12.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.063+0000 7f3cc5400640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 0x7f3cac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.063+0000 7f3cc5400640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 0x7f3cac07b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8000c00 tx=0x7f3cb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.067+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3cc0062f60 con 0x7f3cd006ddf0 2024-09-21T06:36:12.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.265+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3cd00731a0 con 0x7f3cac078b90 2024-09-21T06:36:12.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.266+0000 7f3cc6a00640 1 -- 172.21.15.37:0/2832688927 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3cd00731a0 con 0x7f3cac078b90 2024-09-21T06:36:12.267 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 msgr2=0x7f3cac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 0x7f3cac07b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8000c00 tx=0x7f3cb8009040 comp rx=0 tx=0).stop 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 msgr2=0x7f3cd00c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001220 tx=0x7f3cc000c890 comp rx=0 tx=0).stop 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 shutdown_connections 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f3cac078b90 0x7f3cac07b0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cd0156a50 0x7f3cd00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cd0072d90 0x7f3cd00c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 --2- 172.21.15.37:0/2832688927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd006ddf0 0x7f3cd00c7150 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.268+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 >> 172.21.15.37:0/2832688927 conn(0x7f3cd00b1b90 msgr2=0x7f3cd00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.269+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 shutdown_connections 2024-09-21T06:36:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.269+0000 7f3cd4c00640 1 -- 172.21.15.37:0/2832688927 wait complete. 2024-09-21T06:36:12.292 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T06:36:12.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 -- 172.21.15.37:0/2102939352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 msgr2=0x7fcd9c160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 --2- 172.21.15.37:0/2102939352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c160a50 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005fd0 tx=0x7fcd8402a5d0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 -- 172.21.15.37:0/2102939352 shutdown_connections 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 --2- 172.21.15.37:0/2102939352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c160a50 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 --2- 172.21.15.37:0/2102939352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9c0ca690 0x7fcd9c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.521+0000 7fcda1e00640 1 -- 172.21.15.37:0/2102939352 >> 172.21.15.37:0/2102939352 conn(0x7fcd9c0ccac0 msgr2=0x7fcd9c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.522+0000 7fcda1e00640 1 -- 172.21.15.37:0/2102939352 shutdown_connections 2024-09-21T06:36:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.522+0000 7fcda1e00640 1 -- 172.21.15.37:0/2102939352 wait complete. 2024-09-21T06:36:12.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.524+0000 7fcda1e00640 1 Processor -- start 2024-09-21T06:36:12.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.524+0000 7fcda1e00640 1 -- start start 2024-09-21T06:36:12.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9c06ef30 0x7fcd9c1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd9c0ca690 0x7fcd9c0c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c06fa50 con 0x7fcd9c160640 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c06e500 con 0x7fcd9c0ca690 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.525+0000 7fcda1e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c06fbd0 con 0x7fcd9c06ef30 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9aa00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9c06ef30 0x7fcd9c1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd93e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd9c0ca690 0x7fcd9c0c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c14f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50792/0 (socket says 172.21.15.37:50792) 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 -- 172.21.15.37:0/671981672 learned_addr learned my addr 172.21.15.37:0/671981672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 -- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9c06ef30 msgr2=0x7fcd9c1f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd9c06ef30 0x7fcd9c1f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 -- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd9c0ca690 msgr2=0x7fcd9c0c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd9c0ca690 0x7fcd9c0c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.526+0000 7fcd9b400640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c1f46d0 con 0x7fcd9c160640 2024-09-21T06:36:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcd9b400640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c14f310 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c730 tx=0x7fcd9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94017070 con 0x7fcd9c160640 2024-09-21T06:36:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd94003c60 con 0x7fcd9c160640 2024-09-21T06:36:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940052d0 con 0x7fcd9c160640 2024-09-21T06:36:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c09b9a0 con 0x7fcd9c160640 2024-09-21T06:36:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.527+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c0c3bd0 con 0x7fcd9c160640 2024-09-21T06:36:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.528+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd9c160100 con 0x7fcd9c160640 2024-09-21T06:36:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.529+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd9401a070 con 0x7fcd9c160640 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.530+0000 7fcd93400640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd78078b90 0x7fcd7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.530+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fcd9409a520 con 0x7fcd9c160640 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.531+0000 7fcd93400640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7fcd7807f1c0 0x7fcd78081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.531+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd78081d20 con 0x7fcd7807f1c0 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.531+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fcd940a1050 con 0x7fcd9c160640 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.531+0000 7fcd9aa00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd78078b90 0x7fcd7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.531+0000 7fcd93e00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7fcd7807f1c0 0x7fcd78081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.532+0000 7fcd9aa00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd78078b90 0x7fcd7807b0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b30 tx=0x7fcd8c005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.532+0000 7fcd93e00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7fcd7807f1c0 0x7fcd78081650 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.532+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== osd.3 v2:172.21.15.37:6815/1242254511 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd78081d20 con 0x7fcd7807f1c0 2024-09-21T06:36:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.546+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 --> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcd9c0b5f20 con 0x7fcd7807f1c0 2024-09-21T06:36:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.546+0000 7fcd93400640 1 -- 172.21.15.37:0/671981672 <== osd.3 v2:172.21.15.37:6815/1242254511 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcd9c0b5f20 con 0x7fcd7807f1c0 2024-09-21T06:36:12.547 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:12.547 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:12.547 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.546+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7fcd7807f1c0 msgr2=0x7fcd78081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.546+0000 7fcda1e00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:6815/1242254511,v1:172.21.15.37:6817/1242254511] conn(0x7fcd7807f1c0 0x7fcd78081650 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd78078b90 msgr2=0x7fcd7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fcd78078b90 0x7fcd7807b0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b30 tx=0x7fcd8c005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 msgr2=0x7fcd9c14f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 --2- 172.21.15.37:0/671981672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd9c160640 0x7fcd9c14f310 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c730 tx=0x7fcd9400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcd9b400640 1 -- 172.21.15.37:0/671981672 reap_dead start 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 shutdown_connections 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 >> 172.21.15.37:0/671981672 conn(0x7fcd9c0ccac0 msgr2=0x7fcd9c0aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 shutdown_connections 2024-09-21T06:36:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.547+0000 7fcda1e00640 1 -- 172.21.15.37:0/671981672 wait complete. 2024-09-21T06:36:12.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 -- 172.21.15.37:0/1542240511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6314004800 msgr2=0x7f6314004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 --2- 172.21.15.37:0/1542240511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6314004800 0x7f6314004c10 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f631c006f80 tx=0x7f6314005830 comp rx=0 tx=0).stop 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6335000640 1 -- 172.21.15.37:0/1542240511 reap_dead start 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 -- 172.21.15.37:0/1542240511 shutdown_connections 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 -- 172.21.15.37:0/1542240511 >> 172.21.15.37:0/1542240511 conn(0x7f63300b1b90 msgr2=0x7f63300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 -- 172.21.15.37:0/1542240511 shutdown_connections 2024-09-21T06:36:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.797+0000 7f6337000640 1 -- 172.21.15.37:0/1542240511 wait complete. 2024-09-21T06:36:12.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.800+0000 7f6337000640 1 Processor -- start 2024-09-21T06:36:12.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.800+0000 7f6337000640 1 -- start start 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6314004800 0x7f633009c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63300a0180 0x7f63300a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6330072f10 con 0x7f633015e190 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6330072d90 con 0x7f63300a0180 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f6337000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f6330073090 con 0x7f6314004800 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f632f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f632f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50810/0 (socket says 172.21.15.37:50810) 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.801+0000 7f632f400640 1 -- 172.21.15.37:0/306939427 learned_addr learned my addr 172.21.15.37:0/306939427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632fe00640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6314004800 0x7f633009c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f6335000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63300a0180 0x7f63300a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 -- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6314004800 msgr2=0x7f633009c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6314004800 0x7f633009c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 -- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63300a0180 msgr2=0x7f63300a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63300a0180 0x7f63300a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 -- 172.21.15.37:0/306939427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63300a0ad0 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f632f400640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f631c031040 tx=0x7f631c006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.802+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003a00 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.803+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f631c003c90 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.803+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003e60 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.803+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63301f62c0 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.803+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63301f6880 con 0x7f633015e190 2024-09-21T06:36:12.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.804+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6330073560 con 0x7f633015e190 2024-09-21T06:36:12.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.805+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f631c035020 con 0x7f633015e190 2024-09-21T06:36:12.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.806+0000 7f6327e00640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:12.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.806+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f631c0bb250 con 0x7f633015e190 2024-09-21T06:36:12.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.806+0000 7f632fe00640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:12.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.807+0000 7f632fe00640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 0x7f631007b0a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f6320006200 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:12.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:12.810+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63300a0ad0 con 0x7f633015e190 2024-09-21T06:36:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.005+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f63300707d0 con 0x7f6310078b90 2024-09-21T06:36:13.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.006+0000 7f6327e00640 1 -- 172.21.15.37:0/306939427 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f63300707d0 con 0x7f6310078b90 2024-09-21T06:36:13.007 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 msgr2=0x7f631007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 0x7f631007b0a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f6320006200 comp rx=0 tx=0).stop 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 msgr2=0x7f633015e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f631c031040 tx=0x7f631c006c40 comp rx=0 tx=0).stop 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 shutdown_connections 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633015e190 0x7f633015e5a0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63300a0180 0x7f63300a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.008+0000 7f6337000640 1 --2- 172.21.15.37:0/306939427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6314004800 0x7f633009c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.009+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 >> 172.21.15.37:0/306939427 conn(0x7f63300b1b90 msgr2=0x7f63300b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:13.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.009+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 shutdown_connections 2024-09-21T06:36:13.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.009+0000 7f6337000640 1 -- 172.21.15.37:0/306939427 wait complete. 2024-09-21T06:36:13.034 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T06:36:13.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:13.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 -- 172.21.15.37:0/1690092170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cd8e0 msgr2=0x7fd75c0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 --2- 172.21.15.37:0/1690092170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cd8e0 0x7fd75c0cdcf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd75c15ee70 tx=0x7fd75402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 -- 172.21.15.37:0/1690092170 shutdown_connections 2024-09-21T06:36:13.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 --2- 172.21.15.37:0/1690092170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cd8e0 0x7fd75c0cdcf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 --2- 172.21.15.37:0/1690092170 >> v2:172.21.15.150:3300/0 conn(0x7fd75c0cf370 0x7fd75c0717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 -- 172.21.15.37:0/1690092170 >> 172.21.15.37:0/1690092170 conn(0x7fd75c0d1210 msgr2=0x7fd75c0cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:13.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 -- 172.21.15.37:0/1690092170 shutdown_connections 2024-09-21T06:36:13.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.265+0000 7fd761600640 1 -- 172.21.15.37:0/1690092170 wait complete. 2024-09-21T06:36:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.267+0000 7fd761600640 1 Processor -- start 2024-09-21T06:36:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.267+0000 7fd761600640 1 -- start start 2024-09-21T06:36:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c0cd8e0 0x7fd75c150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 0x7fd75c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75c15b920 0x7fd75c14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c0b3ee0 con 0x7fd75c0cf370 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c0b3d60 con 0x7fd75c0cd8e0 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd761600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd75c0b4060 con 0x7fd75c15b920 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 0x7fd75c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 0x7fd75c14d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50828/0 (socket says 172.21.15.37:50828) 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 -- 172.21.15.37:0/2353856081 learned_addr learned my addr 172.21.15.37:0/2353856081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd75be00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c0cd8e0 0x7fd75c150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd760c00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75c15b920 0x7fd75c14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 -- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75c15b920 msgr2=0x7fd75c14fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75c15b920 0x7fd75c14fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 -- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c0cd8e0 msgr2=0x7fd75c150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c0cd8e0 0x7fd75c150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.269+0000 7fd753400640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c0acc40 con 0x7fd75c0cf370 2024-09-21T06:36:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd753400640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 0x7fd75c14d140 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd74400d740 tx=0x7fd74400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744017070 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd744009d20 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c14dd40 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440106c0 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.270+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c14f8a0 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.271+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd75c14e400 con 0x7fd75c0cf370 2024-09-21T06:36:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.272+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd74401a070 con 0x7fd75c0cf370 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.273+0000 7fd758c00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd738078b90 0x7fd73807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.273+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fd74409b370 con 0x7fd75c0cf370 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.274+0000 7fd758c00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7fd73807f1c0 0x7fd738081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.274+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd744026e50 con 0x7fd73807f1c0 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.274+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fd74409b720 con 0x7fd75c0cf370 2024-09-21T06:36:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.274+0000 7fd75be00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd738078b90 0x7fd73807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.274+0000 7fd760c00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7fd73807f1c0 0x7fd738081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.275+0000 7fd75be00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd738078b90 0x7fd73807b0a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd74c001720 tx=0x7fd74c007e90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.275+0000 7fd760c00640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7fd73807f1c0 0x7fd738081650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.276+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== osd.4 v2:172.21.15.150:6818/2746357825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd744026e50 con 0x7fd73807f1c0 2024-09-21T06:36:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.289+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 --> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd75c0ac9f0 con 0x7fd73807f1c0 2024-09-21T06:36:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd758c00640 1 -- 172.21.15.37:0/2353856081 <== osd.4 v2:172.21.15.150:6818/2746357825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd75c0ac9f0 con 0x7fd73807f1c0 2024-09-21T06:36:13.291 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:13.291 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:13.291 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7fd73807f1c0 msgr2=0x7fd738081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6818/2746357825,v1:172.21.15.150:6819/2746357825] conn(0x7fd73807f1c0 0x7fd738081650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd738078b90 msgr2=0x7fd73807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd738078b90 0x7fd73807b0a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd74c001720 tx=0x7fd74c007e90 comp rx=0 tx=0).stop 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 msgr2=0x7fd75c14d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 --2- 172.21.15.37:0/2353856081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75c0cf370 0x7fd75c14d140 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd74400d740 tx=0x7fd74400dc00 comp rx=0 tx=0).stop 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd760c00640 1 -- 172.21.15.37:0/2353856081 reap_dead start 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 shutdown_connections 2024-09-21T06:36:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 >> 172.21.15.37:0/2353856081 conn(0x7fd75c0d1210 msgr2=0x7fd75c0c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 shutdown_connections 2024-09-21T06:36:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.290+0000 7fd761600640 1 -- 172.21.15.37:0/2353856081 wait complete. 2024-09-21T06:36:13.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 -- 172.21.15.37:0/1662155479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528145bf0 msgr2=0x7fc5280d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 --2- 172.21.15.37:0/1662155479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528145bf0 0x7fc5280d05b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc514009830 tx=0x7fc51402a510 comp rx=0 tx=0).stop 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 -- 172.21.15.37:0/1662155479 shutdown_connections 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 --2- 172.21.15.37:0/1662155479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528145bf0 0x7fc5280d05b0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 -- 172.21.15.37:0/1662155479 >> 172.21.15.37:0/1662155479 conn(0x7fc52806fc80 msgr2=0x7fc528070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 -- 172.21.15.37:0/1662155479 shutdown_connections 2024-09-21T06:36:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.537+0000 7fc52ee00640 1 -- 172.21.15.37:0/1662155479 wait complete. 2024-09-21T06:36:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.539+0000 7fc52ee00640 1 Processor -- start 2024-09-21T06:36:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.539+0000 7fc52ee00640 1 -- start start 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 0x7fc528155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc528145bf0 0x7fc528156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc52806d260 con 0x7fc52815d5e0 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc52806d0e0 con 0x7fc528071160 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.540+0000 7fc52ee00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc52806d3e0 con 0x7fc528145bf0 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50834/0 (socket says 172.21.15.37:50834) 2024-09-21T06:36:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 -- 172.21.15.37:0/961185526 learned_addr learned my addr 172.21.15.37:0/961185526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52da00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 0x7fc528155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 -- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc528145bf0 msgr2=0x7fc528156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc528145bf0 0x7fc528156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 -- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 msgr2=0x7fc528155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 0x7fc528155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 -- 172.21.15.37:0/961185526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc528157a60 con 0x7fc52815d5e0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52da00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 0x7fc528155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.541+0000 7fc52e400640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc52000b970 tx=0x7fc52000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.542+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520019070 con 0x7fc52815d5e0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.542+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc52000cc30 con 0x7fc52815d5e0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.542+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000ce60 con 0x7fc52815d5e0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.542+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc528157d40 con 0x7fc52815d5e0 2024-09-21T06:36:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.542+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281587d0 con 0x7fc52815d5e0 2024-09-21T06:36:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.543+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5280d05b0 con 0x7fc52815d5e0 2024-09-21T06:36:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.544+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc52001c070 con 0x7fc52815d5e0 2024-09-21T06:36:13.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.545+0000 7fc525600640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 0x7fc50807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.545+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fc52009a920 con 0x7fc52815d5e0 2024-09-21T06:36:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.546+0000 7fc52da00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 0x7fc50807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.546+0000 7fc52da00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 0x7fc50807b0a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc51c000f60 tx=0x7fc51c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.550+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc520062d80 con 0x7fc52815d5e0 2024-09-21T06:36:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.748+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc5280b08a0 con 0x7fc508078b90 2024-09-21T06:36:13.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.749+0000 7fc525600640 1 -- 172.21.15.37:0/961185526 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5280b08a0 con 0x7fc508078b90 2024-09-21T06:36:13.750 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 msgr2=0x7fc50807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 0x7fc50807b0a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc51c000f60 tx=0x7fc51c009040 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 msgr2=0x7fc52815f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc52000b970 tx=0x7fc52000be30 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 shutdown_connections 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc508078b90 0x7fc50807b0a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52815d5e0 0x7fc52815f000 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc528145bf0 0x7fc528156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 --2- 172.21.15.37:0/961185526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc528071160 0x7fc528155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.751+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 >> 172.21.15.37:0/961185526 conn(0x7fc52806fc80 msgr2=0x7fc52806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.752+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 shutdown_connections 2024-09-21T06:36:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:13.752+0000 7fc52ee00640 1 -- 172.21.15.37:0/961185526 wait complete. 2024-09-21T06:36:13.778 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T06:36:13.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:14.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 -- 172.21.15.37:0/4146268641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca80ca690 msgr2=0x7f2ca8160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 --2- 172.21.15.37:0/4146268641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca80ca690 0x7f2ca8160100 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005fd0 tx=0x7f2c9802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 -- 172.21.15.37:0/4146268641 shutdown_connections 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 --2- 172.21.15.37:0/4146268641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8160640 0x7f2ca8160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 --2- 172.21.15.37:0/4146268641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca80ca690 0x7f2ca8160100 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 -- 172.21.15.37:0/4146268641 >> 172.21.15.37:0/4146268641 conn(0x7f2ca80ccac0 msgr2=0x7f2ca80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 -- 172.21.15.37:0/4146268641 shutdown_connections 2024-09-21T06:36:14.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.008+0000 7f2cad000640 1 -- 172.21.15.37:0/4146268641 wait complete. 2024-09-21T06:36:14.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.010+0000 7f2cad000640 1 Processor -- start 2024-09-21T06:36:14.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.010+0000 7f2cad000640 1 -- start start 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca806ef30 0x7f2ca80c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80ca690 0x7f2ca80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 0x7f2ca8157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca806fa50 con 0x7f2ca8160640 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca806e500 con 0x7f2ca806ef30 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2cad000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca806fbd0 con 0x7f2ca80ca690 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2ca6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 0x7f2ca8157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2ca6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 0x7f2ca8157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42134/0 (socket says 172.21.15.37:42134) 2024-09-21T06:36:14.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2ca6800640 1 -- 172.21.15.37:0/2363800327 learned_addr learned my addr 172.21.15.37:0/2363800327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2ca5e00640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca806ef30 0x7f2ca80c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.012+0000 7f2c9d400640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80ca690 0x7f2ca80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 -- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80ca690 msgr2=0x7f2ca80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80ca690 0x7f2ca80c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 -- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca806ef30 msgr2=0x7f2ca80c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca806ef30 0x7f2ca80c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca80b5a00 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2ca6800640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 0x7f2ca8157850 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c6b0 tx=0x7f2ca000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0003b70 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca0003d10 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.013+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0005330 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.014+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca80b5ce0 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.014+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca80c7470 con 0x7f2ca8160640 2024-09-21T06:36:14.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.014+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ca80ca470 con 0x7f2ca8160640 2024-09-21T06:36:14.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.016+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ca00054d0 con 0x7f2ca8160640 2024-09-21T06:36:14.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2c9ea00640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c84078b90 0x7f2c8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f2ca009b430 con 0x7f2ca8160640 2024-09-21T06:36:14.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2c9ea00640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2c8407f1c0 0x7f2c84081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ca0019e80 con 0x7f2c8407f1c0 2024-09-21T06:36:14.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f2ca009b7e0 con 0x7f2ca8160640 2024-09-21T06:36:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.017+0000 7f2ca5e00640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c84078b90 0x7f2c8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.018+0000 7f2c9d400640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2c8407f1c0 0x7f2c84081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.018+0000 7f2ca5e00640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c84078b90 0x7f2c8407b0a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2c98033000 tx=0x7f2c98004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.018+0000 7f2c9d400640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2c8407f1c0 0x7f2c84081650 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.019+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== osd.5 v2:172.21.15.150:6802/2870078774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ca0019e80 con 0x7f2c8407f1c0 2024-09-21T06:36:14.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.032+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 --> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2ca80b57b0 con 0x7f2c8407f1c0 2024-09-21T06:36:14.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.033+0000 7f2c9ea00640 1 -- 172.21.15.37:0/2363800327 <== osd.5 v2:172.21.15.150:6802/2870078774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2ca80b57b0 con 0x7f2c8407f1c0 2024-09-21T06:36:14.034 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:14.035 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2c8407f1c0 msgr2=0x7f2c84081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6802/2870078774,v1:172.21.15.150:6803/2870078774] conn(0x7f2c8407f1c0 0x7f2c84081650 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c84078b90 msgr2=0x7f2c8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f2c84078b90 0x7f2c8407b0a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2c98033000 tx=0x7f2c98004730 comp rx=0 tx=0).stop 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 msgr2=0x7f2ca8157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 --2- 172.21.15.37:0/2363800327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8160640 0x7f2ca8157850 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c6b0 tx=0x7f2ca000cb70 comp rx=0 tx=0).stop 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2ca6800640 1 -- 172.21.15.37:0/2363800327 reap_dead start 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 shutdown_connections 2024-09-21T06:36:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 >> 172.21.15.37:0/2363800327 conn(0x7f2ca80ccac0 msgr2=0x7f2ca8074640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 shutdown_connections 2024-09-21T06:36:14.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.034+0000 7f2cad000640 1 -- 172.21.15.37:0/2363800327 wait complete. 2024-09-21T06:36:14.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 -- 172.21.15.37:0/944834819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cc0e0 msgr2=0x7f10fc0702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 --2- 172.21.15.37:0/944834819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc0702a0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0ce670 tx=0x7f10f002a4d0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 -- 172.21.15.37:0/944834819 shutdown_connections 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 --2- 172.21.15.37:0/944834819 >> v2:172.21.15.150:3300/0 conn(0x7f10fc0cf6d0 0x7f10fc15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 --2- 172.21.15.37:0/944834819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc0702a0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 -- 172.21.15.37:0/944834819 >> 172.21.15.37:0/944834819 conn(0x7f10fc0d05c0 msgr2=0x7f10fc06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 -- 172.21.15.37:0/944834819 shutdown_connections 2024-09-21T06:36:14.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.287+0000 7f1102a00640 1 -- 172.21.15.37:0/944834819 wait complete. 2024-09-21T06:36:14.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.290+0000 7f1102a00640 1 Processor -- start 2024-09-21T06:36:14.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.290+0000 7f1102a00640 1 -- start start 2024-09-21T06:36:14.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10fc160420 0x7f10fc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc15d000 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0b4390 con 0x7f10fc0cc0e0 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1102a00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc15d180 con 0x7f10fc160420 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1100c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1100c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42158/0 (socket says 172.21.15.37:42158) 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.291+0000 7f1100c00640 1 -- 172.21.15.37:0/1589716048 learned_addr learned my addr 172.21.15.37:0/1589716048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1101600640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1102000640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10fc160420 0x7f10fc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 -- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10fc160420 msgr2=0x7f10fc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10fc160420 0x7f10fc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 -- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 msgr2=0x7f10fc13b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc13b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 -- 172.21.15.37:0/1589716048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc144f00 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1101600640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc13b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1100c00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c970 tx=0x7f10e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8007bb0 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10e8007e40 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e800f6b0 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.292+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc0ca310 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.293+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc0ca8d0 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.294+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc0ca080 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.295+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10e800f850 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.296+0000 7f10f9600640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 0x7f10dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.296+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f10e809b460 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.297+0000 7f1101600640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 0x7f10dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.298+0000 7f1101600640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 0x7f10dc07b0a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f10f0033000 tx=0x7f10f0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.300+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10fc144f00 con 0x7f10fc0cf6d0 2024-09-21T06:36:14.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.498+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f10fc15d610 con 0x7f10dc078b90 2024-09-21T06:36:14.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.499+0000 7f10f9600640 1 -- 172.21.15.37:0/1589716048 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f10fc15d610 con 0x7f10dc078b90 2024-09-21T06:36:14.501 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T06:36:14.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 msgr2=0x7f10dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 0x7f10dc07b0a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f10f0033000 tx=0x7f10f0004730 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 msgr2=0x7f10fc0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c970 tx=0x7f10e800ce30 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 shutdown_connections 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f10dc078b90 0x7f10dc07b0a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc0cf6d0 0x7f10fc0b0750 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10fc160420 0x7f10fc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 --2- 172.21.15.37:0/1589716048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10fc0cc0e0 0x7f10fc13b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 >> 172.21.15.37:0/1589716048 conn(0x7f10fc0d05c0 msgr2=0x7f10fc15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.501+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 shutdown_connections 2024-09-21T06:36:14.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.502+0000 7f1102a00640 1 -- 172.21.15.37:0/1589716048 wait complete. 2024-09-21T06:36:14.526 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T06:36:14.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:14.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.755+0000 7f0964e00640 1 -- 172.21.15.37:0/3659995550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 msgr2=0x7f09601681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.755+0000 7f0964e00640 1 --2- 172.21.15.37:0/3659995550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f09601681c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f09600b7de0 tx=0x7f095802aa30 comp rx=0 tx=0).stop 2024-09-21T06:36:14.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 -- 172.21.15.37:0/3659995550 shutdown_connections 2024-09-21T06:36:14.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 --2- 172.21.15.37:0/3659995550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f09601681c0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 --2- 172.21.15.37:0/3659995550 >> v2:172.21.15.150:3300/0 conn(0x7f09600b4a20 0x7f09600b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 -- 172.21.15.37:0/3659995550 >> 172.21.15.37:0/3659995550 conn(0x7f09600d18c0 msgr2=0x7f09600d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 -- 172.21.15.37:0/3659995550 shutdown_connections 2024-09-21T06:36:14.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.756+0000 7f0964e00640 1 -- 172.21.15.37:0/3659995550 wait complete. 2024-09-21T06:36:14.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.758+0000 7f0964e00640 1 Processor -- start 2024-09-21T06:36:14.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.758+0000 7f0964e00640 1 -- start start 2024-09-21T06:36:14.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09600b4a20 0x7f0960149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f0960150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09600ced00 0x7f0960150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0960168910 con 0x7f09600b5370 2024-09-21T06:36:14.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09600b33b0 con 0x7f09600ced00 2024-09-21T06:36:14.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.759+0000 7f0964e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f0960168a90 con 0x7f09600b4a20 2024-09-21T06:36:14.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f0960150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f0960150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42176/0 (socket says 172.21.15.37:42176) 2024-09-21T06:36:14.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f095f400640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09600b4a20 0x7f0960149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 -- 172.21.15.37:0/3747993051 learned_addr learned my addr 172.21.15.37:0/3747993051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:14.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f095fe00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09600ced00 0x7f0960150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 -- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09600b4a20 msgr2=0x7f0960149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09600b4a20 0x7f0960149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 -- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09600ced00 msgr2=0x7f0960150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09600ced00 0x7f0960150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.760+0000 7f0956a00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960156a60 con 0x7f09600b5370 2024-09-21T06:36:14.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0956a00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f0960150040 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f094800b470 tx=0x7f0948009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948015020 con 0x7f09600b5370 2024-09-21T06:36:14.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0948011940 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948011d90 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0960156d40 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.761+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0960157300 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.762+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09600b4e30 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.763+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0948018020 con 0x7f09600b5370 2024-09-21T06:36:14.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.764+0000 7f0957e00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f093c078bc0 0x7f093c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.764+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f094809ab20 con 0x7f09600b5370 2024-09-21T06:36:14.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.765+0000 7f0957e00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f093c07f1f0 0x7f093c081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:14.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.765+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f094800de50 con 0x7f093c07f1f0 2024-09-21T06:36:14.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.765+0000 7f095f400640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f093c078bc0 0x7f093c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.765+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f094809f2a0 con 0x7f09600b5370 2024-09-21T06:36:14.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.765+0000 7f095fe00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f093c07f1f0 0x7f093c081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:14.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.766+0000 7f095f400640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f093c078bc0 0x7f093c07b0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.766+0000 7f095fe00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f093c07f1f0 0x7f093c081680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:14.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.767+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== osd.6 v2:172.21.15.150:6810/3705202380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f094800de50 con 0x7f093c07f1f0 2024-09-21T06:36:14.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.780+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 --> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0960157dd0 con 0x7f093c07f1f0 2024-09-21T06:36:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0957e00640 1 -- 172.21.15.37:0/3747993051 <== osd.6 v2:172.21.15.150:6810/3705202380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0960157dd0 con 0x7f093c07f1f0 2024-09-21T06:36:14.783 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:14.784 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:14.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f093c07f1f0 msgr2=0x7f093c081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6810/3705202380,v1:172.21.15.150:6811/3705202380] conn(0x7f093c07f1f0 0x7f093c081680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f093c078bc0 msgr2=0x7f093c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f093c078bc0 0x7f093c07b0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950009060 comp rx=0 tx=0).stop 2024-09-21T06:36:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 msgr2=0x7f0960150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 --2- 172.21.15.37:0/3747993051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09600b5370 0x7f0960150040 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f094800b470 tx=0x7f0948009f90 comp rx=0 tx=0).stop 2024-09-21T06:36:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f095fe00640 1 -- 172.21.15.37:0/3747993051 reap_dead start 2024-09-21T06:36:14.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 shutdown_connections 2024-09-21T06:36:14.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 >> 172.21.15.37:0/3747993051 conn(0x7f09600d18c0 msgr2=0x7f09600b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:14.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 shutdown_connections 2024-09-21T06:36:14.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:14.781+0000 7f0964e00640 1 -- 172.21.15.37:0/3747993051 wait complete. 2024-09-21T06:36:14.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 -- 172.21.15.37:0/2621707042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cc0e0 msgr2=0x7fba140702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 --2- 172.21.15.37:0/2621707042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cc0e0 0x7fba140702a0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fba140ce670 tx=0x7fba0402a4b0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 -- 172.21.15.37:0/2621707042 shutdown_connections 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 --2- 172.21.15.37:0/2621707042 >> v2:172.21.15.150:3300/0 conn(0x7fba140cf6d0 0x7fba1415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 --2- 172.21.15.37:0/2621707042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cc0e0 0x7fba140702a0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 -- 172.21.15.37:0/2621707042 >> 172.21.15.37:0/2621707042 conn(0x7fba140d05c0 msgr2=0x7fba1406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 -- 172.21.15.37:0/2621707042 shutdown_connections 2024-09-21T06:36:15.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.030+0000 7fba13e00640 1 -- 172.21.15.37:0/2621707042 wait complete. 2024-09-21T06:36:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.032+0000 7fba13e00640 1 Processor -- start 2024-09-21T06:36:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.033+0000 7fba13e00640 1 -- start start 2024-09-21T06:36:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba140cc0e0 0x7fba14073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 0x7fba140a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba1415d000 con 0x7fba140cf6d0 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fba140b4390 con 0x7fba14160420 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba13e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fba1415d180 con 0x7fba140cc0e0 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba12000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba12000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42198/0 (socket says 172.21.15.37:42198) 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.034+0000 7fba12000640 1 -- 172.21.15.37:0/3993759018 learned_addr learned my addr 172.21.15.37:0/3993759018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12a00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba140cc0e0 0x7fba14073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba13400640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 0x7fba140a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 -- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba140cc0e0 msgr2=0x7fba14073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba140cc0e0 0x7fba14073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 -- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 msgr2=0x7fba140a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 0x7fba140a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 -- 172.21.15.37:0/3993759018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14073ac0 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba13400640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 0x7fba140a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba12000640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c970 tx=0x7fba0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00007bb0 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba00007e40 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba14073f40 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.035+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0000f6b0 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.036+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba14074500 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.036+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1415d610 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.038+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba0000f850 con 0x7fba140cf6d0 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.039+0000 7fba0aa00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.039+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fba0009b480 con 0x7fba140cf6d0 2024-09-21T06:36:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.039+0000 7fba12a00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.040+0000 7fba12a00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 0x7fb9f407b0a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.043+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba00014030 con 0x7fba140cf6d0 2024-09-21T06:36:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.242+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fba14073c90 con 0x7fb9f4078b90 2024-09-21T06:36:15.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.243+0000 7fba0aa00640 1 -- 172.21.15.37:0/3993759018 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fba14073c90 con 0x7fb9f4078b90 2024-09-21T06:36:15.244 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T06:36:15.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 msgr2=0x7fb9f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 0x7fb9f407b0a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04004730 comp rx=0 tx=0).stop 2024-09-21T06:36:15.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 msgr2=0x7fba140cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c970 tx=0x7fba0000ce30 comp rx=0 tx=0).stop 2024-09-21T06:36:15.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 shutdown_connections 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140cf6d0 0x7fba140cb6d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba14160420 0x7fba140a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 --2- 172.21.15.37:0/3993759018 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba140cc0e0 0x7fba14073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 >> 172.21.15.37:0/3993759018 conn(0x7fba140d05c0 msgr2=0x7fba14068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 shutdown_connections 2024-09-21T06:36:15.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.246+0000 7fba13e00640 1 -- 172.21.15.37:0/3993759018 wait complete. 2024-09-21T06:36:15.272 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T06:36:15.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:36:15.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 -- 172.21.15.37:0/2561887705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b40b3aa0 msgr2=0x7fc8b415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 --2- 172.21.15.37:0/2561887705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b40b3aa0 0x7fc8b415f6b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005fd0 tx=0x7fc8a802a5d0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 -- 172.21.15.37:0/2561887705 shutdown_connections 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 --2- 172.21.15.37:0/2561887705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8b40cdff0 0x7fc8b4162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 --2- 172.21.15.37:0/2561887705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b40b3aa0 0x7fc8b415f6b0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 -- 172.21.15.37:0/2561887705 >> 172.21.15.37:0/2561887705 conn(0x7fc8b40d1210 msgr2=0x7fc8b40cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 -- 172.21.15.37:0/2561887705 shutdown_connections 2024-09-21T06:36:15.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.504+0000 7fc8ba600640 1 -- 172.21.15.37:0/2561887705 wait complete. 2024-09-21T06:36:15.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.507+0000 7fc8ba600640 1 Processor -- start 2024-09-21T06:36:15.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.507+0000 7fc8ba600640 1 -- start start 2024-09-21T06:36:15.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8b4162960 0x7fc8b4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b4157f40 0x7fc8b4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 0x7fc8b41571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b40cec30 con 0x7fc8b415c980 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b40ceab0 con 0x7fc8b4157f40 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8ba600640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b40cedb0 con 0x7fc8b4162960 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8b9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 0x7fc8b41571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8b9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 0x7fc8b41571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42202/0 (socket says 172.21.15.37:42202) 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.508+0000 7fc8b9c00640 1 -- 172.21.15.37:0/1153248834 learned_addr learned my addr 172.21.15.37:0/1153248834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9200640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8b4162960 0x7fc8b4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b3e00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b4157f40 0x7fc8b4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 -- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8b4162960 msgr2=0x7fc8b4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8b4162960 0x7fc8b4155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 -- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b4157f40 msgr2=0x7fc8b4156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b4157f40 0x7fc8b4156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4155850 con 0x7fc8b415c980 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.509+0000 7fc8b9c00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 0x7fc8b41571e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e960 tx=0x7fc8ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.510+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004d20 con 0x7fc8b415c980 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.510+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8ac00cb20 con 0x7fc8b415c980 2024-09-21T06:36:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.510+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005020 con 0x7fc8b415c980 2024-09-21T06:36:15.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.510+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b4155ee0 con 0x7fc8b415c980 2024-09-21T06:36:15.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.510+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b41564a0 con 0x7fc8b415c980 2024-09-21T06:36:15.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.511+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8b4069be0 con 0x7fc8b415c980 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.512+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8ac0051c0 con 0x7fc8b415c980 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.513+0000 7fc8b0c00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc894078bc0 0x7fc89407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.513+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fc8ac09b590 con 0x7fc8b415c980 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.513+0000 7fc8b0c00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7fc89407f1f0 0x7fc894081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.513+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8ac009050 con 0x7fc89407f1f0 2024-09-21T06:36:15.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.513+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac0a2050 con 0x7fc8b415c980 2024-09-21T06:36:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.514+0000 7fc8b9200640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc894078bc0 0x7fc89407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.514+0000 7fc8b3e00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7fc89407f1f0 0x7fc894081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.514+0000 7fc8b9200640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc894078bc0 0x7fc89407b0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8033000 tx=0x7fc8a8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.514+0000 7fc8b3e00640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7fc89407f1f0 0x7fc894081680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.515+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== osd.7 v2:172.21.15.150:6826/1641077969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8ac009050 con 0x7fc89407f1f0 2024-09-21T06:36:15.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.529+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 --> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc8b40b2220 con 0x7fc89407f1f0 2024-09-21T06:36:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.529+0000 7fc8b0c00640 1 -- 172.21.15.37:0/1153248834 <== osd.7 v2:172.21.15.150:6826/1641077969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc8b40b2220 con 0x7fc89407f1f0 2024-09-21T06:36:15.531 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-21T06:36:15.531 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:36:15.532 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7fc89407f1f0 msgr2=0x7fc894081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6826/1641077969,v1:172.21.15.150:6827/1641077969] conn(0x7fc89407f1f0 0x7fc894081680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc894078bc0 msgr2=0x7fc89407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fc894078bc0 0x7fc89407b0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8033000 tx=0x7fc8a8004730 comp rx=0 tx=0).stop 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 msgr2=0x7fc8b41571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 --2- 172.21.15.37:0/1153248834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b415c980 0x7fc8b41571e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e960 tx=0x7fc8ac00ee20 comp rx=0 tx=0).stop 2024-09-21T06:36:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8b9c00640 1 -- 172.21.15.37:0/1153248834 reap_dead start 2024-09-21T06:36:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 shutdown_connections 2024-09-21T06:36:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 >> 172.21.15.37:0/1153248834 conn(0x7fc8b40d1210 msgr2=0x7fc8b4073ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 shutdown_connections 2024-09-21T06:36:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.530+0000 7fc8ba600640 1 -- 172.21.15.37:0/1153248834 wait complete. 2024-09-21T06:36:15.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T06:36:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.777+0000 7fbd93e00640 1 -- 172.21.15.37:0/978987188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94152270 msgr2=0x7fbd940cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.777+0000 7fbd93e00640 1 --2- 172.21.15.37:0/978987188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94152270 0x7fbd940cf210 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009830 tx=0x7fbd8402a530 comp rx=0 tx=0).stop 2024-09-21T06:36:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.778+0000 7fbd93e00640 1 -- 172.21.15.37:0/978987188 shutdown_connections 2024-09-21T06:36:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.778+0000 7fbd93e00640 1 --2- 172.21.15.37:0/978987188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94152270 0x7fbd940cf210 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.778+0000 7fbd93e00640 1 -- 172.21.15.37:0/978987188 >> 172.21.15.37:0/978987188 conn(0x7fbd940d05c0 msgr2=0x7fbd9406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.778+0000 7fbd93e00640 1 -- 172.21.15.37:0/978987188 shutdown_connections 2024-09-21T06:36:15.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.778+0000 7fbd93e00640 1 -- 172.21.15.37:0/978987188 wait complete. 2024-09-21T06:36:15.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.780+0000 7fbd93e00640 1 Processor -- start 2024-09-21T06:36:15.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.780+0000 7fbd93e00640 1 -- start start 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.781+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.781+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd94152270 0x7fbd94159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.781+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 0x7fbd94148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.781+0000 7fbd93e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9415c870 con 0x7fbd940cf750 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.781+0000 7fbd93e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9415c6f0 con 0x7fbd94152680 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd93e00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fbd94074290 con 0x7fbd94152270 2024-09-21T06:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42210/0 (socket says 172.21.15.37:42210) 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 -- 172.21.15.37:0/3989904897 learned_addr learned my addr 172.21.15.37:0/3989904897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd93400640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 0x7fbd94148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92000640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd94152270 0x7fbd94159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 -- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd94152270 msgr2=0x7fbd94159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd94152270 0x7fbd94159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 -- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 msgr2=0x7fbd94148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 0x7fbd94148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd92a00640 1 -- 172.21.15.37:0/3989904897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd94159ef0 con 0x7fbd940cf750 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.782+0000 7fbd93400640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 0x7fbd94148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd92a00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009950 tx=0x7fbd84033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84031b60 con 0x7fbd940cf750 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd84031df0 con 0x7fbd940cf750 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9415a1d0 con 0x7fbd940cf750 2024-09-21T06:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84005280 con 0x7fbd940cf750 2024-09-21T06:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.783+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9415a790 con 0x7fbd940cf750 2024-09-21T06:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.784+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd94074810 con 0x7fbd940cf750 2024-09-21T06:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.785+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd84005420 con 0x7fbd940cf750 2024-09-21T06:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.786+0000 7fbd8aa00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 0x7fbd7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.786+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fbd840bbe70 con 0x7fbd940cf750 2024-09-21T06:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.787+0000 7fbd92000640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 0x7fbd7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.787+0000 7fbd92000640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 0x7fbd7407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbd941f6a50 tx=0x7fbd80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.791+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd8403f030 con 0x7fbd940cf750 2024-09-21T06:36:15.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.989+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fbd9415bd60 con 0x7fbd74078b90 2024-09-21T06:36:15.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.990+0000 7fbd8aa00640 1 -- 172.21.15.37:0/3989904897 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbd9415bd60 con 0x7fbd74078b90 2024-09-21T06:36:15.991 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 msgr2=0x7fbd7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 0x7fbd7407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbd941f6a50 tx=0x7fbd80009040 comp rx=0 tx=0).stop 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 msgr2=0x7fbd941f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009950 tx=0x7fbd84033c10 comp rx=0 tx=0).stop 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 shutdown_connections 2024-09-21T06:36:15.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fbd74078b90 0x7fbd7407b0a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94152680 0x7fbd94148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd94152270 0x7fbd94159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 --2- 172.21.15.37:0/3989904897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd940cf750 0x7fbd941f2de0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 >> 172.21.15.37:0/3989904897 conn(0x7fbd940d05c0 msgr2=0x7fbd9406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 shutdown_connections 2024-09-21T06:36:15.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:15.992+0000 7fbd93e00640 1 -- 172.21.15.37:0/3989904897 wait complete. 2024-09-21T06:36:16.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:36:16.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 -- 172.21.15.37:0/3118389150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec06d9c0 msgr2=0x7fd0ec154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3118389150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec06d9c0 0x7fd0ec154ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005fd0 tx=0x7fd0e402a940 comp rx=0 tx=0).stop 2024-09-21T06:36:16.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 -- 172.21.15.37:0/3118389150 shutdown_connections 2024-09-21T06:36:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3118389150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec06d9c0 0x7fd0ec154ea0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 -- 172.21.15.37:0/3118389150 >> 172.21.15.37:0/3118389150 conn(0x7fd0ec0d1930 msgr2=0x7fd0ec0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 -- 172.21.15.37:0/3118389150 shutdown_connections 2024-09-21T06:36:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.243+0000 7fd0f1200640 1 -- 172.21.15.37:0/3118389150 wait complete. 2024-09-21T06:36:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.246+0000 7fd0f1200640 1 Processor -- start 2024-09-21T06:36:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.246+0000 7fd0f1200640 1 -- start start 2024-09-21T06:36:16.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 0x7fd0ec151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec0b9e40 0x7fd0ec0b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec155560 con 0x7fd0ec0cd250 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1553e0 con 0x7fd0ec0b9e40 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0f1200640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec1556e0 con 0x7fd0ec06d9c0 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.247+0000 7fd0ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42222/0 (socket says 172.21.15.37:42222) 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 -- 172.21.15.37:0/3060119432 learned_addr learned my addr 172.21.15.37:0/3060119432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0e3e00640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec0b9e40 0x7fd0ec0b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0eb400640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 0x7fd0ec151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 -- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 msgr2=0x7fd0ec151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 0x7fd0ec151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 -- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec0b9e40 msgr2=0x7fd0ec0b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec0b9e40 0x7fd0ec0b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 -- 172.21.15.37:0/3060119432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec15e330 con 0x7fd0ec0cd250 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0eb400640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 0x7fd0ec151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0ebe00640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009cd0 tx=0x7fd0e4031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.248+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e403e070 con 0x7fd0ec0cd250 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.249+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0e4004560 con 0x7fd0ec0cd250 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.249+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4043460 con 0x7fd0ec0cd250 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.249+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec15d890 con 0x7fd0ec0cd250 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.249+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec15db30 con 0x7fd0ec0cd250 2024-09-21T06:36:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.250+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ec0cd7a0 con 0x7fd0ec0cd250 2024-09-21T06:36:16.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.251+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0e403a440 con 0x7fd0ec0cd250 2024-09-21T06:36:16.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.253+0000 7fd0e3400640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 0x7fd0c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.253+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fd0e40bc780 con 0x7fd0ec0cd250 2024-09-21T06:36:16.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.253+0000 7fd0eb400640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 0x7fd0c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.254+0000 7fd0eb400640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 0x7fd0c807b0d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc007fd0 tx=0x7fd0dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.257+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0e4084c70 con 0x7fd0ec0cd250 2024-09-21T06:36:16.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.449+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 --> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0ec162f10 con 0x7fd0c8078bc0 2024-09-21T06:36:16.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.451+0000 7fd0e3400640 1 -- 172.21.15.37:0/3060119432 <== mgr.4109 v2:172.21.15.150:6800/3586070453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7fd0ec162f10 con 0x7fd0c8078bc0 2024-09-21T06:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:16.453 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 msgr2=0x7fd0c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 0x7fd0c807b0d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc007fd0 tx=0x7fd0dc009040 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 msgr2=0x7fd0ec0b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009cd0 tx=0x7fd0e4031ea0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 shutdown_connections 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fd0c8078bc0 0x7fd0c807b0d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0cd250 0x7fd0ec0b8d40 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0ec0b9e40 0x7fd0ec0b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 --2- 172.21.15.37:0/3060119432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec06d9c0 0x7fd0ec151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 >> 172.21.15.37:0/3060119432 conn(0x7fd0ec0d1930 msgr2=0x7fd0ec154700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 shutdown_connections 2024-09-21T06:36:16.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.453+0000 7fd0f1200640 1 -- 172.21.15.37:0/3060119432 wait complete. 2024-09-21T06:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-21T06:36:15.227146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261236,"kb_used_data":11720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713484,"statfs":{"total":773094113280,"available":772826607616,"internally_reserved":0,"allocated":12001280,"data_stored":6210152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002762"},"pg_stats":[{"pgid":"1.0","version":"109'74","reported_seq":284,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T06:36:12.095758+0000","last_change":"2024-09-21T06:36:12.095758+0000","last_active":"2024-09-21T06:36:12.095758+0000","last_peered":"2024-09-21T06:36:12.095758+0000","last_clean":"2024-09-21T06:36:12.095758+0000","last_became_active":"2024-09-21T06:27:50.899552+0000","last_became_peered":"2024-09-21T06:27:50.899552+0000","last_unstale":"2024-09-21T06:36:12.095758+0000","last_undegraded":"2024-09-21T06:36:12.095758+0000","last_fullsized":"2024-09-21T06:36:12.095758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'74","last_scrub_stamp":"2024-09-21T06:36:12.095685+0000","last_deep_scrub":"109'74","last_deep_scrub_stamp":"2024-09-21T06:36:12.095685+0000","last_clean_scrub_stamp":"2024-09-21T06:36:12.095685+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:59.046547+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1945600,"data_stored":1219045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1351680,"data_stored":628677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1945600,"data_stored":1219045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:16.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T06:36:16.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 -- 172.21.15.37:0/2833285182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7498004800 msgr2=0x7f7498004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 --2- 172.21.15.37:0/2833285182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7498004800 0x7f7498004c10 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f74b4072d40 tx=0x7f7498005830 comp rx=0 tx=0).stop 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74b8c00640 1 -- 172.21.15.37:0/2833285182 reap_dead start 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 -- 172.21.15.37:0/2833285182 shutdown_connections 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 -- 172.21.15.37:0/2833285182 >> 172.21.15.37:0/2833285182 conn(0x7f74b40b1b90 msgr2=0x7f74b40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 -- 172.21.15.37:0/2833285182 shutdown_connections 2024-09-21T06:36:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.707+0000 7f74bac00640 1 -- 172.21.15.37:0/2833285182 wait complete. 2024-09-21T06:36:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.709+0000 7f74bac00640 1 Processor -- start 2024-09-21T06:36:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.710+0000 7f74bac00640 1 -- start start 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7498004800 0x7f74b40b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 0x7f74b40c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74b40cef00 con 0x7f74b40c4cd0 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74b406e710 con 0x7f74b4155e90 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74bac00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7f74b40cf080 con 0x7f7498004800 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74b8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74b8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42252/0 (socket says 172.21.15.37:42252) 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.711+0000 7f74b8c00640 1 -- 172.21.15.37:0/605357229 learned_addr learned my addr 172.21.15.37:0/605357229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 -- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7498004800 msgr2=0x7f74b40b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7498004800 0x7f74b40b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 -- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 msgr2=0x7f74b40c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74abe00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 0x7f74b40c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 0x7f74b40c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 -- 172.21.15.37:0/605357229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b40b4b10 con 0x7f74b40c4cd0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74abe00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 0x7f74b40c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b8c00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005fd0 tx=0x7f74ac00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac004b20 con 0x7f74b40c4cd0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74ac004cc0 con 0x7f74b40c4cd0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac003bd0 con 0x7f74b40c4cd0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b41f4d80 con 0x7f74b40c4cd0 2024-09-21T06:36:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.712+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b41f5020 con 0x7f74b40c4cd0 2024-09-21T06:36:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.713+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b41f5a20 con 0x7f74b40c4cd0 2024-09-21T06:36:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.715+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74ac004030 con 0x7f74b40c4cd0 2024-09-21T06:36:16.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.716+0000 7f74b0c00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 0x7f749007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.716+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f74ac09b1c0 con 0x7f74b40c4cd0 2024-09-21T06:36:16.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.716+0000 7f74b3e00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 0x7f749007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.717+0000 7f74b3e00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 0x7f749007b0a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009b50 tx=0x7f74a4009f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.720+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74b41f4d80 con 0x7f74b40c4cd0 2024-09-21T06:36:16.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.919+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f74b4074620 con 0x7f74b40c4cd0 2024-09-21T06:36:16.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.923+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v44)=0 v44) ==== 123+0+0 (secure 0 0 0) 0x7f74ac063620 con 0x7f74b40c4cd0 2024-09-21T06:36:16.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.924+0000 7f74b0c00640 1 -- 172.21.15.37:0/605357229 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f74ac062a30 con 0x7f74b40c4cd0 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.926+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 msgr2=0x7f749007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.926+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 0x7f749007b0a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009b50 tx=0x7f74a4009f50 comp rx=0 tx=0).stop 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.926+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 msgr2=0x7f74b40b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.926+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005fd0 tx=0x7f74ac00f3c0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 shutdown_connections 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7f7490078b90 0x7f749007b0a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b40c4cd0 0x7f74b40b2ed0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b4155e90 0x7f74b40c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 --2- 172.21.15.37:0/605357229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7498004800 0x7f74b40b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 >> 172.21.15.37:0/605357229 conn(0x7f74b40b1b90 msgr2=0x7f74b40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 shutdown_connections 2024-09-21T06:36:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:16.927+0000 7f74bac00640 1 -- 172.21.15.37:0/605357229 wait complete. 2024-09-21T06:36:16.950 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:36:16.950 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.181+0000 7fa2dbe00640 1 -- 172.21.15.37:0/2538778539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 msgr2=0x7fa2d406ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.181+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/2538778539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d406ddf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009830 tx=0x7fa2c002a4a0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 -- 172.21.15.37:0/2538778539 shutdown_connections 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/2538778539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 0x7fa2d406ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/2538778539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d406ddf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 -- 172.21.15.37:0/2538778539 >> 172.21.15.37:0/2538778539 conn(0x7fa2d40cca40 msgr2=0x7fa2d40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 -- 172.21.15.37:0/2538778539 shutdown_connections 2024-09-21T06:36:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.182+0000 7fa2dbe00640 1 -- 172.21.15.37:0/2538778539 wait complete. 2024-09-21T06:36:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.184+0000 7fa2dbe00640 1 Processor -- start 2024-09-21T06:36:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.184+0000 7fa2dbe00640 1 -- start start 2024-09-21T06:36:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 0x7fa2d4158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d4157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d406f4a0 con 0x7fa2d4074cb0 2024-09-21T06:36:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d406f320 con 0x7fa2d4073dd0 2024-09-21T06:36:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.185+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d406f620 con 0x7fa2d406e330 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42266/0 (socket says 172.21.15.37:42266) 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 -- 172.21.15.37:0/1451945404 learned_addr learned my addr 172.21.15.37:0/1451945404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d8a00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9400640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 0x7fa2d4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 -- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 msgr2=0x7fa2d4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 0x7fa2d4158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 -- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 msgr2=0x7fa2d4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d4157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.186+0000 7fa2d9e00640 1 -- 172.21.15.37:0/1451945404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41f6ab0 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2d9e00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ea40 tx=0x7fa2cc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00ce40 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fa2cc004c30 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d41f5f30 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc005020 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.187+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41f64f0 con 0x7fa2d4074cb0 2024-09-21T06:36:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.188+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d406ede0 con 0x7fa2d4074cb0 2024-09-21T06:36:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.189+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa2cc0051c0 con 0x7fa2d4074cb0 2024-09-21T06:36:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.190+0000 7fa2d1600640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 0x7fa2b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.190+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fa2cc09ba10 con 0x7fa2d4074cb0 2024-09-21T06:36:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.191+0000 7fa2d9400640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 0x7fa2b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.191+0000 7fa2d9400640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 0x7fa2b407b0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8001040 tx=0x7fa2c8000fd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:17.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.194+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa2cc063830 con 0x7fa2d4074cb0 2024-09-21T06:36:17.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.428+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d41f5c90 con 0x7fa2d4074cb0 2024-09-21T06:36:17.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.428+0000 7fa2d1600640 1 -- 172.21.15.37:0/1451945404 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+3919 (secure 0 0 0) 0x7fa2cc068260 con 0x7fa2d4074cb0 2024-09-21T06:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-21T06:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":120,"btime":"2024-09-21T06:36:01:166455+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6836/1262724555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1262724555},{"type":"v1","addr":"172.21.15.150:6836","nonce":1262724555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/430299811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6835","nonce":430299811},{"type":"v1","addr":"172.21.15.150:6837","nonce":430299811}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4203090791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4203090791},{"type":"v1","addr":"172.21.15.37:6833","nonce":4203090791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2683956020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2683956020},{"type":"v1","addr":"172.21.15.37:6835","nonce":2683956020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[]} 2024-09-21T06:36:17.430 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 120 2024-09-21T06:36:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 msgr2=0x7fa2b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 0x7fa2b407b0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8001040 tx=0x7fa2c8000fd0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 msgr2=0x7fa2d40b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ea40 tx=0x7fa2cc00ef00 comp rx=0 tx=0).stop 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 shutdown_connections 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:6800/3586070453,v1:172.21.15.150:6801/3586070453] conn(0x7fa2b4078b90 0x7fa2b407b0a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d4074cb0 0x7fa2d40b5310 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2d4073dd0 0x7fa2d4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 --2- 172.21.15.37:0/1451945404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2d406e330 0x7fa2d4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 >> 172.21.15.37:0/1451945404 conn(0x7fa2d40cca40 msgr2=0x7fa2d40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 shutdown_connections 2024-09-21T06:36:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-21T06:36:17.431+0000 7fa2dbe00640 1 -- 172.21.15.37:0/1451945404 wait complete. 2024-09-21T06:36:17.454 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T06:36:17.454 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T06:36:17.454 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:36:17.454 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.478 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:36:17.479 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:36:17.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.542 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:36:17.542 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:36:17.542 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.568 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:36:17.568 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:36:17.568 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.634 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:36:17.634 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T06:36:17.635 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T06:36:17.635 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.679 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T06:36:17.680 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T06:36:17.680 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.738 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T06:36:17.738 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T06:36:17.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.802 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T06:36:17.802 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T06:36:17.802 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.870 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T06:36:17.870 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T06:36:17.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.918 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T06:36:17.918 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T06:36:17.918 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:17.973 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T06:36:17.974 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T06:36:17.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.032 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T06:36:18.032 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T06:36:18.032 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.096 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T06:36:18.097 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T06:36:18.097 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T06:36:18.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.194 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T06:36:18.194 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T06:36:18.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.269 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T06:36:18.269 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T06:36:18.270 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T06:36:18.270 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.302 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T06:36:18.302 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T06:36:18.302 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.328 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T06:36:18.328 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T06:36:18.329 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-21T06:36:18.395 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T06:36:18.395 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T06:36:18.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Scrub error on inode' | head -n 1 2024-09-21T06:36:18.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:36:18.476 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T06:36:18.771 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:36:18.771 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T06:36:18.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:36:18.874 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T06:36:18.974 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:36:18.974 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T06:36:19.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:36:19.074 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T06:36:19.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:36:19.284 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T06:36:19.393 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:36:19.393 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T06:36:19.491 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:36:19.491 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T06:36:19.629 INFO:tasks.ceph:Archiving mon data... 2024-09-21T06:36:19.630 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/data/mon.a.tgz 2024-09-21T06:36:19.630 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-21T06:36:19.667 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.kw8Oi11QDp 2024-09-21T06:36:19.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kw8Oi11QDp 2024-09-21T06:36:19.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.kw8Oi11QDp 2024-09-21T06:36:19.960 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.kw8Oi11QDp is 433KB 2024-09-21T06:36:20.049 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.kw8Oi11QDp 2024-09-21T06:36:20.084 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/data/mon.b.tgz 2024-09-21T06:36:20.084 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T06:36:20.119 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.90R7bljqCx 2024-09-21T06:36:20.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.90R7bljqCx 2024-09-21T06:36:20.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.90R7bljqCx 2024-09-21T06:36:20.411 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.90R7bljqCx is 433KB 2024-09-21T06:36:20.499 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.90R7bljqCx 2024-09-21T06:36:20.534 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/data/mon.c.tgz 2024-09-21T06:36:20.534 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T06:36:20.608 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.rGbsGnHnMB 2024-09-21T06:36:20.608 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rGbsGnHnMB 2024-09-21T06:36:20.786 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.rGbsGnHnMB 2024-09-21T06:36:20.881 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.rGbsGnHnMB is 433KB 2024-09-21T06:36:20.970 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.rGbsGnHnMB 2024-09-21T06:36:21.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.004 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T06:36:21.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T06:36:21.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T06:36:21.172 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T06:36:21.207 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T06:36:21.207 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T06:36:21.208 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi037/crash 2024-09-21T06:36:21.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T06:36:21.269 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi150/crash 2024-09-21T06:36:21.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T06:36:21.324 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T06:36:21.324 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.329 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T06:36:21.541 INFO:tasks.ceph:Compressing logs... 2024-09-21T06:36:21.542 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:36:21.546 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:36:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T06:36:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2024-09-21T06:36:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2024-09-21T06:36:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55221.log 2024-09-21T06:36:21.599 INFO:teuthology.orchestra.run.smithi037.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2024-09-21T06:36:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48282.log 2024-09-21T06:36:21.601 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2024-09-21T06:36:21.601 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36215.log 2024-09-21T06:36:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55221.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55221.log.gz 2024-09-21T06:36:21.602 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2024-09-21T06:36:21.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36255.log 2024-09-21T06:36:21.603 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54112.log 2024-09-21T06:36:21.603 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48282.log.gz 2024-09-21T06:36:21.603 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55911.log 2024-09-21T06:36:21.603 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2024-09-21T06:36:21.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36215.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36215.log.gz 2024-09-21T06:36:21.605 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41535.log 2024-09-21T06:36:21.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36255.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36255.log.gz 2024-09-21T06:36:21.605 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T06:36:21.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.1.41535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36495.log 0.0% 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.1.41535.log.gz 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71066.log 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55911.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54112.log.gz 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55911.log.gz 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57550.log 2024-09-21T06:36:21.606 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71693.log 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36495.log.gz 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36535.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71066.log.gz 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42493.log 2024-09-21T06:36:21.608 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57550.log.gz 2024-09-21T06:36:21.609 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-09-21T06:36:21.609 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71693.log.gz 2024-09-21T06:36:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41930.log 2024-09-21T06:36:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T06:36:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.3.41930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41930.log.gz 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42493.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42493.log.gz 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58990.log 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65963.log 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58990.log.gz 2024-09-21T06:36:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43835.log 2024-09-21T06:36:21.612 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-09-21T06:36:21.613 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36175.log 2024-09-21T06:36:21.613 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T06:36:21.613 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65963.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65963.log.gz 2024-09-21T06:36:21.613 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70982.log 2024-09-21T06:36:21.613 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57999.log 2024-09-21T06:36:21.614 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43835.log.gz 2024-09-21T06:36:21.614 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45224.log 2024-09-21T06:36:21.614 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70982.log.gz 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36175.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi150.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36175.log.gz 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57999.log.gz 2024-09-21T06:36:21.615 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45224.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45224.log.gz 2024-09-21T06:36:21.616 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44898.log 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.6.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51745.log 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-09-21T06:36:21.617 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44898.log.gz 2024-09-21T06:36:21.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48998.log 2024-09-21T06:36:21.618 INFO:teuthology.orchestra.run.smithi037.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51745.log.gz 2024-09-21T06:36:21.619 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T06:36:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46638.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-09-21T06:36:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63596.log 2024-09-21T06:36:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.48998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48998.log.gz 2024-09-21T06:36:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71024.log 2024-09-21T06:36:21.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-09-21T06:36:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71024.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63596.log.gz 2024-09-21T06:36:21.621 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70681.log 2024-09-21T06:36:21.622 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71024.log.gz 2024-09-21T06:36:21.622 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T06:36:21.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54262.log 2024-09-21T06:36:21.623 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-09-21T06:36:21.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70615.log 2024-09-21T06:36:21.623 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70681.log.gz 2024-09-21T06:36:21.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47066.log 2024-09-21T06:36:21.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54262.log.gz 2024-09-21T06:36:21.625 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mgr.x.log: 92.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-09-21T06:36:21.625 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T06:36:21.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48194.log 2024-09-21T06:36:21.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47066.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70615.log.gz 2024-09-21T06:36:21.626 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47066.log.gz 2024-09-21T06:36:21.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67588.log 2024-09-21T06:36:21.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-09-21T06:36:21.627 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48194.log.gz 2024-09-21T06:36:21.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36387.log 2024-09-21T06:36:21.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67588.log.gz 2024-09-21T06:36:21.628 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41934.log 2024-09-21T06:36:21.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58530.log 2024-09-21T06:36:21.629 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-09-21T06:36:21.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36387.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36387.log.gz 2024-09-21T06:36:21.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42109.log 2024-09-21T06:36:21.631 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36335.log 2024-09-21T06:36:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71857.log 2024-09-21T06:36:21.631 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58530.log.gz 2024-09-21T06:36:21.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69709.log 2024-09-21T06:36:21.631 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42109.log.gz 2024-09-21T06:36:21.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53914.log 2024-09-21T06:36:21.632 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71857.log.gz 2024-09-21T06:36:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69709.log.gz 2024-09-21T06:36:21.633 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46333.log 2024-09-21T06:36:21.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.3.41934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41934.log.gz 2024-09-21T06:36:21.634 INFO:teuthology.orchestra.run.smithi150.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-09-21T06:36:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T06:36:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41516.log 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52052.log 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53914.log.gz 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.46333.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.2.46333.log.gz 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66623.log 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-09-21T06:36:21.635 INFO:teuthology.orchestra.run.smithi037.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52052.log.gz 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi150.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36335.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36295.log 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66623.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66623.log.gz 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68446.log 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-09-21T06:36:21.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-09-21T06:36:21.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61731.log 2024-09-21T06:36:21.638 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68446.log.gz 2024-09-21T06:36:21.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-09-21T06:36:21.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61731.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-09-21T06:36:21.639 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61731.log.gz 2024-09-21T06:36:21.640 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63014.log 2024-09-21T06:36:21.641 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.1.41516.log: /var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.36295.log: 0.0% 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36295.log.gz 2024-09-21T06:36:21.641 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.1.41516.log.gz 2024-09-21T06:36:21.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59947.log 2024-09-21T06:36:21.641 INFO:teuthology.orchestra.run.smithi037.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-09-21T06:36:21.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55389.log 2024-09-21T06:36:21.642 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63014.log.gz 2024-09-21T06:36:21.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64784.log 2024-09-21T06:36:21.642 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59947.log.gz 2024-09-21T06:36:21.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55389.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55389.log.gz 2024-09-21T06:36:21.643 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66845.log 2024-09-21T06:36:21.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62322.log 2024-09-21T06:36:21.644 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64784.log.gz 2024-09-21T06:36:21.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47570.log 2024-09-21T06:36:21.645 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66845.log.gz 2024-09-21T06:36:21.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-09-21T06:36:21.646 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62322.log.gz 2024-09-21T06:36:21.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47570.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47570.log.gz 2024-09-21T06:36:21.647 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-09-21T06:36:21.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72734.log 2024-09-21T06:36:21.647 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-09-21T06:36:21.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-09-21T06:36:21.648 INFO:teuthology.orchestra.run.smithi037.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-09-21T06:36:21.649 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68227.log 2024-09-21T06:36:21.649 INFO:teuthology.orchestra.run.smithi037.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.72734.log.gz 2024-09-21T06:36:21.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51316.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-09-21T06:36:21.650 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66749.log 2024-09-21T06:36:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71210.log 2024-09-21T06:36:21.651 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68227.log.gz 2024-09-21T06:36:21.652 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60322.log 2024-09-21T06:36:21.652 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66749.log.gz 2024-09-21T06:36:21.653 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70003.log 2024-09-21T06:36:21.653 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71210.log.gz 2024-09-21T06:36:21.654 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60322.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60322.log.gz 2024-09-21T06:36:21.654 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-09-21T06:36:21.654 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68863.log 2024-09-21T06:36:21.655 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70003.log.gz 2024-09-21T06:36:21.655 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53848.log 2024-09-21T06:36:21.655 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-09-21T06:36:21.656 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71899.log 2024-09-21T06:36:21.657 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53848.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68863.log.gz 2024-09-21T06:36:21.657 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36891.log 2024-09-21T06:36:21.657 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53848.log.gz 2024-09-21T06:36:21.658 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69667.log 2024-09-21T06:36:21.658 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71899.log.gz 2024-09-21T06:36:21.659 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63915.log 2024-09-21T06:36:21.659 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36891.log.gz 2024-09-21T06:36:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61030.log 2024-09-21T06:36:21.660 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69667.log.gz 2024-09-21T06:36:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59419.log 2024-09-21T06:36:21.661 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63915.log.gz 2024-09-21T06:36:21.661 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57476.log 2024-09-21T06:36:21.662 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61030.log.gz 2024-09-21T06:36:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70940.log 2024-09-21T06:36:21.663 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59419.log.gz 2024-09-21T06:36:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-09-21T06:36:21.663 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57476.log.gz 2024-09-21T06:36:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71651.log 2024-09-21T06:36:21.664 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70940.log.gz 2024-09-21T06:36:21.665 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46722.log 2024-09-21T06:36:21.665 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-09-21T06:36:21.666 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-09-21T06:36:21.666 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71651.log.gz 2024-09-21T06:36:21.667 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46722.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46722.log.gz 2024-09-21T06:36:21.667 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58678.log 2024-09-21T06:36:21.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-09-21T06:36:21.668 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-09-21T06:36:21.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58678.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29916.log 2024-09-21T06:36:21.668 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58678.log.gz 2024-09-21T06:36:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50385.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T06:36:21.670 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-09-21T06:36:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.tmp-client.admin.29916.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29916.log.gz 2024-09-21T06:36:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54178.log 2024-09-21T06:36:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-09-21T06:36:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57244.log 2024-09-21T06:36:21.672 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54178.log.gz 2024-09-21T06:36:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46985.log 2024-09-21T06:36:21.673 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-09-21T06:36:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67378.log 2024-09-21T06:36:21.674 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57244.log.gz 2024-09-21T06:36:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50882.log 2024-09-21T06:36:21.675 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46985.log.gz 2024-09-21T06:36:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67378.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67378.log.gz 2024-09-21T06:36:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62576.log 2024-09-21T06:36:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-09-21T06:36:21.677 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50882.log.gz 2024-09-21T06:36:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62576.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62576.log.gz 2024-09-21T06:36:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64454.log 2024-09-21T06:36:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54346.log 2024-09-21T06:36:21.679 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-09-21T06:36:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-09-21T06:36:21.680 INFO:teuthology.orchestra.run.smithi037.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.64454.log.gz 2024-09-21T06:36:21.681 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58572.log 2024-09-21T06:36:21.681 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54346.log.gz 2024-09-21T06:36:21.682 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55347.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-09-21T06:36:21.682 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-09-21T06:36:21.683 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44902.log 2024-09-21T06:36:21.683 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58572.log.gz 2024-09-21T06:36:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46806.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-09-21T06:36:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36121.log 2024-09-21T06:36:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50274.log 2024-09-21T06:36:21.685 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44902.log.gz 2024-09-21T06:36:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36121.log: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-09-21T06:36:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44230.log 2024-09-21T06:36:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2024-09-21T06:36:21.688 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50274.log.gz 2024-09-21T06:36:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44230.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44230.log.gzgzip 2024-09-21T06:36:21.688 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2024-09-21T06:36:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71525.log 2024-09-21T06:36:21.690 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2024-09-21T06:36:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2024-09-21T06:36:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62930.log 2024-09-21T06:36:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-09-21T06:36:21.692 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71525.log.gz 2024-09-21T06:36:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62930.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62930.log.gz 2024-09-21T06:36:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72860.log 2024-09-21T06:36:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57783.log 2024-09-21T06:36:21.694 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-09-21T06:36:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72860.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72860.log.gz 2024-09-21T06:36:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65036.log 2024-09-21T06:36:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60922.log 2024-09-21T06:36:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65036.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57783.log.gz 2024-09-21T06:36:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53623.log 2024-09-21T06:36:21.697 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65036.log.gz 2024-09-21T06:36:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-09-21T06:36:21.698 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60922.log.gz 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58720.log 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53623.log.gz 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60433.log 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58720.log.gz 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46337.log 2024-09-21T06:36:21.703 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60433.log.gz 2024-09-21T06:36:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53716.log 2024-09-21T06:36:21.704 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2024-09-21T06:36:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.46337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46337.log.gz 2024-09-21T06:36:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56083.log 2024-09-21T06:36:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56629.log 2024-09-21T06:36:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56083.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53716.log.gz 2024-09-21T06:36:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47528.log 2024-09-21T06:36:21.707 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56083.log.gz 2024-09-21T06:36:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2024-09-21T06:36:21.708 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56629.log.gz 2024-09-21T06:36:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-09-21T06:36:21.709 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47528.log.gz 2024-09-21T06:36:21.710 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47984.log 2024-09-21T06:36:21.710 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-09-21T06:36:21.710 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz -5 2024-09-21T06:36:21.711 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47276.log 2024-09-21T06:36:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53549.log 2024-09-21T06:36:21.712 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47984.log.gz 2024-09-21T06:36:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47276.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47276.log.gz 2024-09-21T06:36:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53586.log 2024-09-21T06:36:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55263.log 2024-09-21T06:36:21.714 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53549.log.gz 2024-09-21T06:36:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47157.log 2024-09-21T06:36:21.715 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53586.log.gz 2024-09-21T06:36:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62660.log 2024-09-21T06:36:21.716 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55263.log.gz 2024-09-21T06:36:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47157.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47157.log.gz 2024-09-21T06:36:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64910.log 2024-09-21T06:36:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36339.log 2024-09-21T06:36:21.718 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62660.log.gz 2024-09-21T06:36:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64910.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64910.log.gz 2024-09-21T06:36:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56773.log 2024-09-21T06:36:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58446.log 2024-09-21T06:36:21.720 INFO:teuthology.orchestra.run.smithi037.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36339.log.gz 2024-09-21T06:36:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67248.log 2024-09-21T06:36:21.721 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56773.log.gz 2024-09-21T06:36:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59545.log 2024-09-21T06:36:21.722 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2024-09-21T06:36:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67248.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67248.log.gz 2024-09-21T06:36:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58404.log 2024-09-21T06:36:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69793.log 2024-09-21T06:36:21.724 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59545.log.gz 2024-09-21T06:36:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58404.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58404.log.gz 2024-09-21T06:36:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56167.log 2024-09-21T06:36:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69096.log 2024-09-21T06:36:21.726 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69793.log.gz 2024-09-21T06:36:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56167.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56167.log.gz 2024-09-21T06:36:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63314.log 2024-09-21T06:36:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64324.log 2024-09-21T06:36:21.728 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69096.log.gz 2024-09-21T06:36:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.63314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63314.log.gz 2024-09-21T06:36:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61811.log 2024-09-21T06:36:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62105.log 2024-09-21T06:36:21.731 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64324.log.gz 2024-09-21T06:36:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61811.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61811.log.gz 2024-09-21T06:36:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73028.log 2024-09-21T06:36:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72692.log 2024-09-21T06:36:21.732 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62105.log.gz 2024-09-21T06:36:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.73028.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73028.log.gz 2024-09-21T06:36:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-09-21T06:36:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72313.log 2024-09-21T06:36:21.735 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72692.log.gz 2024-09-21T06:36:21.735 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44566.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-09-21T06:36:21.735 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72271.log 2024-09-21T06:36:21.736 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2024-09-21T06:36:21.737 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72313.log.gz 2024-09-21T06:36:21.737 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72271.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72271.log.gz 2024-09-21T06:36:21.737 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54973.log 2024-09-21T06:36:21.738 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2024-09-21T06:36:21.739 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gz 2024-09-21T06:36:21.739 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54973.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54973.log.gz 2024-09-21T06:36:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-09-21T06:36:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72944.log 2024-09-21T06:36:21.741 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2024-09-21T06:36:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz --verbose 2024-09-21T06:36:21.742 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.67023.log 2024-09-21T06:36:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60367.log 2024-09-21T06:36:21.743 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72944.log.gz 2024-09-21T06:36:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67023.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67023.log.gz 2024-09-21T06:36:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62846.log 2024-09-21T06:36:21.745 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72440.log 2024-09-21T06:36:21.745 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62846.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60367.log.gz 2024-09-21T06:36:21.746 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62846.log.gz 2024-09-21T06:36:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68380.log 2024-09-21T06:36:21.747 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67290.log 2024-09-21T06:36:21.747 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72440.log.gz 2024-09-21T06:36:21.748 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62744.log 2024-09-21T06:36:21.748 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67290.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68380.log.gz 2024-09-21T06:36:21.749 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67290.log.gz 2024-09-21T06:36:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35988.log 2024-09-21T06:36:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48110.log 2024-09-21T06:36:21.750 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62744.log.gz 2024-09-21T06:36:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55644.log 2024-09-21T06:36:21.751 INFO:teuthology.orchestra.run.smithi037.stderr: 74.7% -- replaced with /var/log/ceph/ceph-client.admin.35988.log.gz 2024-09-21T06:36:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48110.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48110.log.gz 2024-09-21T06:36:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66352.log 2024-09-21T06:36:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59755.log 2024-09-21T06:36:21.753 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55644.log.gz 2024-09-21T06:36:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66352.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66352.log.gz 2024-09-21T06:36:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-09-21T06:36:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66497.log 2024-09-21T06:36:21.755 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59755.log.gz 2024-09-21T06:36:21.756 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48450.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-09-21T06:36:21.756 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63662.log 2024-09-21T06:36:21.757 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65571.log 2024-09-21T06:36:21.757 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66497.log.gz 2024-09-21T06:36:21.758 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59713.log 2024-09-21T06:36:21.758 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63662.log.gz 2024-09-21T06:36:21.759 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-21T06:36:21.760 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65571.log.gz 2024-09-21T06:36:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59713.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59713.log.gz 2024-09-21T06:36:21.760 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50153.log 2024-09-21T06:36:21.761 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-09-21T06:36:21.761 INFO:teuthology.orchestra.run.smithi037.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-21T06:36:21.762 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59839.log 2024-09-21T06:36:21.763 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50153.log.gz 2024-09-21T06:36:21.763 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-21T06:36:21.764 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-09-21T06:36:21.764 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59839.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59839.log.gz 2024-09-21T06:36:21.764 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63056.log 2024-09-21T06:36:21.765 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71483.log 2024-09-21T06:36:21.766 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-21T06:36:21.766 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36031.log 2024-09-21T06:36:21.767 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63056.log.gz 2024-09-21T06:36:21.767 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51796.log 2024-09-21T06:36:21.768 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71483.log.gz 2024-09-21T06:36:21.768 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36031.log: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.36031.log.gz 2024-09-21T06:36:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53512.log 2024-09-21T06:36:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-09-21T06:36:21.770 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51796.log.gz 2024-09-21T06:36:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42703.log 2024-09-21T06:36:21.771 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53512.log.gz 2024-09-21T06:36:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2024-09-21T06:36:21.772 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-09-21T06:36:21.773 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60565.log 2024-09-21T06:36:21.773 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42703.log.gz 2024-09-21T06:36:21.774 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58320.log 2024-09-21T06:36:21.774 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2024-09-21T06:36:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60565.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60565.log.gz 2024-09-21T06:36:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37132.log 2024-09-21T06:36:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63530.log 2024-09-21T06:36:21.776 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58320.log.gz 2024-09-21T06:36:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37132.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37132.log.gz 2024-09-21T06:36:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62403.log 2024-09-21T06:36:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-09-21T06:36:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62403.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63530.log.gz 2024-09-21T06:36:21.779 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62403.log.gz 2024-09-21T06:36:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57587.log 2024-09-21T06:36:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35969.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-09-21T06:36:21.780 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36521.log 2024-09-21T06:36:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59165.log 2024-09-21T06:36:21.781 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57587.log.gz 2024-09-21T06:36:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36521.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36521.log.gz 2024-09-21T06:36:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66226.log 2024-09-21T06:36:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65319.log 2024-09-21T06:36:21.783 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59165.log.gz 2024-09-21T06:36:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66226.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66226.log.gz 2024-09-21T06:36:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36931.log 2024-09-21T06:36:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58636.log 2024-09-21T06:36:21.785 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65319.log.gz 2024-09-21T06:36:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36931.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36931.log.gz 2024-09-21T06:36:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-09-21T06:36:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56551.log 2024-09-21T06:36:21.787 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58636.log.gz 2024-09-21T06:36:21.788 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47654.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-09-21T06:36:21.788 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2024-09-21T06:36:21.789 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.56551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56551.log.gz 2024-09-21T06:36:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-09-21T06:36:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43879.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T06:36:21.791 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2024-09-21T06:36:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66707.log 2024-09-21T06:36:21.792 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-09-21T06:36:21.792 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69919.log 2024-09-21T06:36:21.794 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66707.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66707.log.gz 2024-09-21T06:36:21.794 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41931.log 2024-09-21T06:36:21.795 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62063.log 2024-09-21T06:36:21.795 INFO:teuthology.orchestra.run.smithi037.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T06:36:21.795 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69919.log.gz 2024-09-21T06:36:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41931.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41931.log.gz 2024-09-21T06:36:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69751.log 2024-09-21T06:36:21.797 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50630.log 2024-09-21T06:36:21.797 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62063.log.gz 2024-09-21T06:36:21.798 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-09-21T06:36:21.798 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69751.log.gz 2024-09-21T06:36:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55053.log 2024-09-21T06:36:21.799 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50630.log.gz 2024-09-21T06:36:21.800 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45350.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz --verbose 2024-09-21T06:36:21.800 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.61689.log 2024-09-21T06:36:21.801 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68272.log 2024-09-21T06:36:21.801 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55053.log.gz 2024-09-21T06:36:21.802 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72986.log 2024-09-21T06:36:21.802 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61689.log.gz 2024-09-21T06:36:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68272.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68272.log.gz 2024-09-21T06:36:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70747.log 2024-09-21T06:36:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55999.log 2024-09-21T06:36:21.804 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72986.log.gz 2024-09-21T06:36:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65742.log 2024-09-21T06:36:21.805 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70747.log.gz 2024-09-21T06:36:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55999.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55999.log.gz 2024-09-21T06:36:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51701.log 2024-09-21T06:36:21.807 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66413.log 2024-09-21T06:36:21.807 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65742.log.gz 2024-09-21T06:36:21.807 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67714.log 2024-09-21T06:36:21.808 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51701.log.gz 2024-09-21T06:36:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66413.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66413.log.gzgzip 2024-09-21T06:36:21.808 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65613.log 2024-09-21T06:36:21.809 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T06:36:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59671.log 86.7% 2024-09-21T06:36:21.810 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67714.log.gz 2024-09-21T06:36:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65613.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65613.log.gz 2024-09-21T06:36:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67420.log 2024-09-21T06:36:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65169.log 2024-09-21T06:36:21.811 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59671.log.gz 2024-09-21T06:36:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67420.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67420.log.gz 2024-09-21T06:36:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55095.log 2024-09-21T06:36:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36214.log 2024-09-21T06:36:21.813 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65169.log.gz 2024-09-21T06:36:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55095.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55095.log.gz 2024-09-21T06:36:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69961.log 2024-09-21T06:36:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67546.log 2024-09-21T06:36:21.815 INFO:teuthology.orchestra.run.smithi037.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36214.log.gz 2024-09-21T06:36:21.816 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69961.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69961.log.gz 2024-09-21T06:36:21.816 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44048.log 2024-09-21T06:36:21.817 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72177.log 2024-09-21T06:36:21.817 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67546.log.gz 2024-09-21T06:36:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44048.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44048.log.gz 2024-09-21T06:36:21.818 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55774.log 2024-09-21T06:36:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51491.log 2024-09-21T06:36:21.819 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72177.log.gz 2024-09-21T06:36:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55774.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55774.log.gz 2024-09-21T06:36:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57178.log 2024-09-21T06:36:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59587.log 2024-09-21T06:36:21.820 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51491.log.gz 2024-09-21T06:36:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60139.log 2024-09-21T06:36:21.822 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57178.log.gz 2024-09-21T06:36:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59587.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59587.log.gz 2024-09-21T06:36:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70201.log 2024-09-21T06:36:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.60139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60139.log.gz 2024-09-21T06:36:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-09-21T06:36:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58488.log 2024-09-21T06:36:21.824 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70201.log.gz 2024-09-21T06:36:21.825 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz -- 2024-09-21T06:36:21.825 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.56671.log 2024-09-21T06:36:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68314.log 2024-09-21T06:36:21.826 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58488.log.gz 2024-09-21T06:36:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56671.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56671.log.gz 2024-09-21T06:36:21.827 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50237.log 2024-09-21T06:36:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-09-21T06:36:21.828 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68314.log.gz 2024-09-21T06:36:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-09-21T06:36:21.829 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50237.log.gz 2024-09-21T06:36:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48618.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz 2024-09-21T06:36:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68094.log 2024-09-21T06:36:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46596.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-09-21T06:36:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56905.log 2024-09-21T06:36:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.68094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68094.log.gz 2024-09-21T06:36:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-09-21T06:36:21.833 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66539.log 2024-09-21T06:36:21.833 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56905.log.gz 2024-09-21T06:36:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51274.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-09-21T06:36:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69601.log 2024-09-21T06:36:21.835 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66539.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66539.log.gz 2024-09-21T06:36:21.835 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2024-09-21T06:36:21.835 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65403.log 2024-09-21T06:36:21.836 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69601.log.gz 2024-09-21T06:36:21.837 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T06:36:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55137.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2024-09-21T06:36:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-09-21T06:36:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65403.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65403.log.gz 2024-09-21T06:36:21.838 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69403.log 2024-09-21T06:36:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42193.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gzgzip 2024-09-21T06:36:21.838 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56125.log 2024-09-21T06:36:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54430.log 2024-09-21T06:36:21.840 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69403.log.gz 2024-09-21T06:36:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36078.log 2024-09-21T06:36:21.840 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56125.log.gz 2024-09-21T06:36:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54430.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gz 2024-09-21T06:36:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53309.log 2024-09-21T06:36:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36078.log: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.36078.log.gz 2024-09-21T06:36:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58864.log 2024-09-21T06:36:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58948.log 2024-09-21T06:36:21.843 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53309.log.gz 2024-09-21T06:36:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65826.log 2024-09-21T06:36:21.844 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58864.log.gz 2024-09-21T06:36:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58948.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58948.log.gz 2024-09-21T06:36:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log 2024-09-21T06:36:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42535.log 2024-09-21T06:36:21.846 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65826.log.gz 2024-09-21T06:36:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51106.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gz 2024-09-21T06:36:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43200.log 2024-09-21T06:36:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64718.log 2024-09-21T06:36:21.848 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42535.log.gz 2024-09-21T06:36:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60790.log 2024-09-21T06:36:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-09-21T06:36:21.850 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64718.log.gz 2024-09-21T06:36:21.850 INFO:teuthology.orchestra.run.smithi037.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.43200.log.gz 2024-09-21T06:36:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-09-21T06:36:21.851 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60790.log.gz 2024-09-21T06:36:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69877.log 2024-09-21T06:36:21.852 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-09-21T06:36:21.853 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43682.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-09-21T06:36:21.853 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.44897.log 2024-09-21T06:36:21.854 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-09-21T06:36:21.854 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69877.log.gz 2024-09-21T06:36:21.855 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44897.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-09-21T06:36:21.855 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70267.log 2024-09-21T06:36:21.856 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60268.log 2024-09-21T06:36:21.856 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-09-21T06:36:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66184.log 2024-09-21T06:36:21.857 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70267.log.gz 2024-09-21T06:36:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60268.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60268.log.gz 2024-09-21T06:36:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66268.log 2024-09-21T06:36:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51838.log 2024-09-21T06:36:21.859 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66184.log.gz 2024-09-21T06:36:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62147.log 2024-09-21T06:36:21.860 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66268.log.gz 2024-09-21T06:36:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51838.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51838.log.gz 2024-09-21T06:36:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-09-21T06:36:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-09-21T06:36:21.862 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62147.log.gz 2024-09-21T06:36:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61895.log 2024-09-21T06:36:21.863 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-09-21T06:36:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-09-21T06:36:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35951.log 2024-09-21T06:36:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61095.log 2024-09-21T06:36:21.864 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61895.log.gz 2024-09-21T06:36:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-09-21T06:36:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71609.log 2024-09-21T06:36:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-21T06:36:21.867 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61095.log.gz 2024-09-21T06:36:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71609.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71609.log.gz 2024-09-21T06:36:21.867 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36580.log 2024-09-21T06:36:21.868 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67798.log 2024-09-21T06:36:21.868 INFO:teuthology.orchestra.run.smithi037.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-21T06:36:21.869 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36580.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36580.log.gz 2024-09-21T06:36:21.869 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-09-21T06:36:21.870 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61225.log 2024-09-21T06:36:21.870 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67798.log.gz 2024-09-21T06:36:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44880.log 2024-09-21T06:36:21.871 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-09-21T06:36:21.872 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59461.log 2024-09-21T06:36:21.873 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61225.log.gz 2024-09-21T06:36:21.873 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44880.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44880.log.gz 2024-09-21T06:36:21.873 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42577.log 2024-09-21T06:36:21.874 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59461.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59461.log.gz 2024-09-21T06:36:21.874 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2024-09-21T06:36:21.875 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68176.log 2024-09-21T06:36:21.875 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42577.log.gz 2024-09-21T06:36:21.876 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45140.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2024-09-21T06:36:21.876 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43922.log 2024-09-21T06:36:21.877 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47942.log 2024-09-21T06:36:21.877 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68176.log.gz 2024-09-21T06:36:21.878 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43922.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43922.log.gz 2024-09-21T06:36:21.878 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73110.log 2024-09-21T06:36:21.879 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47942.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47942.log.gz 2024-09-21T06:36:21.879 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65445.log 2024-09-21T06:36:21.880 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.73110.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73110.log.gz 2024-09-21T06:36:21.880 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47234.log 2024-09-21T06:36:21.881 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65445.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T06:36:21.881 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65445.log.gz 2024-09-21T06:36:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47234.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47234.log.gz 2024-09-21T06:36:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53243.log 2024-09-21T06:36:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-09-21T06:36:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42281.log 2024-09-21T06:36:21.884 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53243.log.gz 2024-09-21T06:36:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-09-21T06:36:21.885 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2024-09-21T06:36:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72818.log 2024-09-21T06:36:21.887 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42281.log.gz 2024-09-21T06:36:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52607.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz /var/log/ceph/ceph-client.admin.60856.log 2024-09-21T06:36:21.887 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72818.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72818.log.gz 2024-09-21T06:36:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67504.log 2024-09-21T06:36:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60217.log 2024-09-21T06:36:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67504.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60856.log.gz 2024-09-21T06:36:21.891 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67504.log.gz 2024-09-21T06:36:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58184.log 2024-09-21T06:36:21.892 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53660.log 2024-09-21T06:36:21.892 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60217.log.gz 2024-09-21T06:36:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58184.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58184.log.gz 2024-09-21T06:36:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57867.log 2024-09-21T06:36:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45098.log 2024-09-21T06:36:21.895 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53660.log.gz 2024-09-21T06:36:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2024-09-21T06:36:21.896 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57867.log.gz 2024-09-21T06:36:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55179.log 2024-09-21T06:36:21.897 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45098.log.gz 2024-09-21T06:36:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36467.log 2024-09-21T06:36:21.898 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2024-09-21T06:36:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55179.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55179.log.gz 2024-09-21T06:36:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71735.log 2024-09-21T06:36:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36467.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36467.log.gz 2024-09-21T06:36:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36431.log 2024-09-21T06:36:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50494.log 2024-09-21T06:36:21.902 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71735.log.gz 2024-09-21T06:36:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36431.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36431.log.gz 2024-09-21T06:36:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-09-21T06:36:21.903 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50494.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50494.log.gz 2024-09-21T06:36:21.903 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69244.log 2024-09-21T06:36:21.904 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68098.log 2024-09-21T06:36:21.905 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-09-21T06:36:21.905 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T06:36:21.906 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66141.log 2024-09-21T06:36:21.906 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69244.log.gz 2024-09-21T06:36:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.68098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68098.log.gz 2024-09-21T06:36:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71777.log 2024-09-21T06:36:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49002.log 2024-09-21T06:36:21.908 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66141.log.gz 2024-09-21T06:36:21.909 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71777.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71777.log.gz 2024-09-21T06:36:21.909 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62702.log 2024-09-21T06:36:21.910 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.49002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70069.log 2024-09-21T06:36:21.910 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49002.log.gz 2024-09-21T06:36:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57310.log 2024-09-21T06:36:21.911 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62702.log.gz 2024-09-21T06:36:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54046.log 2024-09-21T06:36:21.913 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70069.log.gz 2024-09-21T06:36:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68797.log 2024-09-21T06:36:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54046.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57310.log.gz 2024-09-21T06:36:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-09-21T06:36:21.915 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54046.log.gz 2024-09-21T06:36:21.916 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60135.log 2024-09-21T06:36:21.916 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68797.log.gz 2024-09-21T06:36:21.917 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70898.log 2024-09-21T06:36:21.917 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-09-21T06:36:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.60135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42151.log 2024-09-21T06:36:21.918 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60135.log.gz 2024-09-21T06:36:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70898.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.70898.log.gz 2024-09-21T06:36:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61521.log 2024-09-21T06:36:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62445.log 2024-09-21T06:36:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42151.log: /var/log/ceph/ceph-client.admin.61521.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42151.log.gz 2024-09-21T06:36:21.921 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61521.log.gz 2024-09-21T06:36:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63981.log 2024-09-21T06:36:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61605.log 2024-09-21T06:36:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62445.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62445.log.gz 2024-09-21T06:36:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44561.log 2024-09-21T06:36:21.923 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63981.log.gz 2024-09-21T06:36:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45896.log 2024-09-21T06:36:21.925 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61605.log.gz 2024-09-21T06:36:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44561.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44561.log.gz 2024-09-21T06:36:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T06:36:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.45896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64287.log 2024-09-21T06:36:21.927 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45896.log.gz 2024-09-21T06:36:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-09-21T06:36:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57624.log 2024-09-21T06:36:21.929 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64287.log.gz 2024-09-21T06:36:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62532.log 2024-09-21T06:36:21.930 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-09-21T06:36:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-21T06:36:21.932 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57624.log.gz 2024-09-21T06:36:21.932 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71126.log 2024-09-21T06:36:21.933 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62532.log.gz 2024-09-21T06:36:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47776.log 2024-09-21T06:36:21.934 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-21T06:36:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58278.log 2024-09-21T06:36:21.935 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71126.log.gz 2024-09-21T06:36:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55686.log 2024-09-21T06:36:21.936 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47776.log.gz 2024-09-21T06:36:21.938 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2024-09-21T06:36:21.938 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58278.log.gz 2024-09-21T06:36:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-09-21T06:36:21.939 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55686.log.gz 2024-09-21T06:36:21.940 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72398.log 2024-09-21T06:36:21.940 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2024-09-21T06:36:21.941 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70813.log 2024-09-21T06:36:21.941 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-09-21T06:36:21.942 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55730.log 2024-09-21T06:36:21.942 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72398.log.gz 2024-09-21T06:36:21.943 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66665.log 2024-09-21T06:36:21.944 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70813.log.gz 2024-09-21T06:36:21.944 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-09-21T06:36:21.945 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55730.log.gz 2024-09-21T06:36:21.946 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71984.log 2024-09-21T06:36:21.946 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66665.log.gz 2024-09-21T06:36:21.947 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2024-09-21T06:36:21.947 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-09-21T06:36:21.948 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58065.log 2024-09-21T06:36:21.948 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71984.log.gz 2024-09-21T06:36:21.949 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62242.log 2024-09-21T06:36:21.949 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2024-09-21T06:36:21.951 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56209.log 2024-09-21T06:36:21.951 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58065.log.gz 2024-09-21T06:36:21.952 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58362.log 2024-09-21T06:36:21.952 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62242.log.gz 2024-09-21T06:36:21.953 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59085.log 2024-09-21T06:36:21.953 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56209.log.gz 2024-09-21T06:36:21.954 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59503.log 2024-09-21T06:36:21.954 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58362.log.gz 2024-09-21T06:36:21.955 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-09-21T06:36:21.955 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59085.log.gz 2024-09-21T06:36:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57661.log 2024-09-21T06:36:21.957 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59503.log.gz 2024-09-21T06:36:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-09-21T06:36:21.958 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-09-21T06:36:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-09-21T06:36:21.959 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57661.log.gz 2024-09-21T06:36:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58906.log 2024-09-21T06:36:21.960 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-09-21T06:36:21.962 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72608.log 2024-09-21T06:36:21.962 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-09-21T06:36:21.962 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70333.log 2024-09-21T06:36:21.963 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58906.log.gz 2024-09-21T06:36:21.964 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49685.log 2024-09-21T06:36:21.964 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72608.log.gz 2024-09-21T06:36:21.965 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44545.log 2024-09-21T06:36:21.966 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70333.log.gz 2024-09-21T06:36:21.967 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-21T06:36:21.967 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49685.log.gz 2024-09-21T06:36:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44225.log 2024-09-21T06:36:21.968 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44545.log.gz 2024-09-21T06:36:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63122.log 2024-09-21T06:36:21.969 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-21T06:36:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56971.log 2024-09-21T06:36:21.970 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44225.log.gz 2024-09-21T06:36:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-09-21T06:36:21.972 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63122.log.gz 2024-09-21T06:36:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61563.log 2024-09-21T06:36:21.973 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56971.log.gz 2024-09-21T06:36:21.974 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44555.log 2024-09-21T06:36:21.974 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-09-21T06:36:21.975 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61563.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T06:36:21.975 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61563.log.gz 2024-09-21T06:36:21.976 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48068.log 87.4% 2024-09-21T06:36:21.977 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44555.log.gz 2024-09-21T06:36:21.977 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62972.log 2024-09-21T06:36:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51968.log 2024-09-21T06:36:21.979 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48068.log.gz 2024-09-21T06:36:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62972.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62972.log.gz 2024-09-21T06:36:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72222.log 2024-09-21T06:36:21.982 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51968.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51968.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44215.log 2024-09-21T06:36:21.982 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:21.983 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48026.log 2024-09-21T06:36:21.983 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72222.log.gz 2024-09-21T06:36:21.984 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-09-21T06:36:21.985 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44215.log.gz 2024-09-21T06:36:21.985 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T06:36:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57717.log 2024-09-21T06:36:21.986 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48026.log.gz 2024-09-21T06:36:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58822.log 2024-09-21T06:36:21.987 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-09-21T06:36:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57717.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57717.log.gz 2024-09-21T06:36:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67462.log 2024-09-21T06:36:21.990 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58822.log.gz 2024-09-21T06:36:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2024-09-21T06:36:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55305.log 2024-09-21T06:36:21.992 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67462.log.gz 2024-09-21T06:36:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51657.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2024-09-21T06:36:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63310.log 2024-09-21T06:36:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55305.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55305.log.gz 2024-09-21T06:36:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61979.log 2024-09-21T06:36:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.63310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63310.log.gz 2024-09-21T06:36:21.996 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2024-09-21T06:36:21.997 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-09-21T06:36:21.997 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61979.log.gz 2024-09-21T06:36:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69281.log 2024-09-21T06:36:21.999 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2024-09-21T06:36:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71389.log 2024-09-21T06:36:22.000 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-09-21T06:36:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69281.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T06:36:22.001 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69281.log.gz 2024-09-21T06:36:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71389.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71389.log.gz 2024-09-21T06:36:22.003 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71168.log 2024-09-21T06:36:22.004 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2024-09-21T06:36:22.005 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71168.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71168.log.gz 2024-09-21T06:36:22.006 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.52892.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70399.log 2024-09-21T06:36:22.007 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2024-09-21T06:36:22.007 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49140.log 2024-09-21T06:36:22.009 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70399.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70399.log.gz 2024-09-21T06:36:22.010 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-09-21T06:36:22.010 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-09-21T06:36:22.012 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-09-21T06:36:22.012 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49392.log: 94.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T06:36:22.013 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49458.log 2024-09-21T06:36:22.013 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-09-21T06:36:22.014 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48702.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-09-21T06:36:22.015 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64250.log 2024-09-21T06:36:22.016 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2024-09-21T06:36:22.016 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49458.log.gz 2024-09-21T06:36:22.017 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56547.log 2024-09-21T06:36:22.018 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64250.log.gz 2024-09-21T06:36:22.018 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52178.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz 2024-09-21T06:36:22.019 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72776.log 2024-09-21T06:36:22.020 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.56547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56547.log.gz 2024-09-21T06:36:22.020 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60988.log 2024-09-21T06:36:22.021 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64361.log 2024-09-21T06:36:22.021 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72776.log.gz 2024-09-21T06:36:22.023 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63728.log 2024-09-21T06:36:22.023 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60988.log.gz 2024-09-21T06:36:22.024 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64361.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64361.log.gz 2024-09-21T06:36:22.024 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59331.log 2024-09-21T06:36:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63728.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63728.log.gz 2024-09-21T06:36:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61343.log 2024-09-21T06:36:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65656.log 2024-09-21T06:36:22.027 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59331.log.gz 2024-09-21T06:36:22.028 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42619.log 2024-09-21T06:36:22.028 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61343.log.gz 2024-09-21T06:36:22.029 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65656.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T06:36:22.029 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65656.log.gz 2024-09-21T06:36:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71941.log 2024-09-21T06:36:22.031 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42619.log.gz 2024-09-21T06:36:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64047.log 2024-09-21T06:36:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71941.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71941.log.gz 2024-09-21T06:36:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69207.log 2024-09-21T06:36:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64047.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T06:36:22.035 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64047.log.gz 2024-09-21T06:36:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42409.log 2024-09-21T06:36:22.037 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69207.log: gzip -5 89.3% --verbose -- /var/log/ceph/ceph-client.admin.61647.log 2024-09-21T06:36:22.037 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69207.log.gz 2024-09-21T06:36:22.038 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71252.log 2024-09-21T06:36:22.038 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42409.log.gz 2024-09-21T06:36:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62618.log 2024-09-21T06:36:22.039 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61647.log.gz 2024-09-21T06:36:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64868.log 2024-09-21T06:36:22.041 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71252.log.gz 2024-09-21T06:36:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45877.log 2024-09-21T06:36:22.042 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62618.log.gz 2024-09-21T06:36:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62488.log 2024-09-21T06:36:22.043 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64868.log.gz 2024-09-21T06:36:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.45877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45877.log.gz 2024-09-21T06:36:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-09-21T06:36:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-09-21T06:36:22.046 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62488.log.gz 2024-09-21T06:36:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70507.log 2024-09-21T06:36:22.048 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz/var/log/ceph/ceph-client.admin.56359.log: 2024-09-21T06:36:22.048 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-09-21T06:36:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67109.log 2024-09-21T06:36:22.050 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52760.log 2024-09-21T06:36:22.050 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70507.log.gz 2024-09-21T06:36:22.051 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66925.log 2024-09-21T06:36:22.051 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67109.log.gz 2024-09-21T06:36:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52760.log: /var/log/ceph/ceph-client.admin.66925.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52760.log.gz 2024-09-21T06:36:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66925.log.gz 2024-09-21T06:36:22.054 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2024-09-21T06:36:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65235.log 2024-09-21T06:36:22.056 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45266.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-09-21T06:36:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50002.log 2024-09-21T06:36:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65103.log 2024-09-21T06:36:22.058 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65235.log.gz 2024-09-21T06:36:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59375.log 2024-09-21T06:36:22.060 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50002.log.gz 2024-09-21T06:36:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65103.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65103.log.gz 2024-09-21T06:36:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55564.log 2024-09-21T06:36:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59375.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59375.log.gz 2024-09-21T06:36:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53782.log 2024-09-21T06:36:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66455.log 2024-09-21T06:36:22.064 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55564.log.gz 2024-09-21T06:36:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53782.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53782.log.gz 2024-09-21T06:36:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59797.log 2024-09-21T06:36:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66455.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66455.log.gz 2024-09-21T06:36:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56839.log 2024-09-21T06:36:22.068 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-09-21T06:36:22.068 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59797.log.gz 2024-09-21T06:36:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64520.log 2024-09-21T06:36:22.070 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56839.log.gz 2024-09-21T06:36:22.071 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50840.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-09-21T06:36:22.072 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62021.log 2024-09-21T06:36:22.073 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64520.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64520.log.gz 2024-09-21T06:36:22.073 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67756.log 2024-09-21T06:36:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43478.log 2024-09-21T06:36:22.075 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62021.log.gz 2024-09-21T06:36:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67756.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67756.log.gz 2024-09-21T06:36:22.076 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.65487.log 2024-09-21T06:36:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43478.log.gz 2024-09-21T06:36:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68929.log 2024-09-21T06:36:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65784.log 2024-09-21T06:36:22.079 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65487.log.gz 2024-09-21T06:36:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68929.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68929.log.gz/var/log/ceph/ceph-client.admin.65784.log: 2024-09-21T06:36:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65529.log 2024-09-21T06:36:22.081 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65784.log.gz 2024-09-21T06:36:22.082 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54805.log 2024-09-21T06:36:22.084 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65277.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65529.log.gz 2024-09-21T06:36:22.084 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54805.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54805.log.gz 2024-09-21T06:36:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2024-09-21T06:36:22.087 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-09-21T06:36:22.087 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65277.log.gz 2024-09-21T06:36:22.088 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56713.log 2024-09-21T06:36:22.088 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2024-09-21T06:36:22.089 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73068.log 2024-09-21T06:36:22.089 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-09-21T06:36:22.090 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56713.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56713.log.gz 2024-09-21T06:36:22.091 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63443.log 2024-09-21T06:36:22.091 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.73068.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.73068.log.gz 2024-09-21T06:36:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45392.log 2024-09-21T06:36:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72482.log 2024-09-21T06:36:22.093 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63443.log.gz 2024-09-21T06:36:22.095 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45392.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45392.log.gz 2024-09-21T06:36:22.095 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-09-21T06:36:22.096 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72482.log.gz 2024-09-21T06:36:22.096 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36246.log 2024-09-21T06:36:22.098 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64398.log 2024-09-21T06:36:22.098 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-09-21T06:36:22.099 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61437.log 2024-09-21T06:36:22.099 INFO:teuthology.orchestra.run.smithi037.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36246.log.gz 2024-09-21T06:36:22.100 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60499.log 2024-09-21T06:36:22.100 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64398.log.gz 2024-09-21T06:36:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61437.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61437.log.gz 2024-09-21T06:36:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54585.log 2024-09-21T06:36:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56293.log 2024-09-21T06:36:22.103 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60499.log.gz 2024-09-21T06:36:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54585.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54585.log.gz 2024-09-21T06:36:22.105 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63488.log 2024-09-21T06:36:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70549.log 2024-09-21T06:36:22.106 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9%/var/log/ceph/ceph-client.admin.63488.log: -- replaced with /var/log/ceph/ceph-client.admin.56293.log.gz 2024-09-21T06:36:22.107 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63488.log.gz 2024-09-21T06:36:22.107 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72524.log 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47486.log 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi150.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70549.log.gz 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72524.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72524.log.gz 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.69133.log 2024-09-21T06:36:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47486.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47486.log.gz 2024-09-21T06:36:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59246.log 2024-09-21T06:36:22.114 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-09-21T06:36:22.114 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69133.log.gz 2024-09-21T06:36:22.114 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44006.log 2024-09-21T06:36:22.114 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59246.log.gz 2024-09-21T06:36:22.115 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43420.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-09-21T06:36:22.116 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-09-21T06:36:22.117 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44006.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44006.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36615.log 2024-09-21T06:36:22.117 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:22.118 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56251.log 2024-09-21T06:36:22.118 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-09-21T06:36:22.120 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45308.log 2024-09-21T06:36:22.120 INFO:teuthology.orchestra.run.smithi037.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36615.log.gz 2024-09-21T06:36:22.121 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47402.log 2024-09-21T06:36:22.121 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56251.log.gz 2024-09-21T06:36:22.122 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52136.log 2024-09-21T06:36:22.122 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45308.log.gz 2024-09-21T06:36:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65868.log 2024-09-21T06:36:22.124 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47402.log.gz 2024-09-21T06:36:22.125 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52136.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52136.log.gz 2024-09-21T06:36:22.125 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.51613.log 2024-09-21T06:36:22.126 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65868.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65868.log.gz 2024-09-21T06:36:22.126 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52286.log 2024-09-21T06:36:22.128 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-21T06:36:22.128 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-09-21T06:36:22.129 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-09-21T06:36:22.129 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52286.log.gz 2024-09-21T06:36:22.130 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48366.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-21T06:36:22.131 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62786.log 2024-09-21T06:36:22.131 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43742.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-09-21T06:36:22.132 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-09-21T06:36:22.133 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67065.log 2024-09-21T06:36:22.133 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62786.log.gz 2024-09-21T06:36:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52220.log: /var/log/ceph/ceph-client.admin.67065.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-09-21T06:36:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67065.log.gz 2024-09-21T06:36:22.136 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.0.52474.log 2024-09-21T06:36:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45182.log 2024-09-21T06:36:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.52474.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55957.log 2024-09-21T06:36:22.139 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.0.52474.log.gz 2024-09-21T06:36:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64826.log 2024-09-21T06:36:22.140 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45182.log.gz 2024-09-21T06:36:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55957.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55957.log.gz 2024-09-21T06:36:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53980.log 2024-09-21T06:36:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64826.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.64826.log.gz /var/log/ceph/ceph-client.admin.50962.log 2024-09-21T06:36:22.143 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44894.log 2024-09-21T06:36:22.144 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53980.log.gz 2024-09-21T06:36:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50962.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2024-09-21T06:36:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-09-21T06:36:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44894.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44894.log.gz 2024-09-21T06:36:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47444.log 2024-09-21T06:36:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42661.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-09-21T06:36:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42237.log 2024-09-21T06:36:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44544.log 2024-09-21T06:36:22.149 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47444.log.gz 2024-09-21T06:36:22.151 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47360.log 2024-09-21T06:36:22.151 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42237.log.gz 2024-09-21T06:36:22.152 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52694.log 2024-09-21T06:36:22.152 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2024-09-21T06:36:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47360.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47360.log.gz 2024-09-21T06:36:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69170.log 2024-09-21T06:36:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59881.log 2024-09-21T06:36:22.155 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52694.log.gz 2024-09-21T06:36:22.156 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69170.log: gzip -5 --verbose -- 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69170.log.gz 2024-09-21T06:36:22.156 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.72356.log 2024-09-21T06:36:22.158 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61853.log 2024-09-21T06:36:22.158 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59881.log.gz 2024-09-21T06:36:22.159 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71294.log 2024-09-21T06:36:22.159 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72356.log.gz 2024-09-21T06:36:22.160 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61853.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61853.log.gz 2024-09-21T06:36:22.161 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68512.log 2024-09-21T06:36:22.161 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71294.log.gz 2024-09-21T06:36:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42769.log 2024-09-21T06:36:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.68512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-09-21T06:36:22.163 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68512.log.gz 2024-09-21T06:36:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42769.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz -5 2024-09-21T06:36:22.165 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67630.log 2024-09-21T06:36:22.165 INFO:teuthology.orchestra.run.smithi037.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T06:36:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57513.log 2024-09-21T06:36:22.166 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-09-21T06:36:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-09-21T06:36:22.167 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67630.log.gz 2024-09-21T06:36:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53177.log 2024-09-21T06:36:22.168 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57513.log.gz 2024-09-21T06:36:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52094.log 2024-09-21T06:36:22.169 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-09-21T06:36:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67153.log 2024-09-21T06:36:22.171 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53177.log.gz 2024-09-21T06:36:22.171 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67197.log 2024-09-21T06:36:22.171 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52094.log.gz 2024-09-21T06:36:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67153.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.67153.log.gz 2024-09-21T06:36:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71567.log 2024-09-21T06:36:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-09-21T06:36:22.174 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67197.log.gz 2024-09-21T06:36:22.175 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.71567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54931.log 2024-09-21T06:36:22.175 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71567.log.gz 2024-09-21T06:36:22.176 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52556.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-09-21T06:36:22.176 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47857.log 2024-09-21T06:36:22.177 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51411.log 2024-09-21T06:36:22.177 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54931.log.gz 2024-09-21T06:36:22.178 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47857.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47857.log.gz 2024-09-21T06:36:22.178 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64213.log 2024-09-21T06:36:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64952.log 2024-09-21T06:36:22.180 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51411.log.gz 2024-09-21T06:36:22.180 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63392.log 2024-09-21T06:36:22.180 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64213.log.gz 2024-09-21T06:36:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72902.log 2024-09-21T06:36:22.182 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64952.log.gz 2024-09-21T06:36:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.63392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70855.log 2024-09-21T06:36:22.183 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63392.log.gz 2024-09-21T06:36:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72902.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72902.log.gz 2024-09-21T06:36:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-09-21T06:36:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55484.log 2024-09-21T06:36:22.185 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70855.log.gz 2024-09-21T06:36:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64652.log 2024-09-21T06:36:22.186 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-09-21T06:36:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53475.log 2024-09-21T06:36:22.187 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55484.log.gz 2024-09-21T06:36:22.188 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.64652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-09-21T06:36:22.189 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64652.log.gz 2024-09-21T06:36:22.190 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2024-09-21T06:36:22.190 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53475.log.gz 2024-09-21T06:36:22.190 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-09-21T06:36:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52826.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-09-21T06:36:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69469.log 2024-09-21T06:36:22.192 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2024-09-21T06:36:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49751.log 2024-09-21T06:36:22.193 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-09-21T06:36:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54388.log 2024-09-21T06:36:22.194 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69469.log.gz 2024-09-21T06:36:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61479.log 2024-09-21T06:36:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54388.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49751.log.gz 2024-09-21T06:36:22.196 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54388.log.gz 2024-09-21T06:36:22.197 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51884.log 2024-09-21T06:36:22.197 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67906.log 2024-09-21T06:36:22.198 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61479.log.gz 2024-09-21T06:36:22.199 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51884.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51884.log.gz 2024-09-21T06:36:22.199 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65361.log 2024-09-21T06:36:22.199 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58762.log 2024-09-21T06:36:22.201 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67906.log.gz 2024-09-21T06:36:22.201 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65361.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65361.log.gz 2024-09-21T06:36:22.201 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-09-21T06:36:22.202 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.58762.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49326.log 2024-09-21T06:36:22.202 INFO:teuthology.orchestra.run.smithi037.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.58762.log.gz 2024-09-21T06:36:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43964.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-09-21T06:36:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48152.log 2024-09-21T06:36:22.204 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69337.log 2024-09-21T06:36:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49326.log: /var/log/ceph/ceph-client.admin.48152.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49326.log.gz 2024-09-21T06:36:22.205 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48152.log.gz 2024-09-21T06:36:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-09-21T06:36:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59629.log 2024-09-21T06:36:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51004.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69337.log.gz 2024-09-21T06:36:22.207 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-09-21T06:36:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69535.log 2024-09-21T06:36:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2024-09-21T06:36:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59629.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59629.log.gz 2024-09-21T06:36:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67336.log 2024-09-21T06:36:22.209 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69535.log.gz 2024-09-21T06:36:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70135.log 2024-09-21T06:36:22.210 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2024-09-21T06:36:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52478.log 2024-09-21T06:36:22.211 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67336.log.gz 2024-09-21T06:36:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70135.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T06:36:22.213 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70135.log.gz 2024-09-21T06:36:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.52478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52478.log.gz 2024-09-21T06:36:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66581.log 2024-09-21T06:36:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2024-09-21T06:36:22.216 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66581.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66581.log.gz 2024-09-21T06:36:22.216 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70465.log 2024-09-21T06:36:22.217 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59288.log 2024-09-21T06:36:22.217 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2024-09-21T06:36:22.219 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.70465.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70465.log.gz 2024-09-21T06:36:22.219 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67672.log 2024-09-21T06:36:22.219 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59288.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59288.log.gz 2024-09-21T06:36:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T06:36:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2024-09-21T06:36:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67672.log.gz 2024-09-21T06:36:22.224 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.48744.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2024-09-21T06:36:22.225 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56041.log 2024-09-21T06:36:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66057.log 2024-09-21T06:36:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56041.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56041.log.gz 2024-09-21T06:36:22.229 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66057.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66057.log.gz 2024-09-21T06:36:22.229 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46764.log 2024-09-21T06:36:22.230 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61937.log 2024-09-21T06:36:22.232 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46764.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46764.log.gz 2024-09-21T06:36:22.233 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61937.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61937.log.gz 2024-09-21T06:36:22.234 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62888.log 2024-09-21T06:36:22.235 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36613.log 2024-09-21T06:36:22.236 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.62888.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.62888.log.gz 2024-09-21T06:36:22.238 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36613.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36613.log.gz 2024-09-21T06:36:22.238 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69835.log 2024-09-21T06:36:22.240 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51926.log 2024-09-21T06:36:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.69835.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69835.log.gz 2024-09-21T06:36:22.242 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51926.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51926.log.gz 2024-09-21T06:36:22.243 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49081.log 2024-09-21T06:36:22.244 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65700.log 2024-09-21T06:36:22.245 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49081.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49081.log.gz 2024-09-21T06:36:22.247 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.65700.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65700.log.gz 2024-09-21T06:36:22.247 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66099.log 2024-09-21T06:36:22.249 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52010.log 2024-09-21T06:36:22.250 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.66099.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66099.log.gz 2024-09-21T06:36:22.251 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52010.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52010.log.gz 2024-09-21T06:36:22.252 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47318.log 2024-09-21T06:36:22.253 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67840.log 2024-09-21T06:36:22.254 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47318.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47318.log.gz 2024-09-21T06:36:22.256 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.67840.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67840.log.gz 2024-09-21T06:36:22.257 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2024-09-21T06:36:22.258 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55818.log 2024-09-21T06:36:22.259 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43785.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2024-09-21T06:36:22.261 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55818.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55818.log.gz 2024-09-21T06:36:22.261 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72566.log 2024-09-21T06:36:22.264 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72566.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72566.log.gz 2024-09-21T06:36:22.264 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72650.log 2024-09-21T06:36:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57933.log 2024-09-21T06:36:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.72650.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.72650.log.gz 2024-09-21T06:36:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42027.log 2024-09-21T06:36:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61160.log 2024-09-21T06:36:22.270 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57933.log.gz 2024-09-21T06:36:22.271 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log 2024-09-21T06:36:22.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42027.log: /var/log/ceph/ceph-client.admin.61160.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42027.log.gz 2024-09-21T06:36:22.272 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61160.log.gz 2024-09-21T06:36:22.275 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50756.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-09-21T06:36:22.374 INFO:teuthology.orchestra.run.smithi037.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T06:36:22.749 INFO:teuthology.orchestra.run.smithi037.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T06:36:22.760 INFO:teuthology.orchestra.run.smithi037.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T06:36:22.923 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T06:36:22.925 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T06:36:22.942 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T06:36:23.014 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T06:36:23.090 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T06:36:23.093 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T06:36:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m1.514s 2024-09-21T06:36:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m6.034s 2024-09-21T06:36:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.640s 2024-09-21T06:36:23.260 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T06:36:23.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-21T06:36:23.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-21T06:36:23.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-21T06:36:23.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-21T06:36:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T06:36:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-21T06:36:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T06:36:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-21T06:36:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-21T06:36:23.557 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T06:36:23.559 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-21T06:36:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m1.985s 2024-09-21T06:36:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m6.079s 2024-09-21T06:36:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m1.903s 2024-09-21T06:36:23.560 INFO:tasks.ceph:Archiving logs... 2024-09-21T06:36:23.561 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi037/log 2024-09-21T06:36:23.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:36:25.601 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi150/log 2024-09-21T06:36:25.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:36:26.336 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:36:26.344 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:36:26.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:36:26.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:36:26.465 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:36:26.474 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:36:26.475 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:36:26.479 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:36:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:36:26.512 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:36:26.517 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:36:26.517 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-21T06:36:26.517 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -110us[ -105us] +/- 53ms 2024-09-21T06:36:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 +214us[ +214us] +/- 52ms 2024-09-21T06:36:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -1151ns[+3829ns] +/- 84ms 2024-09-21T06:36:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 -90us[ -52us] +/- 53ms 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 +183us[ +221us] +/- 52ms 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -41us[-3022ns] +/- 84ms 2024-09-21T06:36:26.519 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:36:26.520 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:36:26.528 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:36:26.529 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:36:26.588 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:36:26.725 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:36:26.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T06:36:26.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:26.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T06:36:26.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:26.963 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:36:26.975 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:36:26.989 INFO:teuthology.task.internal:Duration was 977.955196 seconds 2024-09-21T06:36:26.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:36:27.004 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:36:27.004 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:36:27.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:36:27.092 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:36:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:36:27.600 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:36:27.600 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-09-21T06:36:27.600 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:36:27.650 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T06:36:27.650 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:36:27.721 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:36:27.721 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:36:27.725 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:36:27.784 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:36:27.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:36:27.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:36:27.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:36:27.964 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:36:27.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:36:28.019 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:36:28.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:36:28.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:36:28.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:36:28.155 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-09-21T06:36:28.159 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-21T06:36:28.189 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:36:28.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:28.222 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:36:28.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:36:28.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:36:28.271 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:36:28.272 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi037 2024-09-21T06:36:28.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:36:28.336 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913546/remote/smithi150 2024-09-21T06:36:28.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:36:28.400 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:36:28.400 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:36:28.404 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:36:28.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:36:28.450 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:36:28.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:36:28.460 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:36:28.460 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:36:28.478 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:36:28.509 INFO:teuthology.orchestra.run.smithi037.stdout: 393246 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:36 /home/ubuntu/cephtest 2024-09-21T06:36:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:36 /home/ubuntu/cephtest 2024-09-21T06:36:28.516 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:36:28.526 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:36:28.551 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} duration: 977.9551959037781 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T06:36:28.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:36:28.642 INFO:teuthology.run:pass